Multiple Measures of Teacher and Principal Effectiveness

Teacher Evaluation Phase III February 27 28, 2013Rosanne Javorsky and Dr. Patricia DiRienzorosanne.javorsky@aiu3.netpatricia.dirienzo@aiu3.net

11Collaborative CodeSilence electronic devicesMinimize the use of those devices during presentations and activitiesObserve the time frames and signalsLimit sidebar conversationsKeep your materials organized22Project Development - GoalTo develop a teacher effectiveness model that: will reform the way we evaluate teachers

and identify critical components of teacher training and professional growth

33Project Development - Background$800,000 Gates Foundation grant to facilitate the development of statewide policy, tools and processes to evaluate teachers and principals in which student achievement is a significant factor affecting performance ratings

PDE is closely following the work of the Pittsburgh Public Schools PPS recipients of $40 million Gates Foundation grant that is more comprehensive in scope but similar in redesigning evaluation policy, tools and processes442010-2011 Phase IThe following were pilot sites for Phase I:

Allentown School DistrictCornell School DistrictMohawk School DistrictIU5 Northwest Tri-County Intermediate Unit552011-2012 Phase IIParticipation Levels

85 Districts11 CTCs11 IUs8 Charters662011-2012 Phase IIBeginning in January, 2012 One formal observation and one walk-through observation per participating teacher

Reporting the EvidenceEvidence collected and shared with Mathematica not PDE772012-2013 Phase IIIParticipants consist of:

Volunteer DistrictsRTTT DistrictsSIG Schools

882012-2013 Phase IIIVolunteer District Requirements:

Choose the number of schools, administrators and teachers to be involvedOne Observation and one walkthrough with each participating teacherCollect data using PDE tools and submit to Mathematica

992012-2013 Phase IIIRTTT District Requirements:

District-wide participation in both teacher and principal effectiveness 3 year phased-in commitment as outlined by the Performance Measures (see RTTT LEA Grant Guidelines, Participant Resources, Pg. 8) One observation and one walkthrough with each participating teacherCollect data using PDE tools and submit to Mathematica

10102012-2013 Phase IIISIG School Requirements:

School-wide participation in both teacher and principal effectivenessOne observation and one walkthrough with each participating teacherCollect data using PDE tools and submit to Mathematica

11111213

14Note the system This is what is being proposed for Pennsylvania. (Proposed legislation is moving through House and Senate Education Committees in Harrisburg.) What we will be working on during the Teacher Evaluation Pilot II is the observation/evidence component of the model presented. What the other component areas will look like will be described in the following slides. Nothing has been set in stone yet.

Correlating evidence of behaviors noted in previous slide with growth in student achievementAllows the observation of the behaviors above to be utilized in evaluating teachers for whom no formal, standardized assessment data may be available.

14Next Steps Value-Added Models (15%)Mathematica Policy Research ProposalDevelop value-added models (VAMs) for estimating the contribution of individual teachers and principals to growth in student achievement.Provide estimates based on VAMs for teacher and principal evaluation.Assess the strength of relationships between VAM-based and observation-based measures of performance, to inform decisions about which observation-based measures should be included and how to evaluate teachers in non-tested grades and subjects.Develop and calculate summary performance indicators across value-added measures and between value-added and other measures. Synthesize the findings in annual reports that document the models continual development over time.

15About Us: Collecting Information, Conducting Research, Informing Policy Mathematica Policy Research is dedicated to improving public well-being by bringing the highest standards of quality, objectivity, and excellence to bear on information collection and analysis for our partners and clients. The company has been at the forefront of assessing the effectiveness of policies and programs since 1968. Considered an architect of social policy research, Mathematica conducted the first social policy experiment in the United States, the New Jersey Negative Income Tax Experiment, to test ways of encouraging low-income individuals to work.Today, Mathematicas capabilities and reach have grown to respond to new public policy challenges and information needs. Our 800+ employees are national leaders in thorough and innovative program evaluation, objective policy research and interpretation, sophisticated survey design and data collection, and comprehensive performance measurement and data management. Mathematica works across the country and around the globe, serving federal agencies, state and local governments, foundations, universities, professional associations, and businesses. Our studies and analysis have yielded information to guide decisions in wide-ranging policy areas, from health, education, early childhood, and family support to nutrition, employment, disability, and international development. Click here to request a copy of the 40th anniversary report.

http://www.mathematica-mpr.com

15Next Steps Teacher Reports and Student-Teacher Linkages (15%)Teacher value-added reports for individual teachers by grade/subject/year using a robust statistical report.Teacher diagnostic reports for insight on effectiveness with students by achievement level and subgroup.Administrator summary reports for authorized users in a particular school or district.Drill down capacity to individual student-level projections based on classroom rosters.Teacher-level value-added reports require student-teacher linkages, which capture the instructor(s) responsible for a students learning in the tested grade/subject. It is critical that this linkage system provide accurate information that is validated by individual teachers.Web-based teacher reporting for PSSA grades 4-8Web-based teacher reporting for PSSA grades 4-8 and Keystone Exams1616Multiple Measures of Teacher Evaluation - Anticipated Evidence (50%)Principal/Supervisor classroom observations, including evidence that demonstrates behaviors associated with improving student achievement:

Domains (Danielsons Framework for Teaching)

Planning and preparationClassroom environmentInstructionProfessional responsibilities1717Teacher Evaluation Rubric4 Domains22 Components

4 Levels of Performance for each componentUnsatisfactoryProgressing/Needs ImprovementProficientDistinguished 1818Questions or Comments?19Up to three questions or concerns you hope will be addressed. share at your table value add report out.19Introduction to the DomainsOBJECTIVES: Participants will learn

How their thinking about good teaching compares to the framework we will use.That the Framework represents good common sense, and much that we already know, about teaching.The form and content of Domains 1, 2, 3 and 4.20Show: Slide with learning intentions. Let participants read silently (30 sec)

20Having an ImpactIf we want to impact student achievement and growth

Then we must impact teaching and learningAnd of course, we know what good teaching is2121Worksheet #1- Pg.3Participant Materials

Lets begin by collecting our thinking about good teaching.22

Distribute: Worksheet #1 or direct participants to their packet of participant materials- page 3

22Wisdom of Practice

What are the qualities of teaching most tightly tied to student learning? 23(@10 minutes, total)

The goal of this activity is to elicit what the audience knows about good teaching. In the subsequent activity, we will connect this schema to the research.

Give Directions: Tell participants to work alone and list on the left-hand side of the sheet the qualities of teaching that they know impact student learning.

Make sure participants work on the left-hand side of the paper.Allow 3 minutes: no talking or collaborating during this timeStop: After 3 minutes, ask participants to end their writing and to take a quick look at the list of an elbow partner, comparing for similarities and differences.Ask: After 2 -4 minutes, ask how many people had identical lists. (Same words, same order, etc.)When no one raises their hand, explain that this is why a common definition of practice is beneficial.

Next, ask the audience to raise their hand if their list was highly similar to their partners. Many participants will raise their hand, so point out that this wisdom of practice if reflected in the Framework for Teaching. 23The DomainsPlanning and Preparation

The Classroom Environment

Instruction

Professional Responsibilities24(5 minutes total)The goal of this activity is to connect participants qualities of good teaching to the domains of the Framework.

Give Directions: Show the slide The Domains and explain that these are the four domains of the Framework. Tell participants to put a #1 beside any item on their list that is related most to Domain 1, a #2 beside any items that most relate to domain 2 and so on for all four domains of the FrameworkRemind: Only one number per item on the list; decide on best fit.Allow @ 3 minutes.

24A Framework for Teaching:Components of Professional Practice25Domain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical SpaceThe goal of this activity is to expand participants knowledge of the Framework domains to include the 22 components they contain.

Show: Slide, The Framework for Teaching and invite participants to adjust the numbers on their list if they need to, based on this additional information.Allow 3 minutes

Say: How many had number four as the least frequently occurring item on the list? Why do you think this is so? (because most of us think of the items in Domain 4 as just things we have to do and not components that are connected to student learning.

Explain that the items on the right-hand side of the chart are the on-stage components of the Framework, those that we see when we observe in a classroom. On the other hand, Domains 1 and 4, on the left, are the off-stage domains of the Framework. They occur outside the act of teaching.

Tell participants that, just like any other content, the Framework has its own vocabulary. There are four domains, and 22 components. Ask them to try to use the correct terms when referring to these concepts going forward.

Note: if the Framework for Teaching is to be used for teacher evaluation, point out that the off-stage domains are important, and because they are behind the scenes, we have to make sure to collect information about them before and after the observed lesson. 25Worksheet #2-pg 4Participant Materials26Identifying the Domains

Extending our learning of the domains(10 minutes)

The goal of this activity is to conclude the initial learning about the Framework by having participants match scenarios to the correct Domain of the Framework. Now, invite participants to extend their learning about the Domains of the Framework, and direct them to Worksheet #2.

Instruct them to work as a table group and to place the number of the Domain in the space beside each statement, indicating which Domain is most closely related to that statement.

Only one domain number per items is permissible.

Allow @5 minutes, then process chorally: facilitator says letter of the item and the class calls out the letter. If there is dissonance, stop and discuss.

Answers to WORKSHEET #2:A. 2B. 3C. 1D. 4E. 1F. 2G. 4 H. 1I. 3J. 4K. 2

1A Framework for Teaching:Components of Professional PracticeDomain 4: Professional Responsibilitiesa. Reflecting on Teachingb. Maintaining Accurate Recordsc. Communicating with Familiesd. Participating in a Professional Communitye. Growing and Developing Professionallyf. Showing ProfessionalismDomain 3: Instructiona. Communicating with Studentsb. Using Questioning and Discussion Techniquesc. Engaging Students in Learningd. Using Assessment in Instructione. Demonstrating Flexibility and ResponsivenessDomain 1: Planning and Preparationa. Demonstrating Knowledge of Content and Pedagogyb. Demonstrating Knowledge of Studentsc. Setting Instructional Outcomesd. Demonstrating Knowledge of Resourcese. Designing Coherent Instructionf. Designing Student Assessments Domain 2: The Classroom Environmenta. Creating an Environment of Respect and Rapportb. Establishing a Culture for Learningc. Managing Classroom Proceduresd. Managing Student Behaviore. Organizing Physical Space27The goal of this slide is to illustrate that not all components of teaching can be seen during a lesson.

Show: The slide of the Framework for Teaching and remind participants that it contains four domains and 22 components, all of which describe the qualities of teaching that are tied to student learning.Remind: That the brown Domains, 1 and 4, are the off-stage domains. That is, they dont happen during teaching, they happen before or after.

The green domains are the on-stage domains that are directly experienced during the act of teaching. This is important because teaching is much more than what we can see during the teaching of the lesson.

It also suggests that teacher evaluation is much more than just watching a lesson; it includes gather information about a teachers planning, and also about the professional responsibilities that the teacher regularly performs. Say that we will see how this concept of on and off stage is played out in the Pennsylvania teacher evaluation process.

271Features of A Framework for Teaching28Generic: applies to all grade levels, content areas

Not a checklistEvidence based/reflective

Not prescriptive: tells the what of teaching, not how

Comprehensive: not just what we can see

Inclusive: Novice to Master teacher

Show: The slide containing the features of the Framework and explain them as follows: (2 3 minutes)

Generic: Its useful to a district to have ONE definition that applies to all teaching situations, all grade levels and all content areas.Not a checklist: Teaching is too complex to be captured in a checklist. Well see in a moment that the Framework has rubrics to describe it.Not prescriptive: Doesn't tell teachers how to do their work; rather, it describes WHAT qualities of teaching are most likely to produce student learningComprehensive: As we just saw, it contains both off-stage and on-stage aspects of teaching Inclusive: Applies to all teachers, from the newest to the most experienced.

Ask: Participants to turn and talk to an elbow partner for 2 minutes about why these qualities of the Framework are important and useful.

28

29Direct: Participants to their copies of the Framework rubrics, found in the Resource Section of your BinderShow: Slide containing the rubric for component 2a. Invite: Participants, with an elbow partner, to examine their rubrics for the components of the Framework and to discuss what they notice. (4 5 minutes), then, Point out: That the rubrics contain four Levels of Performance: UnsatisfactoryProgressing/Needs ImprovementProficientDistinguished

(Remind participants that this is another vocabulary word: Levels of Performance.)

Also point out that some of the components are shaded. These are the power components, which means that they are especially useful to us in growing our practices. While all components are important, it is sometimes useful to streamline our thinking about them, since 22 seems like a lot. We cant really separate the power components from the others realistically, but if we want to give any components special attention, then these ones would be good choices. Say: That the rubrics are useful in making the Framework more complete, and enable us all to acquire a common vision of teaching at its higher (and lower) levels. It is useful for many purposes to have these levels of performance: self-assessment, use with new teachers, evaluation, etc. It makes the process of observing teaching less subjective. Tell participants that well now do an activity that gives us an additional experience with the Framework.

291Lets ReviewFind the handout entitled Framework AssessmentComplete on your ownReview with at your table.

30Begin by telling participants that many administrators feel that they know Danielsons Framework because they have been using it through the PDE-426, 427, and 428.

But when using the Framework and rubric as part of observations, you really need to have a much more in-depth understanding of the 4 domains, the 22 components, and the elements. We have a way for you to assess your own background knowledge.

Have the participants complete the assessment on their own, and then have a table discussion about their answers. Direct them to the rubric to assist when they are having the table discussions.30A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space31Trainers note: participants should be seated in tables of 5 or more participants.

Introduce the activity:

Direct participants attention to the blue domain, Instruction.

Say: Referring to the slide of the framework, explain that today well take a deep dive into one of the domains of the Framework, Domain 3. We chose this on-stage domain because it is what most people think of when they think about teaching.

31132Domain 3 Components3a Communicating with Students

3bUsing Questioning and Discussion Techniques

3cEngaging Students in Learning

3dUsing Assessment During Instruction

3eDemonstrating Flexibility and Responsiveness32Direct participants to their rubric of Domain 3 and give them a moment or two to skim them.

Explain a few points:3a has a lot to do with explaining directions and procedures3b refers to a technique used to advance learning3c is the most important component of the entire Framework, the purpose for which all other components exist3d is about assessing students during the act of teaching so that instruction can be modified as it is delivered, if necessary3e has to do with modifying teaching based on either an appropriate teachable moment, or based on evidence that instruction is not working in some way.

Direct participants to Worksheet #3 in their Participant Materials, found on page 5. 32An In-Depth Look at Domain 3In your supply box, there are little cards with letters on themDistribute one to each personTake Worksheet 3a (p. 5) out of your binder along with the rubricFind the table with your letter and introduce yourself to the others in this new group.3333An In-Depth Look at Domain 3Each group will be working on one of the components within Domain 3:

Groups A & F Component 3aGroups B & G Component 3bGroups C & H Component 3cGroups D & I Component 3dGroups E & J Component 3e3434Worksheet #3a Pg. 5 Participant Materials

35Create a specific example of your assigned component: Ms. T says to Joey, You dummy!

Do not restate the rubric: Ms. T is rude to a student.The goal of this activity is for participants to be able to translate the words in the rubric to a specific example that illustrates the level of performance.

Ask participants to letter off around their table, a through e, representing the 5 components of Domain 3. Once each person has been assigned a letter (component) of Domain 3, ask them to reassemble into component-alike groups: all letter as together, bs together, and so on. ** Modification for large group Pick a letter from the supply box on your table. The following letters will work on the following components: 3a A & F; 3b B & G; 3c C & H; 3d D & I; 3e E & I. Find the table with your letter on it.

Once participants are settled in their expert groups, explain (in @ 3 minutes) the process of the activity, in which the goal of the group is to create four very specific examples of their assigned component at each of the four levels of performance. The examples should be created collaboratively and written on the worksheet by all members of the group, not just one recorder.Give a couple more examples in addition to the one on the slide, of specific vs. restating:Specific: 7 students raised their hands to ask for clarification on the directionsRestating: The teachers directions were vague.

Specific: The teacher stopped the lesson, said she could tell students were confused, and regrouped them according to their understandingRestating: The teacher made a smooth, major adjustment to the lesson.Allow: 10 - 15 minutes for small groups to finish the assignment. Note: you will need to check each groups work early to make sure they are creating specific for instances, not restating the rubric and that everyone is writing the generated examples on their own papers.

35Worksheet #3b Pg. 6Participant MaterialsGeneralizing about Levels of Performance 36What do all the examples of unsatisfactory have in common?

What do all the needs improvement/progressing examples have in common?

Proficient?

Distinguished?The goal of this activity is for participants to generalize the characteristics of the four levels of performance of the Framework, from the examples they hear.

Once participants have finished in their expert groups, regroup participants back in their home groups Once they are resettled, explain the activity: Invite participants to go around their new group and have each person read the unsatisfactory example created for their component. While participants are sharing, the rest of the participants in the group should use worksheet 3b, found on page 5 of Participant Materials, to write down the characteristics of the unsatisfactory level that they are hearing as their colleagues share. They are not to write each example; rather, they are to write the characteristics of the level of performance they are hearing.

Explain to participants that they are to repeat this process for the basic, proficient and distinguished levels of performance, going around their group and having each member share their example for that level of performance, while the other members write the characteristics of that level on worksheet 3b.

Allow 12 15 minutes for this activity.

While participants are working, write Unsatisfactory, Basic, Proficient, and Distinguished at the top of a piece of chart paper and display the slide.

36Conclusions: Levels of Performance37Unsatisfactory: Potential for harm

Progressing/Needs Improvement: Inconsistent, novice

Proficient: Consistent, competent

Distinguished: Unusually excellent, no one lives here permanently in all components

Conclude: by summarizing each level of performance as follows and inviting participants to add to their charts if necessary: :

Unsatisfactory is dangerous. Harm to students can be done at this level: physical, social, emotional or cognitive harm are possible at this level of teaching performance.

Basic is the novice level. Its hallmark is inconsistency: sometimes the teacher gets it right, and sometimes not. Even experienced teachers can have basic characteristics in their practice for a time if their teaching assignment changes significantly.

Proficient is the level at which we expect most of our experienced teachers to function. This is good, solid, consistent teaching.

Distinguished is the exceptional level of practice. No one lives at this level, nor has every characteristic of this level in all components, because that would mean perfection in teaching, and no teacher is perfect. Distinguished is not just good, or even excellent, teaching. It is teaching that sets itself apart as unusual in its outstanding level of excellence

We can all grow and improve, so while we have excellence in our practices sometimes, it is not a place we live all the time in all components. 37Components of Domain 3: Engagement3a: Communicating with Student

3b: Using Questioning and Discussion Techniques

3c: Engaging Students in Learning

3d: Using Assessment in Instruction

3e: Demonstrating Flexibility and Responsiveness

38Show: The slide, Components of Domain Three and say:Engaging Students in Learning, is the most important component in the entire Framework.

Ask participants to chat at their table groups about the difference between time-on-task and engagement. Allow them @ 2 minutes to do so.

Then say:

Time-on-task means doing what the teacher asks you to do, regardless of whether the task is challenging or not. . Engagement , on the other hand, is brain sweat, or real, challenging thinking. Many times students are on-task, that is, they are doing what the teacher asked them to, but that task does not require real, challenging thinking by each student. Therefore, the students are on-task, but not engaged, which often explains why teachers teach and students dont learn. Engagement, then, is different from time-on-task, and is about what the brain is doing and whether it constitutes rigor.

Engagement is harder to see than time-on-task. Engagement can be thought of as difficult, in a good way. Students would not describe the assignment as easy, or a piece of cake, when engagement is present. Another way to think if it might be mental tippy toes.

381A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space39Display slide: The Framework for Teaching, Components of Professional Practice, and remind participants that previously we have overviewed the four domains, and have studied Domain 3, an on-stage domain, closely. In this activity, we will learn about an off-stage domain, Domain 1, Planning and Preparation. Point out that while Domains 2 and 3 are observable in the classroom (on stage), Domains 1 and 4 require other ways of indicating a teachers skill (off stage). Artifacts, professional conversation, or portfolios may be used.

Be careful in your language: we cant say that no components of Domain 1 or 4 could EVER be seen during a lesson, but rather, that these components are more likely to be seen in other ways.

391Domain 1: Planning and Preparation401a: Demonstrating Knowledge of Content and Pedagogy

1b: Demonstrating Knowledge of Students

1c: Setting Instructional Outcomes

1d: Demonstrating Knowledge of Resources

1e: Designing Coherent Instruction

1f: Designing Student AssessmentsDisplay slide: Domain 1 Planning and Preparation and tell participants that Domain 1 contains the components of a good lesson plan, and briefly discuss the different components. These are the items that a teacher thinks about when planning a lesson. Mention that certainly teachers know more about content and pedagogy (D1a) than we can see in a given lesson. Likewise for D1b, D1d and D1f. Therefore, we like to talk to teachers from time to time, including before an announced observation, to dig a little deeper and to record some of what the teacher tells us. Ask participants to look over the levels of performance rubric for Domain 1 and ask them to read the distinguished level for each component.

Have them note that at the distinguished level of 1f, students help to design some of their own assessments, which helps to explain why this level is not typical, but is exceptional.

Allow 3 minutes.

40Component 1a: Demonstrating Knowledge of Content and PedagogyTeacher wrote a scholarly articleLesson plans/structure/content/relevanceTeacher explanation of probable Students misconceptionsTeachers answers to student questions during classTeacher presented a workshop to facultyTeacher explains the structure of discipline prior to lessonTeacher tells observer how this lesson fits into the larger unitTeacher adjusts the lesson midstream based on Students misconceptionsTeacher poses different levels of content questions during the lessonTeacher states how this lesson connects to content standards41The goal of this slide is to assist participants to begin to identify the kinds of facts teachers share that can be evidence for Domain1.

Tell participants that Domain 1 is about the back story of teaching. Its about planning, and all the things the teacher knows, but cant show, during the act of instruction. Teachers make hundreds of decisions for each lesson that they plan. We dont expect them to write all this information, but when we talk to them from time to time, we can collect some of it, by writing down what the teacher tells us.

Invite participants to think about the on-versus-off stage concept. Have them read SILENTLY the above 10 statements, and think about which could be seen during teaching (on-stage) and which could not.

Allow a minute or two, but enforce silence while everyone reads and thinks. When everyone is ready, show the next slide.

Participants have this slide as a handout and can circle their answers on the paper.1Component 1a: Demonstrating Knowledge of Content and PedagogyTeacher wrote a scholarly articleLesson plans/structure/content/relevanceTeacher explanation of probable Students misconceptionsTeachers answers to student questions during classTeacher presented a workshop to facultyTeacher explains the structure of discipline prior to lessonTeacher tells observer how this lesson fits into the larger unitTeacher adjusts the lesson midstream based on Students misconceptionsTeacher poses different levels of content questions during the lessonTeacher states how this lesson connects to content standards42When you show this slide, mention that the red-colored statements are related to what would happen during teaching, so they are Domain 3 items, not Domain 1 items.

On the other hand, the rest of the items are things that would NOT occur during the teaching of the lesson, but could happen before or after. They give us more insight into what a teacher knows and thinks about when planning a lesson.

Tell the audience that our next activity will give us the opportunity to review the kinds of questions we might ask a teacher to learn more about Domain 1 in their practice. 1Worksheet #4 Pg. 7Participant Materials43 Read the Domain 1 questions.

Why are they important?The goal of this activity is for participants to learn the kinds of information about the components of Domain 1 that a teacher possesses, but that we dont always see easily.

A second goal is for the audience to understand that unless teachers are given an opportunity to respond to questions about the hidden parts of their practice, we cannot know about them.

Direct participants to the worksheet, found on page 7 of Participant Materials.

Explain to participants that this form contains Domains 1 and 4, and questions to elicit information from the teacher about them.

Review the form and the questions it contains. Say that when we ask these types of questions, we get at some of the teachers knowledge that might not be visible during the teaching of the lesson.

Allow a few minutes for people to discuss the questions, and some of the kinds of answers teachers might give. Tell participants to jot dot (bullet, briefly) some of the kinds of information a teacher might provide in response to the prompts for Domain 1 ONLY. (We will study Domain 4 later)Allow 10 minutes.

Walk around as participants work to see if they are providing the kinds of background info about that component that might not be seen during a lesson. For example, for component 1b, teachers know more about students than we might see during one lesson. What kinds of things are these? How does the teacher get information about the students? How is this used in planning lessons?Activity: Collecting responses

If time permits, invite participants to get up, move about the room, and collect some teacher responses for one or more of the components of Domain 1 from someone in the room with whom they dont usually work.

Allow 5 minutes.

43Lesson Plan Data Collection ToolPage 7- Worksheet #4

Practice Evidence Collection

Write what is said - not what you think about it/your interpretation of it.44

The purpose of this slide is to let participants know that observers can record teachers responses to the questions on page 7 of participant materials

Tell participants that we will be using this tool for collecting evidence of Domain 1 to collect some facts about this shy domain from a pre-observation conference. Explain that the pre-observation conferences purpose is to elicit some of this otherwise invisible information.

Say that we will now watch a short video clip of a pre-observation conference for a middle school math teacher. In it, we should hear some facts from him about his students, his lesson, the unit in which it is situated, etc. We will write down some of this information as we hear it, in the correct component box on pg. 6.

Tell participants that they are to write what they hear, not editorial commentary, advice or opinion.

Now, show the pre-observation conference clip of High School Social Studies Pre-Observation video from the ASCD/Danielson/Framework video. (Be sure to show ONLY the pre-observation conference, not the lesson) and ask participants to write as they watch. (You should collect evidence also.)

At the conclusion of the pre-observation conference, invite participant to share the evidence they collected and to write down any evidence they hear from others that they did not hear or have time to collect.

Close this activity by reminding that Domain 1 is off stage and that when a teacher talks about it and we capture some of that information, it becomes more visible. This is the reason for the pre-observation conference.

You will have other opportunites to practice collecting evidenceYou cant hear cant 1A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space45Display slide of Components of the Framework. Remind participants that they have already studied two domains of A Framework for Teaching, Domains 3 and 1 respectively. Explain that we are now ready to explore the second on-stage domain of the framework, Domain 2, and that we will do so using a graphic organizer structure and by answering a focus question in a modified jigsaw fashion.

451Domain 2: The Classroom Environment2a: Creating an Environment of Respect and Rapport

2b: Establishing a Culture for Learning

2c: Managing Classroom Procedures

2d: Managing Student Behavior

2e: Organizing the Physical Space46Discuss each component briefly, mentioning that Domain 2 is sometimes thought of as the affective domain of the framework, because the level of the teachers performance in components 2a, Creating an Environment of Respect and Rapport, and 2d, Managing Student Behavior, contribute significantly to how students feel about school and about themselves. Component 2b: Establishing a Culture for Learning, also speaks to the teachers beliefs about students abilities to learn; therefore, Domain 2 contributes significantly to students emotions toward school and learning.

Other points to be made about the components:2a is not only about how the teacher treats the students and vise versa, but also about how students treat each other.Another way to think about 2b is that its all about rigor and setting high expectations2c has to do with extracting maximum time for teaching/learning, by managing procedures seamlessly2d is done best when its invisible2e is related to making the physical space match the learning goals

46Worksheet #5 Pg. 8Participant MaterialsBrowse Domain 2 of your Rubric

Reflect and answer questions on Worksheet #5Independently

Table Share47Browse Domain 2 of your RubricReflect and answer questions on Worksheet #5IndependentlyTable Share

47A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesParticipating in a Professional CommunityGrowing and Developing ProfessionallyShowing ProfessionalismDomain 3: InstructionCommunicating with StudentsUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSetting Instructional OutcomesDemonstrating Knowledge of ResourcesDesigning Coherent InstructionDesigning Student Assessments Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical Space48Explain to participants that the red-colored components are those we have already studied. We are now ready to investigate Domain 4, its relationship to student learning and its importance to novices and to experienced teachers. Remind participants that Domain 4 is the other off-stage domain of the framework. Its safe to say that Domain 4 is virtually never visible during the act of teaching, yet each component is connected to student learning in specific ways.

48149 Domain 4: Professional Responsibilities 4a: Reflecting on Teaching4b: Maintaining Accurate Records4c: Communicating with Families4d: Participating in a Professional Community4e: Growing and Developing Professionally4f: Showing ProfessionalismBriefly discuss each of the components of Domain 4, making the following points:

The role of reflection in adult learning is well-documented. However, many teachers do not reflect in the ways that produce learning.Recordkeeping is designed to inform teachers about trends and patterns in learning, both individual and group, so that instruction can be modified appropriately.Notification is not communication as it is intended in this component.Professional learning communities (PLCs) are an important support for teacher learning4f primarily focuses on advocacy for students who are traditionally underserved49Worksheet #4 Pg. 7Participant Materials50Skim the rubrics in Domain 4.Have a table conversation about HOW these components might impact student learning.Go to an expert group (get a numbered card from your supply box) and move to the table for that group.Develop a Distinguished response for your component.The goal of this activity is for participants to learn the evidence of Domain 4 in a teachers practice.

Ask participants to locate and read their rubrics for Domain 4. Allow about 5 minutes for this study.

Then invite participants to turn and talk to someone near to them about how each component of Domain 4 might impact student learning. Allow 3 minutes, or more, if necessary.

Then direct participants to their Participant Materials, page 7. We have worked on the left-hand side of this page, Domain 1, earlier. Now we will revisit it for Domain 4.

Remind participants that Domain 4 is the other off-stage domain. We would not see these components during teaching, but questions posed to the teacher can elicit information about his/her practice in this Domain.

After getting into expert groups a through f - participants to read the prompt associated with each D4 component of their expert group and write a possible excellent answer to each work as a group. They should reference their rubrics for Domain 4 at the proficient or distinguished level as they do so. ** Modification for large group have participants take a number from their supply box. The following numbers will work on the following components 4a 1 & 7; 4b 2 & 8; 4c 3 & 9; 4d 4 & 10; 4e 5 & 11; 4f 6 & 12.

Allow @ 12 minutes. Select some non-volunteers to share one or more of their responses. Make clear, in closing, that teachers can answer these questions during pre- or post-teaching conferences, to provide more evidence about their practice. 50Domain 4 Expert GroupsGroups 1 & 7 Component 4aGroups 2 & 8 Component 4bGroups 3 & 9 Component 4cGroups 4 & 10 Component 4dGroups 5 & 11 Component 4eGroups 6 & 12 Component 4f

5151Uses of A Framework for Teaching52Self-AssessmentReflectionPeer CoachingTeacher EvaluationMentoring and InductionProfessional Growth PlansNow display the slide, Uses of the Framework and remind participants that the Framework articulates what good teaching looks like and what it doesnt look like. It provides us with a common language, a roadmap for the terrain, so to speak. Use of the Framework for Teaching for these various purposes tends to elevate the work .52Benefits of Using a Framework for Teaching53Common languageSimilar vision for good teaching and how it can be improvedGreater validity and reliability in the teacher evaluation processChanges in novice thinkingOpportunities for collaborationInvite participants to consider the slide for 30 seconds or so without discussion. Then invite volunteers to share any observations they have about the benefits of using the Framework within teaching and school practice. 53Doing Teacher Evaluation Right 545 Rules/Best Practices

Evidence

5455Why Evaluate Teaching?Quality Assurance

Professional LearningTell participants that we will be exploring how to use best practices for teacher evaluation, using the PA Teacher evaluation process.

Remind them that there are two purposes for teacher evaluation as state on the slide, but that mostly we focus only on the first one: getting a grade

Best practices research focuses on the second one, professional learning, because most teachers practices are good. Still we all need to grow, since all students can always learn more, so the practices we will experiences are not focused on gotcha, but on how we acknowledge strengths and continue to grow. 5556Doing Teacher Evaluation RightOBJECTIVES: Participants will learn

The role of the Framework in teacher assessment

Best practices in teacher evaluation

How to conduct teacher evaluation to accommodate quality assurance and professional learning, too

How to differentiate for novices, experienced teachers, and at-risk teachers

How to create buy-in for all stakeholders.

Invite the audience to read the goals silently and to mentally identify the one(s) that are most meaningful for them.

In your packet is an article entitled the widget effect that you might wish to use with your teachers. When providing professional development or sharing it highlights the major reasons to improve our evaluation processes and provides suggestions on how to do so. You can find the article in the resource section of your binder.5657The Widget EffectCount off by a, b, c, d at your table

Read the following sections: a p9 and pp 10-14 b p9 and pp 15-18 c p9 and pp 19-23 d p9 and pp 26-30

Then identify the 3 most important facts from your section The goal for this activity is for participants to learn about the flaws in traditional teacher evaluation processes.

Arrange participants in groups of four or more participants, and ask them to count off by a, b, c, and d. As each person to then read the pages noted on the slide. Encourage them to highlight or underline the most important components in their sections, and then select the three MOST important.

Participants are to work alone. Enforce quiet so that all can read and study.

Allow 15 minutes or so for the reading.5758The Widget Effect Share and LearnAt your table, have each person share the three most important ideas culled from the reading.

Mark your own copy accordingly.

Discuss your thinking.Once everyone has finished reading, invite the participants at their tables to share their three most important facts. Ask the other table members to mark their articles accordingly.

After completing this, as them to discuss their thinking about the article and why it is a good decision for Pennsylvania to move toward a more rigorous, research-based process for teacher evaluation.

Allow 15 minutes for this activity.5859Card Sort p10Participant MaterialsRead your card

Using a Post-It note, identify:DomainComponentElement

Share at your table as instructed; come to consensus

NOTE: Prior to this activity, you will need to distribute the colored Card Sort cards, so that there is one card for each person and that each table group has cards of all four colors.

The purpose of this activity is for participants to revisit the Framework for Teaching and to connect an individual scenario to one domain/component/element and level of performance.

Ask them to work alone for a couple minutes to read their scenario and to assign it to a domain/component/element of the Framework (using p10 of the Participant Materials), and to write this on their post-it note and affix it to their card..

While there may be more than one possible connection, they are to select the best one. Do not let them confer at this time, but rather, work alone. Allow @ 3 5 minutes.

At the end of this time, Tell participants that the color of the cards is significant: Yellow=Domain 1, Blue=Domain 2, Pink=Domain 3, and Green=Domain4. Invite correction on their post-its at this time.

Next, ask participants to share their scenarios with their table group, and develop consensus about the best answer. 5960RewriteSelect one scenario at your table

Determine tentative Level of Performance

Rewrite at higher and lower levels using rubric characteristicsNow, tell participants to select one of the cards, and to think about what level of performance - Unsatisfactory, Basic, Proficient or Distinguished - that scenario would represent if it were typical of that teachers overall practice.

Allow them to have a table conversation about ONE scenario, using their rubrics, to begin to circle around the appropriate level of performance, for about 3 -4 minutes.

Now, tell them that if they found some language in, say, the basic level and some language in the proficient level, that both apply to the scenario, they should feel free to use both: TEACHING BEHAVIOR DOESNT JUST LIVE IN ONE LEVEL OF PERFORMANCE, it is most often a mixture.

Ask them to write on a second Post-It the phrases from one or more levels of performance that match with the scenario they chose, and to affix this to the Post-it.

60615 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processThe role of teacher learningTransparencyInvite participants to notice the five best practices/rules for doing teacher evaluation right.

Explain that we will we working on each of these practices during the rest of our time together. 6162Overarching QuestionsWho does the thinking?

Who does the learning and growing? Tell participants that, in the classroom, we know that the work of learning needs to be done by the students themselves, not by the teacher. That is, we look to see who is doing the real thinking in classrooms, and we know that when students do it, real learning happens.

In teacher evaluation historically, the position of the teacher within the evaluation process has NOT been that of learner: Teachers have tended to be passive: the observer collects the information, analyzes it, and presents the conclusions to the teacher who primarily listens.

If teachers are to grow their already competent practices, then THEY need to do some of that work: collecting, analyzing and concluding. This changed role of the teacher makes some of the traditional teacher evaluation practices look quite different, which we will see as we continue our learning. 6263Rule # 1: Defensible Definition of TeachingStart with a defensible definition of good teaching that is studied, and understood, by all stakeholders.Tell participants that we spent the earlier part of this training experiencing Pennsylvanias definition of good teaching, which is the Framework for Teaching, otherwise known as the research-based standards for teaching.

Now we will experience a quick review of that definition of practice. 6364A Framework for Teaching:Components of Professional PracticeDomain 4: Professional ResponsibilitiesReflecting on TeachingMaintaining Accurate RecordsCommunicating with FamiliesContributing to the School and DistrictGrowing and Developing ProfessionallyShowing Professionalism

Domain 3: InstructionCommunicating Clearly and AccuratelyUsing Questioning and Discussion TechniquesEngaging Students in LearningUsing Assessment in InstructionDemonstrating Flexibility and ResponsivenessDomain 1: Planning and PreparationDemonstrating Knowledge of Content and PedagogyDemonstrating Knowledge of StudentsSelecting Instruction GoalsDemonstrating Knowledge of ResourcesDesigning Coherent InstructionAssessing Student Learning Domain 2: The Classroom EnvironmentCreating an Environment of Respect and RapportEstablishing a Culture for LearningManaging Classroom ProceduresManaging Student BehaviorOrganizing Physical SpaceRemind participants that we looked at the four domains, and we learned that two of the four domains are off-stage and that, to learn more about these in a teachers practice, we can talk with the teacher, pose questions, examine artifacts, and so on.

6465Rule # 2:Differentiation of Evaluative Process

Differentiate the processes of evaluation for:

novices experienced teachers teachers at riskExplain that teachers are different the same ways that students are different: skill level and experience. Good teacher evaluation systems typically honor these differences by providing closer supervision and support to those teachers who need it and allow greater variation and independence to teachers whose practices show evidence of competence. 65

Differentiated Evaluation SampleNovice/Untenured

Very close observation and assessmentExperienced/Tenured

Presumption of professionalismAt-Risk

Not punitiveFormal and informal observation of teaching is key + teacher interviews + artifacts Structured process 1/3yr. Other years: informals + teacher interviews+ professional goal-settingIntensive, extensive team-based support based on persistent unsatisfactory performance in one or more components2 4 formal times per year; multiple informal observationsProfessional Goal-Setting: Choose from a list of rigorous, approved activitiesClear goals, outcomes, evidence and timelines anchorNo self-directed activitiesActivities produce evidence which is then evaluated Designed for the teacher who can, and wishes, to improve66Briefly mention that that most typical forms of differentiation include frequent novice observations (2 4/year) and experienced teachers being evaluation formally less frequently with alternate evaluations taking the form of Professional Growth Plans.

Say that Pennsylvania is in Phase I of its new evaluation program, so we dont yet know what differentiation will look like in its final form, but that novice must be evaluated more frequently than experienced teachers. This makes good sense, since we must make consequential decisions about these professions that will have a last impact upon school culture.

Make clear to participants that we have some suggestions/ideas for differentiation especially for experienced teachers and using the framework with the intent to reduce the workload for evaluators and increase the learning for teachers but we will hold that for follow up as this is for pilot of the process and we dont know what PAs system will look like at the moment.6667Overarching QuestionsWho does the thinking?

Who does the learning and growing? Tell participants that, in the classroom, we know that the work of learning needs to be done by the students themselves, not by the teacher. That is, we look to see who is doing the real thinking in classrooms, and we know that when students do it, real learning happens.

In teacher evaluation historically, the position of the teacher within the evaluation process has NOT been that of learner: Teachers have tended to be passive: the observer collects the information, analyzes it, and presents the conclusions to the teacher who primarily listens.

If teachers are to grow their already competent practices, then THEY need to do some of that work: collecting, analyzing and concluding. This changed role of the teacher makes some of the traditional teacher evaluation practices look quite different, which we will see as we continue our learning. 67685 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparencyShow this slide and tell participants that we have now discussed two of the three best practices for doing teacher evaluation right. Next we will learn about evidence, a keystone of the process. 6869Rule # 3:Evidence Driven ProcessLet evidence-not opinion-anchor the processExplain that typical evaluation systems involve observers watching the lesson and then writing down what they thought about it. They write advice, opinion, suggestions, likes, dislikes, etc.

Say that this is not the practice in an evidence-driven system for two reasons:Evidence, or facts, form the foundation of good decision-making. We always need to collect the relevant facts first, then make decisions.

In a growth-focused model, the evaluator AND the teacher evaluate the evidence separately, and then compare their thinking. In this way, learning is done by the learner, the teacher.

69Evidence or Opinion?The teacher has a warm relationship with the students.The teacher said that the South should have won the Civil War.The table groups were arranged in 2 x 2 pods. The materials and supplies were organized well.Wait time was insufficient for student thinking.The teacher stated that students have learned to add 2-digit numbers in preparation for todays lesson. 6 students, questioned randomly, did not know the days learning goals.70Ask participants to read the statements on the slide SILENTLY and decide which are evidence and which are opinion.

Participants have this slide as a handout, so they can circle their responses before going over the correct answers.

After a minute or so, provide the correct answers:OEEOOEE

If any questions arise, explain, for #2, that it is a fact that the teacher said the words in #2. this makes it evidence.

Say further that words like warm relationship , organized well, or insufficient thinking are opinions, no descriptions of what was seen and heard. So, what made the observer THINK the relationship was warm? What was ssen and heard? What makes a room well-organized? What was seen and heard? 70Evidence or Opinion?The teacher has a warm relationship with the students.The teacher said that the South should have won the Civil War.The table groups were arranged in 2 x 2 pods. The materials and supplies were organized well.Wait time was insufficient for student thinking.The teacher stated that students have learned to add 2-digit numbers in preparation for todays lesson. 6 students, questioned randomly, did not know the days learning goals.71Ask participants to read the statements on the slide SILENTLY and decide which are evidence and which are opinion.

After a minute or so, provide the correct answers:OEEOOEE

If any questions arise, explain, for #2, that it is a fact that the teacher said the words in #2. this makes it evidence.

Say further that words like warm relationship , organized well, or insufficient thinking are opinions, no descriptions of what was seen and heard. So, what made the observer THINK the relationship was warm? What was ssen and heard? What makes a room well-organized? What was seen and heard? 71

Worksheet #6 Pg. 11Participant Materials72EVIDENCEis a factual reporting of events, may include teacher/student actions and behaviors,may include artifacts prepared by the teacher, students or others,avoids personal opinion or biases,and is selected using professional judgment by the observer and/or the teacher.Allow participants a moment to read this slide, and then say that we will be doing a series of activities now to teach us about evidence and how to collect it.

Direct participants into their Participant Materials, Worksheet #6 -, Evidence vs. Opinion. And invite them to complete it as a table group. (If time is a factor, you may choose divide the assignment.)

Explain that the items in the left column are types of statements an observer might write based upon an observation. Some of them are evidence and some of them are not. They should decide if each statement is evidence or opinion and should write E or O in the column based on their decision.

If the statement is opinion, they should REWRITE the statements to be a statement of evidence.

Finally, for each statement, they should decide which domain and component of the Framework to which each statement refers and write that domain/component (d/c) in the final column.

Allow @ 10 minutes, more if necessary, then review the answers as a whole group.

(Note: there is no absolute correct answer to the domain and component connections for each item, but you should listen carefully for rationale and push for better thinking if an answer seems weak.

Tell participants if they need it when working with others, there is additional practice information found on p4 of Participant Resources.7273Observation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 1, 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment RubricNow tell participants that we will consider evidence in light of the evaluation process.

Remind them that there are two off-stage and two on-stage domains, so we have to use a process that elicits evidence of all these components, not just domains 2 and 3 , which is typically the custom.

Direct participants now to the evidence collection form for Domains 1 and 4 found on p7 of the Participant Materials. Explain that this is a tool to use during the pre-and post-teaching conferences to collect the off-stage evidence as part of an announced observation. Say that good evaluation systems consist of a mixture of announced and unannounced observations, and that for the announced ones, collecting evidence about Domains 1 and 4 is an essential part.

Ask participants to read the questions associated with Domain 1 and to discuss why these questions are helpful. Make it clear that this is NOT a daily lesson plan document, it is intended for an announced observation, to elicit additional information about planning.

Note further that the most efficient process is for the teacher to provide this completed document, (all D4 components need not be answered) to the principal PRIOR to the pre-observation conference and then can be added-to during the pre-observation conference.

7374

Worksheet #7 Pg. 13Participant MaterialsWatch the pre-observation conferenceCollect evidence on your Lesson Plan formDocument evidence only no opinions, please!Prepare to show the video by asking participants to locate their lesson plan form on p13 of Participant Materials. Tell them they will write down what they hear the teacher saying about his upcoming lesson.

Show the video clip of the ASCD Framework video for the 6th grade middle school math lesson, 2 minutes in length.

Afterward, ask participants to compare the evidence they collected. Allow about 3 4 minutes.

Invite them next to discuss what questions they would pose to the teacher if they could speak with him, and point out that were they able to do so, they would record this evidence on this form as well.

Make sure that participants recorded under component 4e the information that the teacher offered about his professional development with the Smartboard.

Say that it is important to use the pre-observation conference to elicit as much information about the upcoming lesson as possible.

7475Daily Lesson Plans1c: What will students learn?

1e: How will I teach it to them?

1f: How will I measure which students learned it? Say that for daily lesson planning, answering the above three questions is best.

Mention that 1c is not about what students will DO, its about what they will learn. Also, direct participants to the resources package, :

Need a resource for working with teachers about Formative Assessment Go to Resources PG 5Formative Assessment Tools, and ask them to skim it. Point out that this is a bank of formative assessments that teachers can use.

75EvidenceObservation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 1, 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment Rubric76Next, direct attention to the evidence collection process once more and point out that after the pre-observation conference, the observation is conducted and evidence is collected during it.. Ask them to review the evidence collection document found on page 18 of their materials.

Say that this is the tool, (or the PA counterpart) that should be used to collect evidence DURING the observation.

We will be asked to stretch ourselves and learn to collect evidence in real time and not to recopy it. Mention briefly that after step 2, this evidence is PROVIDED TO THE TEACHER. It is just facts, not opinion or judgment, and this step is crucial to the process.

Then the teacher uses the evidence to conduct a self-assessment on the rubrics, prior to the post-teaching conference.

We will be learning more about this process now as we practice collecting evidence.

Direct participants to page 14 Worksheet #8 of their participant materials and reinforce that this is the evidence collection tool for the on-stage aspect of this process, the classroom observation. Point out that it contains spaces to write evidence for Domains 2 and 3 and that it is important to write the evidence we see during the observation ON THE DOCUMENT, not someplace else. This practice improves reliability, so ask participants to put away their other materials and to prepare to write on page 14 as we watch the video of the lesson for the middle school math teacher.

7677Types of Observation EvidenceVerbatim scripting of teacher or student comments: Could one person from each table collect materials?

Descriptions of observed teacher or student behavior:The teacher stands by the door, greeting students as they enter.

Numeric information about time, student participation, resource use, etc.:Three students of the eighteen offer nearly all of the comments during discussion.

An observed aspect of the environment:The assignment is on the board for students to do while roll is taken. Say that we are moving now toward collecting evidence7778

Worksheet #8 Pg.14Participant MaterialsWatch the lessonCollect evidence of what you see and hearIf you arent sure where to write the evidence, just write itThis is practice; relaxAs you prepare to watch the video of the middle school math lesson for which you collected evidence of planning, ask participants to skim the components of D2 and D3, focusing on the priority components which are shaded.

Say that they should collect evidence of the priority components, and, if they wish, any other components for which they are able.

Remind them that evidence is what you see and hear; not what you think about it. We are describing only.

Show the video of the lesson and collect evidence with the participants.

Following the lesson, ask participants to sit quietly for a few minutes reviewing their evidence and to add any evidence that they didn'tt have time to record.

After 3 4 minutes, ask participants to compare their evidence with others sitting near them, and to add any evidence they missed, and to reorganize their evidence (arrows are fine) based on discussion.

Allow @ 10 minutes for this activity.

78Points about EvidenceAll questions are not about 3b

Engagement is about the nature of the work and who does it

Formative assessments should assess whether EACH student met the objectives. 79Help participants to think about the points on the slide.79EvidenceObservation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 1, 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment Rubric80Direct participant attention to the next step in the process: sharing evidence and teacher self assessment. Explain that in a growth model for teacher evaluation, BOTH the teacher and evaluator assess the lesson, and they do so based on the evidence that is shared.

Once the teacher receives the evidence, he/she is invited to add to it, optionally.

Then, using the rubric, the teacher self-assesses the lesson for the priority components and any others for which there is sufficient evidence. And then provides this assessment to the evaluator, IN ADVANCE of the post-teaching conference.

This is a significant departure from traditional practice, so invite participants to turn and talk to someone at another table about this process to check their understanding.

Now, show participants the self-assessment p. 15-19 of the teacher whose lesson we just watched and ask them to react to it in their table groups: where do they agree with the teachers analysis of the evidence? Where do they think differently?

Allow @ 20 minutes for them to discuss their thoughts about the teachers analysis of the evidence. Remind them that they are not invited to share their own opinions that are not supported by the evidence. 80After the ObservationShare a copy of the evidence gathered with the teacher, and give the teacher a copy of the rubric to self-evaluate. The teacher will return these to you prior to the post-observation conference.

Review the evidence and evaluate the teachers lesson using the rubric.

Compare the teachers self-assessment to your evaluation. Use the areas of non-agreement to guide the post-observation conference.

81Tell participants that the next part of the process of evaluation is for the evaluator to review the teachers self-assessment, as we just did, and then to mark the components of agreement only on their own rubric.

They are to work as a table group and to locate their own rubric in their participant materials, pp. 23 26, the priority rubric components. They are to consider, for now, just these components, in their work.

Note: this is very difficult for participants. They seem to automatically want to mark all components, whether they agree with the teachers self-assessment or not. You will need to watch closely to see that participants are doing this work correctly. 8182Conclusions about the ObservationWhere did your group mark the lesson at or above proficient?

Where did your group mark the lesson below proficient?Key points to make about the lesson:

In general, planning for this lesson (or the evidence we have of it) was not especially strong. Assessment was not planned for; nor were the objectives particularly specific.

Overall, Domain 2 was a strength with the exception of 2b.

2b Culture for Learning, and 3c Engaging Students in Learning are NOT strengths in this lesson because:The lesson relied almost solely on volunteers.If a student did not volunteer, s/he could have done nothing in the lesson except review the three division problems in the Do Now.Listening is not learning.The structure of the group activity allowed for one person to do all of the learning.Most of the activity was at the lower end of Blooms Taxonomy.8283Paradigm ShiftWho collects/provides evidence?Both teacher and evaluator

Evaluation is not done TO you; it is done with you and for youPoint out that this growth-focused process of evaluation positions the teacher as an active participant in the process. Ask participants to turn to an elbow partner and describe where, in the process, the teacher provides evidence and elicit the following:The lesson plan and pre-conference Following the observation, teacher may add to the evidenceDuring the post-conference.

83Concluding about the LessonWhere did your group mark the lesson at or above proficient?

Where did your group mark below proficient? 84WRAP UP refer to two resources Flowchart on Pg. 3 Resources and the Teacher Observation Map. of the process in resources pg. 4

Make the key points here about the lesson:

In general, planning for this lesson (or the evidence we have of it) is not especially strong. Assessment was not planned for; objectives were not particularly specific.

Overall, Domain 2 was a strength, with the exception of 2b.

2b, Culture for Learning, and 3c, Engaging Student in Learning are not strengths of this lesson, because:-The lesson relied on volunteers almost solely-If a person did not volunteer, s/he could have done nothing in the lesson except the three review division problems of the Do Now-listening is not learning-The structure of the group activity allowed for one person to do all the learning-Most of the activity was at the lower end of Blooms

84Lunch 40 minutes85EvidenceObservation-based Assessment: Process and EvidenceProcessEvidencePre-ObservationDomains 1 and 4Standard Lesson Plan with Components of Domain 1- Evidence provided by TeacherObservation:Domains 1, 2 and 3Standard Evidence Collection Document Shared with TeachersPost-TeachingDomains: 1, 2, 3 and 4Teacher Self-Assessment, Rubrics and additions/correction of evidence gatheredCollaborative AssessmentDomains 1, 2, 3 and 4Evaluator Rubric and Teacher Self-Assessment Rubric86Tell participants to revisit the steps in the process. Remind them that:

Teachers get a copy of the evidence immediately following the lessonTeachers may add to the evidenceTeachers use the evidence to complete a self-assessmentTeachers assess the lesson by highlighting the appropriate rubric phrasesTeachers provide this assessment TO THE OBSERVER IN ADVANCE OF THE POST TEACHING CONFERENCEThe observer review the teachers evidence prior to the post.The observer highlights, on his/her rubric the COMPONENTS OF AGREEMENT ONLY prior to the postThe observer LEAVES BLANK the components of difference prior to the post8687The Purpose of the Post-Observation ConferenceTo discuss the components of difference (not yet marked by observer)

To elicit any evidence that still remains to be added about the lesson

To arrive at an assessment on the rubric for components of difference.87Example:Post-Observation ConferenceEvaluatorTeacher88Use two People (Eric and Rosanne) to read the Post-Observation Conference (this is the poor example)88What were your overall impressions of the conference?

What suggestions/feedback did the observer make to the teacher?

How did the teacher respond?Individually:8989What suggestions/feedback did the observer make to the teacher?

How did the teacher respond?

What changes in Teacher Practice might you expect to see as a result of this conference?How would you know?Debrief as a Group:90Facilitators conduct what might be a typical post-conference using the script provided as a framework. USE THE TRADITIONAL POST-CONFERENCE HANDOUT provided

It is CRITICAL that the suggestions in the script be used as they serve as the basis for upcoming slides and discussion points.

90Words NOT to Use in the Post-Observation ConferenceDefendProveArgueConvince

Avoid language that suggests opposition of that might bring about a defensive response9191Language for the Post-Observation ConferenceSay more about. . .Comment on the evidence for. . .Lets look at the rubric for. . . What is the best match for. . .?Whats the backstory for. . .?

9292Overarching QuestionWorksheet #11 Pg. 21Who does the thinking? Who does the learning and growing?

93The goals of this activity is for participants to learn the skill of turning suggestions into questions.

Point out, again, that this work positions the TEACHER as an analyst of the evidence, of thinker, problem solver and generator of next steps.

Corrections and respectful differing thinking are encouraged on both sides, but evaluators are asked to avoid the telling model, since telling is not the best way to produce learning.

Tell participants that we will now explore ways to elicit FROM THE TEACHER suggestions and next steps.

Direct participants to their participant materials, page 26 Worksheet 11, and tell them change the advice they see into a question.

Work the first example with the group: Instead of Try calling on genders more equally, a question might be, What are some strategies you might use to insure that boys and girls get called upon equally? Invite the group to work with a partner to do the same with the remaining examples.

Process by calling upon non-volunteers for responses.

93Walk-Through942/20/2013Region 594Domains 1 and 4Domains 1, 2 and 3Domains 1, 2, 3, and 4Domains 1, 2, 3, and 4Agreed Upon Areas of Focus95Point out that the Framework for Teaching and the Teacher Evaluation Pilot can provide opportunities to implement change

That change can be in the form of professional development, changes in instructional delivery, classroom management, etcBut with the need to implement change comes the responsibility to monitor its effectiveness Walk-throughs can become a valuable part of this new process.95Strong evaluation systems include a series of brief, focused, unannounced, walk-throughs.The Walk-Through96Explain to participants that in addition to the announced observation process that includes the pre-/observation/post-, a valid evaluation process should include a series of brief, FOCUSED, unannounced walk-throughs.

96Must be based on an agreed upon area of focus

Quick-take

5 15 minutes

Any Domain or Component

Formative Assessment

See page 16 of the Resource Materials

The Walk-Through97These provide the teacher with a regular source of feedback, and offer the opportunity to compare the quality of teaching when it is an announced event with the quality when the event is unannounced.

The walk-through can be used as a way to provide feedback to the teacher on an agreed area of focus it is highly recommended that the walkthroughs take place after a formal observation and post-conference assessment so that the walk-through has a purpose.

The focus should be the one or two areas identified during the post-conference assessment as areas the teacher would focus his/her learning and growth

The walkthrough is a formative assessment of the progress a teacher is making toward the agreed upon goals/targets established in the post-conference

A couple of examples:

As an evaluator, you and one of your strongest teachers have just completed a post-observation conference and the teacher would like to focus on moving more towards distinguished in Domain 1, Component F Designing Student Assessments (teacher is Proficient)He/She has decided they could do a better job of differentiating assessments for student abilities and would like to attempt having students contribute to the development of assessmentsHe/She might agree to solicit from students what they think they should be able to demonstrate as proof of having accomplished a new learning.This would become the area of focus for your walkthrough

As an evaluator, you and one of novice teachers have completed a post-observation conference and the teacher would like to focus on 3b Engaging Students in Learning (You and the teacher agree the teacher is Needs Improvement/Progressing)He/She has decided they need to improve their questioning skills and ask questions designed to elicit thoughtful responses and questions that are at a much higher level of rigor.This would become the area of focus for your walkthrough

Remind participants:

The Post-Observation/Conference can be used as a means of establishing the area of focus for the walkthroughs.

You can do more than one walkthrough and it is in fact encouraged, if needed for formative assessment of teacher progress but for the purposes of the pilot, you are required to do only one per participating teacher

There is no requirement, only a strong recommendation that you conduct walk-throughs AFTER the formal observation.

This may or may not be the process that is similar to the one youre currently using in your district you may see some similarities and some differences with your process. This is what we are asking for the pilot and will elicit feedback on the process before moving forward with any actual implementation.

If you chose to do so before a Formal Observation, you must conduct a pre-conference interview to determine the agreed upon area of focus and the focus must be in one of the 4 domains.

97985 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparency9899Rule # 4:Teacher Learning IntegralConduct evaluations in such a way that they produce teacher learning.99100Professional LearningLearning is done by the learner; it is mental WORK. - Charlotte Danielson

Who does the mental work in your evaluation process? 100101The Nature of Professional Learning:

Mental Work for TeachersReflection on practiceCollaborationSelf-assessmentSelf-directed inquiry (action research)Feedback based upon evidenceTell participants that when teacher evaluation is conducted intentionally to produce teacher learning, it deliberately incorporates the characteristics found on the slide.

Ask participants to discuss with a partner where in the process we have studied each of these characteristics are found.

Say that the methods we have recommended for doing this work are deliberate. They are required to assure that learning happens,. 101102Narrative-FreeEvaluationThe rubric contains the narrative.Select the language that matches the evidence.The teacher participates in language selection.The highlighter is the tool to use!A summative domain statement is optional.Finally, explain that this process avoids the traditional narrative.

A rubric might be thought of as a bank of potential narrative statements. We select the ones that best match with the evidence, by highlighting. Once we have done this, and attached the evidence to the rubric, no narrative is required.

102103Supporting Teachers CorrectlyDirectiveCollaborativeNon-DirectiveFrom evaluator to teacherBack and forthFrom teacher to evaluatorImmoral, illegal, dangerous, cluelessBoth have ideas to contributeThe teacher deserves to take the leadDrowningSwimmingChampionship Swimming1031045 Rules for Teacher EvaluationDefensible definition of teachingDifferentiation of evaluative processesEvidence-driven processTeacher learning integralTransparencyIn closing, make it clear that an important aspect of this training is to make the process and the role of teachers clear.

It is expected that this process (or a representative version of it) will be shared with teachers in each school into which this work arrives. 104105Rule # 5: TransparencyTeachers must learn the rubrics and the process.

How might this happen in your setting? 105106Involving All StakeholdersMany teacher evaluation systems fail due to resistance that comes from the perception that the evaluation system resulted from the secret efforts of an elite few.106Notification is NOT CommunicationCommunication is two-waynot one-way107107Concluding about the LessonWhere did your group mark the lesson at or above proficient?

Where did your group mark below proficient? 108WRAP UP refer to two resources Flowchart on Pg. 3 Resources and the Teacher Observation Map. of the process in resources pg. 4

Make the key points here about the lesson:

In general, planning for this lesson (or the evidence we have of it) is not especially strong. Assessment was not planned for; objectives were not particularly specific.

Overall, Domain 2 was a strength, with the exception of 2b.

2b, Culture for Learning, and 3c, Engaging Student in Learning are not strengths of this lesson, because:-The lesson relied on volunteers almost solely-If a person did not volunteer, s/he could have done nothing in the lesson except the three review division problems of the Do Now-listening is not learning-The structure of the group activity allowed for one person to do all the learning-Most of the activity was at the lower end of Blooms

108Annotated Framework for TeachingExplore the document and:Note the benefits of using the Framework within teaching and school practice.How does it compare to the rubric itself?What is the value of this document for developing teacher effectiveness?10910 self reflection10 table talk5 minute share out

109Complete Framework for TeachingSelf-AssessmentReflectionPeer CoachingTeacher EvaluationMentoring and InductionProfessional Growth Plans110110111Building Evaluator ReliabilityReliability refers to similarity of conclusion/consistency

Consistency is a function of consensus-building activities

Evaluators must practice consensus-building activities regularly

Now, tell participants that we will continue to build our own reliability by collecting and comparing evidence for Domains 2, 3.

Make sure participants are using the evidence collection form for D2, D3 found on page 20-22of their materials.

Make sure that they write on this document when they collect evidence for the on-stage components for the elementary teaching lesson.

Show the video and collect evidence along with the group. 111Paradigm ShiftWho Collects/Provides Evidence?Both teacher and evaluator

Evaluation is not done TO you; it is done with you and for you112Point out that this growth-focused process of evaluation positions the teacher as an active participant in the process. Ask participants to turn to an elbow partner and describe where, in the process, the teacher provides evidence and elicit the following:The lesson plan and pre-conference Following the observation, teacher may add to the evidenceDuring the post-conference.

112113In ConclusionDeeper meanings of the Framework for TeachingThe nature of differentiated teacher evaluationHow to collect accurate evidence of teaching and use it, with rubrics, to assess performanceHow to conduct teacher evaluations appropriatelyHow to make changes in teacher evaluation that reduce suspicion and distrust113Now What?How do you make this happen?What supports do you need?

Resources:PDE website www.education.state.pa.us Teacher Effectiveness Wikihttp://iu3teacherevaluationpilot.wikispaces.comBinder materialsrosanne.javorsky@aiu3.net 412.394.5792patricia.dirienzo@aiu3.net 412-394-5762

114114

ElementLevel of Performance

UnsatisfactoryNeeds Improvement or ProgressingProficientDistinguished

Teacher Interaction with StudentsTeacher interaction with at least some students is negative, demeaning, sarcastic, or inappropriate to the age or culture of the students. Students exhibit disrespect for teacher.Teacher-student interactions are generally appropriate but may reflect occasional inconsistencies, favoritism, or disregard for students cultures. Students exhibit only minimal respect for teacher.Teacher-student interactions are friendly and demonstrate general warmth, caring, and respect. Such interactions are appropriate to developmental and cultural norms. Students exhibit respect for teacher.Teacher demonstrates genuine caring and respect for individual students. Students exhibit respect for teacher as an individual, beyond that for the role.

Student Interaction

Student interactions are characterized by conflict, sarcasm, or put-downs.

Students do not demonstrate negative behavior toward on another.Student interactions are generally polite and respectful.Students demonstrate genuine caring for one another as individuals and as students.

Domain 2: the classroom environment

Component 2a: Creating an Environment of Respect and Rapport

Elements:

Teacher interaction with studen