teacher evaluation pilot ii allegheny intermediate unit november 30 and december 1, 2011

Teacher Evaluation Pilot IITeacher Evaluation Pilot IIAllegheny Intermediate UnitAllegheny Intermediate Unit

November 30 and December 1, 2011November 30 and December 1, 2011

Rosanne Javorsky and Dr. Patricia DiRienzo [email protected] [email protected]

Collaborative Code

Silence electronic devicesMinimize the use of those

devices during presentations and activities

Observe the time frames and signals

Limit sidebar conversationsKeep materials organized

2

Project Development - GoalProject Development - Goal

To develop a teacher effectiveness model that:

• will reform the way we evaluate teachers• and identify critical components of teacher

training and professional growth

3

Project Development - BackgroundProject Development - Background$800,000 Gates Foundation grant to facilitate the

development of statewide policy, tools and processes to evaluate teachers and principals in which student achievement is a significant factor affecting performance ratings

PDE is closely following the work of the Pittsburgh Public Schools – PPS recipients of $40 million Gates Foundation grant that is more comprehensive in scope but similar in redesigning evaluation policy, tools and processes

4

2010-2011 Pilot I Sites2010-2011 Pilot I Sites

5

The following were the pilot sites:

Allentown School DistrictCornell School DistrictMohawk School DistrictNorthwest Tri-County Intermediate

Unit (IU 5)

7

Next Steps – Value-Added Models – (15%)Next Steps – Value-Added Models – (15%)Mathematica Policy Research Proposal

◦ Develop value-added models (VAMs) for estimating the contribution of individual teachers and principals to growth in student achievement.

◦ Provide estimates based on VAMs for teacher and principal evaluation.

◦ Assess the strength of relationships between VAM-based and observation-based measures of performance, to inform decisions about which observation-based measures should be included and how to evaluate teachers in non-tested grades and subjects.

◦ Develop and calculate summary performance indicators across value-added measures and between value-added and other measures.

◦ Synthesize the findings in annual reports that document the model’s continual development over time.

8

Next Steps – Teacher Reports and Student-Next Steps – Teacher Reports and Student-Teacher Linkages (15%)Teacher Linkages (15%) Teacher value-added reports for individual teachers by

grade/subject/year using a robust statistical report. Teacher diagnostic reports for insight on effectiveness with

students by achievement level and subgroup. Administrator summary reports for authorized users in a

particular school or district. Drill down capacity to individual student-level projections

based on classroom rosters. Teacher-level value-added reports require student-teacher

linkages, which capture the instructor(s) responsible for a student’s learning in the tested grade/subject. It is critical that this linkage system provide accurate information that is validated by individual teachers.

Web-based teacher reporting for PSSA grades 4-8 Web-based teacher reporting for PSSA grades 4-8 and

Keystone Exams

9

Multiple Measures of Teacher Evaluation - Multiple Measures of Teacher Evaluation - Anticipated Evidence (50%)Anticipated Evidence (50%)Principal/Supervisor classroom observations,

including evidence that demonstrates behaviors associated with improving student achievement:

Domains

◦ Planning and preparation◦ Classroom environment◦ Instruction◦ Professional responsibilities

10

Teacher Evaluation RubricTeacher Evaluation Rubric

4 Domains◦22 Components

4 possible ratings for each component◦Unsatisfactory◦Progressing/Needs Improvement◦Proficient◦Distinguished

11

A Framework for Teaching:A Framework for Teaching:Components of Professional PracticeComponents of Professional Practice

Domain 4: Professional Responsibilities•Reflecting on Teaching•Maintaining Accurate Records•Communicating with Families•Participating in a Professional Community•Growing and Developing Professionally•Showing Professionalism

Domain 3: Instruction•Communicating with Students•Using Questioning and Discussion Techniques•Engaging Students in Learning•Using Assessment in Instruction•Demonstrating Flexibility and

Responsiveness

Domain 1: Planning and Preparation•Demonstrating Knowledge of Content

and Pedagogy•Demonstrating Knowledge of Students•Setting Instructional Outcomes•Demonstrating Knowledge of Resources•Designing Coherent Instruction•Designing Student Assessments

Domain 2: The Classroom Environment•Creating an Environment of Respect

and Rapport•Establishing a Culture for Learning•Managing Classroom Procedures•Managing Student Behavior•Organizing Physical Space

12

Next Steps – Pilot SchoolsNext Steps – Pilot SchoolsBeginning in January

◦One Formal Observation and One walk-through observation per participating teacher Can you reduce pilot size? YES

Reporting out the Evidence◦After school is out◦Evidence collected to be shared with

Mathematica – not PDE How? Still working out the details

Questions or Questions or Comments?Comments?

13

14

Introduction to the DomainsIntroduction to the Domains

OBJECTIVES: Participants will learn…

How their thinking about good teaching compares to the framework we will use

That the Framework represents good common sense, and much that we already know, about teaching

The form and content of Domains 1, 2, 3, and 4

15

Having an ImpactHaving an ImpactIf we want to impact student

achievement and growth …

◦Then we must impact teaching and learning,

◦And, we have to agree on what good teaching is…

15

16

Worksheet #1- Pg.3Worksheet #1- Pg.3Participant MaterialsParticipant Materials

Let’s begin by collecting our thinking about good teaching

17

Wisdom of PracticeWisdom of Practice

What are the qualities of teaching most tightly tied to student

learning?

18

The DomainsThe Domains

1. Planning and Preparation

2. The Classroom Environment

3. Instruction

4. Professional Responsibilities

19

A Framework for Teaching:A Framework for Teaching:Components of Professional Components of Professional PracticePractice


Domain 3: Instruction•Communicating with Students•Using Questioning and Discussion Techniques•Engaging Students in Learning•Using Assessment in Instruction•Demonstrating Flexibility and Responsiveness





20

Worksheet #2 – Pg. 4Worksheet #2 – Pg. 4Participant MaterialsParticipant Materials

Identifying the Domains: Extending our Learning of

the Domains


21

Domain 4: Professional Responsibilitiesa. Reflecting on Teachingb. Maintaining Accurate Recordsc. Communicating with Familiesd. Participating in a Professional Communitye. Growing and Developing Professionallyf. Showing Professionalism

Domain 3: Instructiona. Communicating with Studentsb. Using Questioning and Discussion Techniquesc. Engaging Students in Learningd. Using Assessment in Instructione. Demonstrating Flexibility and Responsiveness

Domain 1: Planning and Preparationa. Demonstrating Knowledge of Content and Pedagogyb. Demonstrating Knowledge of Studentsc. Setting Instructional Outcomesd. Demonstrating Knowledge of Resourcese. Designing Coherent Instructionf. Designing Student Assessments

Domain 2: The Classroom Environmenta. Creating an Environment of Respect

and Rapportb. Establishing a Culture for Learningc. Managing Classroom Proceduresd. Managing Student Behaviore. Organizing Physical Space

Features of Features of A Framework for TeachingA Framework for Teaching

22

Generic: applies to all grade levels, content areas

Not a checklist◦ Evidence based/reflective

Not prescriptive: tells the “what” of teaching, not “how”

Comprehensive: not just what we can see

Inclusive: Novice to Master teacher

23

ELEMENT

L E V E L O F P E R F O R M A N C E

UNSATISFACTORY NEEDS IMPROVEMENT OR

PROGRESSING PROFICIENT DISTINGUISHED

Teacher Interaction with Students

Teacher interaction with at least some students is negative, demeaning, sarcastic, or inappropriate to the age or culture of the students. Students exhibit disrespect for teacher.

Teacher-student interactions are generally appropriate but may reflect occasional inconsistencies, favoritism, or disregard for students’ cultures. Students exhibit only minimal respect for teacher.

Teacher-student interactions are friendly and demonstrate general warmth, caring, and respect. Such interactions are appropriate to developmental and cultural norms. Students exhibit respect for teacher.

Teacher demonstrates genuine caring and respect for individual students. Students exhibit respect for teacher as an individual, beyond that for the role.

Student Interaction

Student interactions are characterized by conflict, sarcasm, or put-downs.

Students do not demonstrate negative behavior toward on another.

Student interactions are generally polite and respectful.

Students demonstrate genuine caring for one another as individuals and as students.

DOMAIN 2: THE CLASSROOM ENVIRONMENT COMPONENT 2A: CREATING AN ENVIRONMENT OF RESPECT AND RAPPORT

Elements: Teacher interaction with students Student interaction

Figure 6.7


24







25

Domain 3 Components

3a Communicating with Students

3b Using Questioning and Discussion Techniques

3c Engaging Students in Learning

3d Using Assessment During Instruction

3e Demonstrating Flexibility and Responsiveness

25

Worksheet #3a – Pg. 5 Worksheet #3a – Pg. 5 Participant MaterialsParticipant Materials

26

Create a specific example of your assigned component: Ms. T says to Joey, “You dummy!”

Do not restate the rubric: Ms. T is rude to a student.

Worksheet #3b – Pg. 6Worksheet #3b – Pg. 6Participant MaterialsParticipant Materials

27

What do all the examples of unsatisfactory have in common?

What do all the needs improvement/progressing examples have in common?

Proficient?

Distinguished?

Conclusions: Levels of Conclusions: Levels of PerformancePerformance

28

Unsatisfactory: Potential for harm

Progressing/Needs Improvement: Inconsistent, novice

Proficient: Consistent, competent

Distinguished: Unusually excellent, no one “lives”here permanently in all components

Components of Domain 3: Components of Domain 3: EngagementEngagement

29

3a: Communicating with Student

3b: Using Questioning and Discussion Techniques

3c: Engaging Students in Learning

3d: Using Assessment in Instruction

3e: Demonstrating Flexibility and Responsiveness

30

Let’s Take a Break – 15 Let’s Take a Break – 15 minutesminutes

30


31



Domain 1: Planning and Preparation•Demonstrating Knowledge of Content and Pedagogy•Demonstrating Knowledge of Students•Setting Instructional Outcomes•Demonstrating Knowledge of Resources•Designing Coherent Instruction•Designing Student Assessments



Domain Domain 11: Planning and Preparation: Planning and Preparation

32

1a: Demonstrating Knowledge of Content and Pedagogy

1b: Demonstrating Knowledge of Students

1c: Setting Instructional Outcomes

1d: Demonstrating Knowledge of Resources

1e: Designing Coherent Instruction

1f: Designing Student Assessments

33

Component Component 11a: Demonstrating a: Demonstrating Knowledge of Content/PedagogyKnowledge of Content/Pedagogy

1. Teacher wrote a scholarly article

2. Lesson plans/structure/content/relevance

3. Teacher explanation of probable Students’ misconceptions

4. Teacher’s answers to student questions during class

5. Teacher presented a workshop to faculty

6. Teacher explains the structure of discipline prior to lesson

7. Teacher tells observer how this lesson fits into the larger unit

8. Teacher adjusts the lesson midstream based on students’ misconceptions

9. Teacher poses different levels of content questions during the lesson

10. Teacher states how this lesson connects to content standards

34

Component Component 11a: Demonstrating a: Demonstrating Knowledge of Content/PedagogyKnowledge of Content/Pedagogy

1. Teacher wrote a scholarly article

2. Lesson plans/structure/content/relevance

3. Teacher explanation of probable Students’ misconceptions

4. Teacher’s answers to student questions during class

5. Teacher presented a workshop to faculty

6. Teacher explains the structure of discipline prior to lesson

7. Teacher tells observer how this lesson fits into the larger unit

8. Teacher adjusts the lesson midstream based on students’ misconceptions

9. Teacher poses different levels of content questions during the lesson

10. Teacher states how this lesson connects to content standards

34


35

Read the Domain 1 questions.

Why are they important?

36


Lesson Plan Data Collection Tool

Practice evidence collection

Write what is said, not what you think about it.


37







Domain 2: The Classroom EnvironmentDomain 2: The Classroom Environment

2a: Creating an Environment of Respect and Rapport

2b: Establishing a Culture for Learning

2c: Managing Classroom Procedures

2d: Managing Student Behavior

2e: Organizing the Physical Space

38


Browse Domain 2 of your Rubric

Reflect and answer questions on Worksheet #5◦ Independently

Table Share

39


40







41

Domain 4: Professional Domain 4: Professional Responsibilities Responsibilities

4a: Reflecting on Teaching

4b: Maintaining Accurate Records

4c: Communicating with Families

4d: Participating in a Professional Community

4e: Growing and Developing Professionally

4f: Showing Professionalism


42

Skim the rubrics in Domain 4.Have a table conversation about HOW

these components might impact student learning.

Self-select an “expert” group.Develop a Distinguished response for

your component.

Uses of A Framework for Uses of A Framework for TeachingTeaching

43

Self-Assessment

Reflection

Peer Coaching

Teacher Evaluation

Mentoring and Induction

Professional Growth Plans

Benefits of Using a Framework Benefits of Using a Framework for Teachingfor Teaching

44

Common language

Similar vision for good teaching and how it can be improved

Greater validity and reliability in the teacher evaluation process

Changes in novice thinking

Opportunities for collaboration

Doing Teacher Evaluation Doing Teacher Evaluation Right Right

45

5 Rules

Evidence

46

Why Evaluate Why Evaluate Teaching?Teaching?

Quality Assurance

Professional Learning

47

Doing Teacher Evaluation Doing Teacher Evaluation RightRight

OBJECTIVES: Participants will learn…

The role of the Framework in teacher assessment

Best practices in teacher evaluation

How to conduct teacher evaluation to accommodate quality assurance and professional learning, too

How to differentiate for novices, experienced teachers, and at-risk teachers

How to create buy-in for all stakeholders.

48

“The Widget Effect”

1. Count off by “a, b, c, d”

2. Read the following sections: a – p9 and pp 10-14 b – p9 and pp 15-18 c – p9 and pp 19-23 d – p9 and pp 26-30

3. Then identify the 3 most important facts from your section

49

“The Widget Effect” – Share and Learn

At your table, have each person share the three most important ideas culled from the reading.

Mark your own copy accordingly.

Discuss your thinking.

50

5 “Rules” for Teacher 5 “Rules” for Teacher EvaluationEvaluation

1. Defensible definition of teaching2. Differentiation of evaluative

processes3. Evidence-driven process4. The role of teacher learning5. Transparency

51

Overarching QuestionsOverarching Questions

Who does the thinking?

Who does the learning and growing?

52

Rule # 1: Rule # 1: Defensible Definition of Defensible Definition of TeachingTeaching

Start with a defensible definition

of good teaching that is studied,

and understood, by all stakeholders.

53


Domain 4: Professional Responsibilities•Reflecting on Teaching•Maintaining Accurate Records•Communicating with Families•Contributing to the School and District•Growing and Developing Professionally•Showing Professionalism

Domain 3: Instruction•Communicating Clearly and

Accurately•Using Questioning and Discussion Techniques•Engaging Students in Learning•Using Assessment in Instruction•Demonstrating Flexibility and Responsiveness

Domain 1: Planning and Preparation•Demonstrating Knowledge of Content and Pedagogy•Demonstrating Knowledge of Students•Selecting Instruction Goals•Demonstrating Knowledge of Resources•Designing Coherent Instruction•Assessing Student Learning



54

Card Sort – p10Card Sort – p10Participant MaterialsParticipant Materials

Read your cardUsing a Post-It note, identify:

• Domain• Component• Element

Share with table mates as instructed; get consensus

55

RewriteRewrite

Select one scenario at your table

Determine tentative Level of Performance

Rewrite at higher and lower levels using rubric characteristics

56


1. Defensible definition of teaching

2. Differentiation of evaluative processes

3. Evidence-driven process

4. Teacher learning integral

5. Transparency

57

Rule # 2:Rule # 2:Differentiation of Evaluative Differentiation of Evaluative ProcessProcess

Differentiate the processes of evaluation for:

◦ novices◦ experienced teachers◦ teachers at risk

58

Differentiated EvaluationDifferentiated Evaluation

Novice/Untenured

Very close observation and

assessment

Experienced/Tenured

Presumption of professionalism

At-Risk

Not punitive

Formal and informal

observation of teaching is key +

teacher interviews +

artifacts

Structured process 1/3yr. Other years: informals +

teacher interviews+ professional goal-setting

Intensive, extensive team-based support

based on persistent

unsatisfactory performance in

one or more components

2 – 4 formal times per year;

multiple informal observations

Professional Goal-Setting: Choose from a list of rigorous,

approved activities

Clear goals, outcomes,

evidence and timelines anchor

No self-directed activities

Activities produce evidence

which is then evaluated

Designed for the teacher who can, and wishes, to improve

59

Overarching QuestionsOverarching Questions

Who does the thinking?

Who does the learning and growing?

60

5 “Rules5 “Rules” ” for Teacher for Teacher EvaluationEvaluation

1. Defensible definition of teaching2. Differentiation of evaluative

processes3. Evidence-driven process4. Teacher learning integral5. Transparency

61

Rule # 3:Rule # 3:Evidence Driven ProcessEvidence Driven Process

Let evidence-not opinion-

anchor the process

62

Evidence or Opinion?Evidence or Opinion?1. The teacher has a warm relationship with the

students.2. The teacher said that the South should have

won the Civil War.3. The table groups were arranged in 2 x 2 pods. 4. The materials and supplies were organized

well.5. Wait time was insufficient for student thinking.6. The teacher stated that students have learned

to add 2-digit numbers in preparation for today’s lesson.

7. 6 students, questioned randomly, did not know the day’s learning goals.

63

EvidenceEvidence or or OpinionOpinion??1. The teacher has a warm relationship with the

students.2. The teacher said that the South should have

won the Civil War.3. The table groups were arranged in 2 x 2 pods. 4. The materials and supplies were organized

well.5. Wait time was insufficient for student thinking.6. The teacher stated that students have learned

to add 2-digit numbers in preparation for today’s lesson.

7. 6 students, questioned randomly, did not know the day’s learning goals.


64

EVIDENCE• is a factual reporting of events, • may include teacher/student actions and

behaviors,• may include artifacts prepared by the

teacher, students or others,• avoids personal opinion or biases,• and is selected using professional

judgment by the observer and/or the teacher.

65

Observation-based Observation-based Assessment: Process and Assessment: Process and

EvidenceEvidence

Process Evidence

Pre-ObservationDomains 1 and 4

Standard Lesson Plan with Components of Domain 1- Evidence provided by Teacher

Observation:Domains 1, 2 and 3

Standard Evidence Collection Document – Shared with Teachers

Post-TeachingDomains: 1, 2, 3 and 4

Teacher Self-Assessment, Rubrics and additions/correction of evidence gathered

Collaborative AssessmentDomains 1, 2, 3 and 4

Evaluator Rubric and Teacher Self-Assessment Rubric

66


Watch the pre-observation conference

Collect evidence on your Lesson Plan form

Write evidence only

67

Daily Lesson PlansDaily Lesson Plans

1c: What will students learn?

1e: How will I teach it to them?

1f: How will I measure which students learned it?

68

Evidence…Observation-based Evidence…Observation-based Assessment: Process and Assessment: Process and EvidenceEvidence

Process Evidence









69

Types of Observation EvidenceTypes of Observation Evidence

Verbatim scripting of teacher or student comments: “Could one person from each table collect materials?”

Descriptions of observed teacher or student behavior:The teacher stands by the door, greeting students as they enter.

Numeric information about time, student participation, resource use, etc.:Three students of the eighteen offer nearly all of the comments during discussion.

An observed aspect of the environment:The assignment is on the board for students to do while roll is taken.

70

Worksheet #8 – Pg.14Worksheet #8 – Pg.14Participant MaterialsParticipant Materials

Watch the lesson Collect evidence of what you see

and hear If you aren’t sure where to write

the evidence, just write it This is practice; relax

71

Points about EvidencePoints about EvidenceAll questions are not about 3b

Engagement is about the nature of the work and who does it

Formative assessments should assess whether EACH student met the objectives.

72


Process Evidence









73

Mark Components of Mark Components of AgreementAgreement

DO mark the components of agreement

Do NOT mark components with which you are not in agreement

74

Conclusions about the Observation

Where did your group mark the lesson at or above proficient?

Where did your group mark the lesson below proficient?

75

Concluding about the Concluding about the LessonLesson

Where did your group mark the lesson at or above proficient?

Where did your group mark below proficient?

76


Process Evidence









77

Types of Observation EvidenceTypes of Observation Evidence

Verbatim scripting of teacher or student comments: “Could one person from each table collect materials?”

Descriptions of observed teacher or student behavior:The teacher stands by the door, greeting students as they enter.

Numeric information about time, student participation, resource use, etc.:Three students of the eighteen offer nearly all of the comments during discussion.

An observed aspect of the environment:The assignment is on the board for students to do while roll is taken.

78

Building Evaluator ReliabilityBuilding Evaluator Reliability

Reliability refers to similarity of conclusion/consistency

Consistency is a function of consensus-building activities

Evaluators must practice consensus-building activities regularly

79

Worksheet #10 – Pgs. 23-24Participant Materials

Take your observation notes from the elementary lesson (Worksheet #8) and compare to the list contained in Worksheet #10

Check off the evidence noted there that you recorded on your form

Assessing the LessonAssessing the Lesson

80

On a clean copy of the rubric, highlight the phrases that match the evidence you collected while observing the elementary lesson.

Remember – you can highlight phrases in multiple levels of the same component.

81

Paradigm ShiftParadigm Shift

Who collects/provides evidence?

Both teacher and evaluator

Evaluation is not done TO you;

it is done with you and for you

82


Process Evidence









83

The Purpose of the The Purpose of the Post-Observation ConferencePost-Observation Conference

To discuss the components of difference (not yet marked by observer)

To elicit any evidence that still remains to be added about the lesson

To arrive at an assessment on the rubric for components of difference.

84

Words NOT to Use in the Words NOT to Use in the Post-Observation ConferencePost-Observation Conference

“Defend…” “Prove…” “Argue…” “Convince…”

Avoid language that suggests opposition

that might bring about a defensive response.

85

Language for the Language for the Post-Observation Post-Observation

ConferenceConference

“Say more about. . .” “Comment on the evidence

for. . .”“Let’s look at the rubric for. . .” “What is the best match for. . .”“What’s the backstory for. . .”

86






5. Transparency

87

Rule # 4:Rule # 4:Teacher Learning IntegralTeacher Learning Integral

Conduct evaluations in such a way that they

produce teacher learning.

88

Professional LearningProfessional Learning

“Learning is done by the learner; it is mental WORK.”

- Charlotte Danielson

Who does the mental work in your evaluation process?

89

The Nature of ProfessionalThe Nature of Professional Learning: Learning:

Mental Work for TeachersMental Work for Teachers

Reflection on practice

Collaboration

Self-assessment

Self-directed inquiry (action research)

Feedback based upon evidence

90

““Narrative-Narrative-FreeFree””EvaluationEvaluationThe rubric contains the narrative.

Select the language that matches the evidence.

The teacher participates in language selection.

The highlighter is the tool to use!

A summative domain statement is optional.

91

Supporting Teachers Supporting Teachers CorrectlyCorrectly

Directive CollaborativeNon-

DirectiveFrom evaluator to teacher

Back and forthFrom teacher to evaluator

Immoral, illegal, dangerous, clueless

Both have ideas to contribute

The teacher deserves to take the lead

Drowning SwimmingChampionship Swimming

92






5. Transparency

93

Rule # 5: Rule # 5: TransparencyTransparency

Teachers must learn the rubrics

and the process.

How might this happen in your setting?

94

Involving All Involving All StakeholdersStakeholders

Many teacher evaluation systems fail due to resistance that comes from the perception that the evaluation system resulted from the secret efforts of an elite few.

Notification is NOT Notification is NOT CommunicationCommunication

95

Communication is two-waynot one-way

97

In Conclusion…

Deeper meanings of the Framework for Teaching

The nature of differentiated teacher evaluation

How to collect accurate evidence of teaching and use it, with rubrics, to assess performance

How to conduct teacher evaluations appropriately

How to make changes in teacher evaluation that reduce suspicion and distrust

For additional information or support:

Rosanne [email protected]

Dr. Patricia [email protected]

98

teacher evaluation pilot ii allegheny intermediate unit november 30 and december 1, 2011

Documents

teacher valueadded reports

steps teacher reports

teacher diagnostic reports

studentteacher linkages

webbased teacher reporting

teacher specific data15

teacher effectiveness

valueadded measures