7/27/2019 Uconn Neag Peac 7-10-13

1/26

Educator EvaluationPilot Program Study

Round 2 Findings

Center for Education Policy AnalysisNeag School of Education

Morgaen Donaldson, Casey Cobb, Rachael Gabriel,Richard Gonzales, Kimberly LeChasseur and Sarah Woulfin

7/27/2019 Uconn Neag Peac 7-10-13

2/26

Data Collection January - mid-April, 2013

Data collected on second phase of SEEDimplementationo Mid-year check-ins

o Observations

o Feedback and conferences

Interviews with same sample of educatorsinterviewed in fall, 2012

Surveys of teachers in most schools in interviewsample

2

7/27/2019 Uconn Neag Peac 7-10-13

3/26

Interviews Solicited interviews from all individuals interviewed in

fall, 2012

209 interview respondents

3

7/27/2019 Uconn Neag Peac 7-10-13

4/26

Table 1. Interview and Focus Group Sample

District/ConsortiumSchools District

Leaders Principals Teachers Specialists Total

Bethany 1 0 2 6 0 8

Branford 3 0 0 10 2 12

Bridgeport 3 2 1 25 8 36

CEFS 4 2 3 34 0 39

CREC 5 1 4 29 2 36

Litchfield/Region 6 5 0 3 19 2 24

Norwalk 1 2 0 1 0 3Waterford 0 2 0 0 0 2

Windham 4 2 4 31 11 48

Windsor 1 0 1 0 0 1Total 27 11 18 155 25 209

4

7/27/2019 Uconn Neag Peac 7-10-13

5/26

Surveys Invited all districts and schools in interview sample to

participate

684 teachers from eight pilot districts/consortiaresponded to survey (25 schools)

Overall response rate = 45%

Response rate ranged from 24% to 79% acrosssample sites

5

7/27/2019 Uconn Neag Peac 7-10-13

6/26

Table 2. Survey Sample (n=684, response rate=45%)

Note: 25 of 36 schools in the overall sample participated

District/Consortium

Schools Sample

n n %

Bethany 1 16 2%

CEFS 3 69 10%

CREC 6 142 21%

Litchfield/Region 6 5 80 12%

Norwalk 3 58 9%

Waterford 3 84 12%Windham 3 138 20%

Windsor 1 97 14%

Total 25 684 100%

6

7/27/2019 Uconn Neag Peac 7-10-13

7/26

Findings1. Most teachers understand SEED

2. Most districts/schools are making strides towardsimplementing SEED teacher evaluation with fidelityo An increase in classroom observations compared to previous years

o Completion of SEED procedures

3. When implemented well, SEED yields positive outcomeso Teachers report talking with evaluator about their practice is valuable

o Evidence of changed and improved practices in some settings

4. Most districts/schools focused on compliance ratherthan leveraging SEED model to improve practiceo Lack of in-depth feedback opportunities or professional development tied to

observation data

5. Most districts/states delayed enactment of SEED schoolleader evaluation

7

7/27/2019 Uconn Neag Peac 7-10-13

8/26

1. Most teachers understand SEED

56% of teachers surveyed agreed/strongly agreedthat they understood and were comfortable withSEED procedures

Tenured teachers reported significantly less

understanding/comfort than non-tenured teacherso 66% non-tenured vs. 55% tenured agreed/strongly agreed

Secondary teachers reported significantly lessunderstanding/comfort than elementary teacherso 72% elementary vs. 54% middle school, 53% high school agreed/strongly

agreed

Much greater clarity on SEED than in fall, 2012

8

7/27/2019 Uconn Neag Peac 7-10-13

9/26

2. Fidelity of Implementation

Most districts/schools are making strides towardsimplementing SEED with fidelity

Observations: All administrators report struggling tocomplete mandated number of observations, yet

among teachers surveyed by April:o 42% reported they had been formally observed at least twice

o 55% reported they had been informally observed at least twice

9

7/27/2019 Uconn Neag Peac 7-10-13

10/26

10

10

48

31

10

1

17

27 28

17

10

0

10

20

30

40

50

60

0 1 2 3 More than 3

Percent

age

Number of Observations

Figure 1. Number of Observations(nI=609; nF=613)

Formal

Informal

7/27/2019 Uconn Neag Peac 7-10-13

11/26

Figure 2. Time Spent Being Observed Compared to Pre-SEED

3%5%

38%

32%

22%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

PercentageofTeachers

A lot less time A bit less time About the same A bit more time A lot more time

11

7/27/2019 Uconn Neag Peac 7-10-13

12/26

Discussions:Figure 3. Time spent in discussion with evaluator under SEED

compared to pre-SEED

1% 1%

26% 28%

44%

6%8%

47%

27%

13%

6% 7%

45%

28%

14%

0%

10%

20%

30%

40%

50%

60%

A lot less time A bit less time About the Same A bit more time A lot more time

Goal-setting

Talking about your practice with your principal/evaluator before observations

Talking about your practice with your principal/evaluator after observations

12

7/27/2019 Uconn Neag Peac 7-10-13

13/26

Feedback

43% of teachers surveyed agreed or stronglyagreed that they were receiving and usingfeedback under SEED

Significantly fewer tenured teachers reportedreceiving and using feedback than non-tenuredteacherso 35% of tenured teachers agreed or strongly agreed vs. 60% of non-

tenured teachers.

13

7/27/2019 Uconn Neag Peac 7-10-13

14/26

Other SEED Processes

Figure 4. Time Spent on Improvement-Related Tasks under SEEDCompared to Pre-SEED

5%7%

54%

24%

11%

1% 2%

37%

30% 30%

0%

10%

20%

30%

40%

50%

60%

70%

80%

A lot less time A bit less time About the Same A bit more time A lot more time

Talking about practice with other teachers Analysis of data about your students

14

7/27/2019 Uconn Neag Peac 7-10-13

15/26

3. Positive Outcomes of SEED

Figure 5. Value of Time Spent Being Observed Under SEED

3%

12%

22%

44%

19%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

PercentageofTeacher

s

Not at all valuable Not very valuable Neutral Somewhat valuable Very valuable

15

7/27/2019 Uconn Neag Peac 7-10-13

16/26

Figure 6: Value of Discussion With Evaluator Under SEED

6%

16%

18%

43%

18%

3%

12%

27%

39%

19%

2%

8%

19%

43%

27%

0%

10%

20%

30%

40%

50%

60%

Not at all valuable Not very valuable Neutral Somewhat valuable Very valuable

Goal-setting

Talking about your practice with your principal/evaluator before observations

Talking about your practice with your principal/evaluator after observations

16

7/27/2019 Uconn Neag Peac 7-10-13

17/26

Figure 7. Value of Time Spent on Improvement-Related Tasks

3% 1%

10%

3%

11%

11%

41%

32%

34%

53%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Spent more time analyzing student data Spent more time talking about practice

with other teachers

Very valuable

Somewhat valuable

Neutral

Not very valuable

Not at all valuable

17

7/27/2019 Uconn Neag Peac 7-10-13

18/26

Promising Practices

Observations of videosLeaders rate teachersvideo-recorded instruction using the rubric. Theleader and teacher use the rubric and video todiscuss how the teachers instruction could be

improved. Teachers report that this reduced theiranxiety to perform in front a live observer andhelped them understand leaders feedback.

Complementary observersSchools have used

complementary observers (teachers or centraloffice administrators).

18

7/27/2019 Uconn Neag Peac 7-10-13

19/26

4. Most districts/schools focused on compliance ratherthan leveraging SEED model to improve practice

In interviews, school leaders and teachers reportedthat leaders generally emphasized paperwork andreporting rather than implementing SEED to develophuman capital

19

7/27/2019 Uconn Neag Peac 7-10-13

20/26

Midyear check-ins

Characterized by a technical, procedural focusrather than an emphasis on improving instruction,assessment, and student performance

Most mid-year check-ins lasted 15-20 minutes anddid not result in revised SLOs

Some teachers were not aware that they couldchange their SLOs if justified

20

7/27/2019 Uconn Neag Peac 7-10-13

21/26

Debriefs/conversations

Only 30% of teachers surveyed reported that they spent

more time talking with their evaluator about their

practice after being observed under SEED than they did

last year and that the experience was very valuable

Most debrief conversations were late and relativelyperfunctory (about 15-20 minutes long)

Some debrief conversations were conducted through

My Learning Plan or via email

Almost no teachers reported that they had receivedspecific recommendations of professional growth

opportunities during debriefs

In one school, no debrief conversations were reported

21

7/27/2019 Uconn Neag Peac 7-10-13

22/26

Teacher Perceptions at Mid-Year

Teachers WhoStrongly Agreed or Agreed Their Evaluatorhas Knowledge to Evaluate Them Accurately ranged

from 18% to 62% across eight districts (overall 51%, n=601)

Teachers WhoStrongly Agreed or Agreed TheirEvaluator has Sufficient Time & Resources to EvaluateThem Accurately ranged from 7% to 24% across eight

districts (overall 17%, n=599)

Teachers WhoStrongly Agreed or Agreed

SEED Could Improve Instructional Practice ranged from

11% to 32% across eight districts (overall=22%; n=603)

22

7/27/2019 Uconn Neag Peac 7-10-13

23/26

Administrator Perceptions at Mid-Year

According to interviews of school leaders:o SEED has potential but:

o Scheduling and completing required number of observations is

challenging

o Reporting requirements (i.e. My Learning Plan) are cumbersome

23

7/27/2019 Uconn Neag Peac 7-10-13

24/26

SLOs/IAGDs

59% of teachers report that their SLOs are bothrigorous and attainable

48% of teachers in our sample said that their SLOswere only somewhat rigorous

36% said that their SLO is less than likely to beattained

24

7/27/2019 Uconn Neag Peac 7-10-13

25/26

Administrator evaluation

All district leaders and principals report that theadministrator portion of SEED is being implemented

All districts began to implement the SEEDadministrator evaluation quite late (i.e. December-

January)

Districts are implementing the minimum of theadministrator evaluation

In the view of principals, SEED does not differ muchfrom their prior evaluation systems

25

7/27/2019 Uconn Neag Peac 7-10-13

26/26

Recommendations

Offer professional development to administrators and

teachers specific to each phase of implementation

Bolster professional development focused on coaching

aspects of seed (i.e. Mid-years; debriefs; feedback)

Publicize and promote the complementary observer role

Streamline paperwork/reporting requirements

Better align administrator and teacher evaluation

26