the delaware performance appraisal system-ii (dpas-ii)

The Delaware Performance

Appraisal System-II

(DPAS-II) 2013-2014 Review

August 2014

DPAS-II COMPONENTS

Components I-IV:

Observation

1. Planning and Preparation

2. Classroom Environment

3. Instruction

4. Professional Responsibilities

Component V:

Student Improvement

First measure of student growth (50%)

Second measure of student growth (50%)

Satisfactory Unsatisfactory Exceeds Satisfactory Unsatisfactory

2

Summative Ratings Chart

Total # of

Satisfactory ratings

in Components I-IV

Rating in Component V Summative Rating

3/4 or 4/4 Exceeds Highly Effective

3/4 or 4/4 Satisfactory Effective

2/4 Exceeds or Satisfactory Effective

3/4 or 4/4 Unsatisfactory Needs

Improvement

0/4 or 1/4 Exceeds or Satisfactory Needs

Improvement

0/4 or 1/4 or 2/4 Unsatisfactory Ineffective

3

N=11,592

Distribution of Component V Educator Groups

(2013-14)

4

Notes: All data are based on district 2013-14 data entry into the state’s Evaluation Reporting System (ERS). The total

number of educators with Educator Group Numbers indicated in ERS is 11,592.

Group 1 includes educators who instruct reading and/or mathematics in grades 3 through 10 and are the educator-of-

record for at least 10 students. Group 2 includes anyone who teaches in any grade or subject other than ELA and/or

mathematics (DCAS-tested). Group 3 include anyone who does not meet the criteria for Group I or Group II educators

such as school nurses, psychologists, and guidance counselors.

70%

46% 34%

27%

53% 65%

Group 1 (n=1630) Group 2 (n=2489) Group 3 (n=1540)

Share of Educators in 2013-14 with "Highly Effective" or "Effective" Summative Ratings

by Educator Group Effective Highly Effective

5

Notes: All data are based on district 2013-14 data entry into the state’s Evaluation Reporting System (ERS). The number of

educators statewide with summative evaluation ratings reported is 1630 for Group 1, 2489 for Group 2, and 1540 for Group 3.

Group 1 includes educators who instruct reading and/or mathematics in grades 3 through 10 and are the educator-of-record for

at least 10 students. Group 2 includes anyone who teaches in any grade or subject other than ELA and/or mathematics (DCAS-

tested). Group 3 include anyone who does not meet the criteria for Group I or Group II educators such as school nurses,

psychologists, and guidance counselors.

0

10

20

30

40

50

60

70

80

90

100

% R

ated

Hig

hly

-Eff

ecti

ve

Share of Educators with "Highly Effective" Summative Rating in 2013-14

by Educator Group and School District

Group 1 Group 2 Group 3

Notes: All data are based on district 2013-14 data entry into the state’s Evaluation Reporting System (ERS). The chart

suppresses data for districts with less than 15 educators in a given educator group.

Group 1 includes educators who instruct reading and/or mathematics in grades 3 through 10 and are the educator-of-record for

at least 10 students. Group 2 includes anyone who teaches in any grade or subject other than ELA and/or mathematics (DCAS-

tested). Group 3 include anyone who does not meet the criteria for Group I or Group II educators such as school nurses,

psychologists, and guidance counselors. 6

1

2

3

3

3

3

3

4

3

2

5

3

3

7

5

81

80

80

79

88

74

76

78

82

76

82

78

84

88

16

17

17

19

9

22

19

19

16

19

15

20

9

6

Selecting Instructional Goals

Designing Coherent Instruction

Knowledge of Content & Pedagogy

Demonstrating Knowledge of Students

Designing Student Assessments

Managing Classroom Procedures

Managing Student Behavior

Create Environment to Support Learning

Organizing Physical Space

Engaging Students in Learning

Demonstrating Flexibility

Communicate Clearly & Accurately

Using Questioning Techniques

Using Assessments in Instruction

2013-14 Distribution of Ratings on DPAS-II Criteria for Components I-III (% of Educators)

Unsatisfactory Basic Proficient Distinguished

Co

mp

on

en

t I

Co

mp

on

en

t II

Co

mp

on

en

t III

7


number of educators rated on the DPAS-II criteria listed above ranged from 3,369 (Organizing Physical Space) to 6,485

(Managing Classroom Procedures). The number of educators with ratings entered for each of the criteria may vary as

districts were allowed to waive certain criteria and the quality of data entry varied across Delaware schools.

Notes: All data are based on district 2013-14 data entry into the state’s Evaluation Reporting System (ERS). The

number of educators with “managing student behavior” ratings in each district is as follows: District A (47), District B

(56), District C (945), District D (272), and District E (349). 8

-2% -3% -5% -7%

96% 96%

85%

70%

80%

4% 2%

12%

25% 12%

District A District B District C District D District E

% o

f E

du

cato

rs R

ate

d

2013-14 Distribution of Ratings on DPAS-II Criterion: "Managing Student Behavior"

by District

Distinguished

Proficient

Basic

Unsatisfactory

Notes: All data are based on district 2013-14 data entry into the state’s Evaluation Reporting System (ERS). The

number of educators with “using questioning techniques” ratings in each district is as follows: District A (45), District B

(55), District C (923), District D (269), and District E (351). 9

-1 -2 -3 -9 -12 -12

95 92

80 79 85

4 5

11 9

3

District B District C District A District D District E

% o

f E

ducato

rs R

ate

d

2013-14 Distribution of Ratings on DPAS-II Criterion: "Using Questioning Techniques"

by District

Distinguished

Proficient

Basic

Unsatisfactory

DPAS-II Components 1-4 Distribution (2013-14)

Component I-IV Ratings Percent

Percent with 0 “Satisfactory” Ratings < 1%

Percent of with 1 “Satisfactory” Rating < 1%



Percent with 4 “Satisfactory” Ratings 99%

10

All data are based on district 2013-14 data entry into the state’s Evaluation Reporting System (ERS). The number of

educators with Components 1-4 data entered into ERS is 6,353.

COMPONENT V MEASURES & GROUPS

MEASURE A

Growth targets are based on DCAS instructional scale

scores and student growth targets, which are provided by

the DDOE.

Exceeds Satisfactory Unsatisfactory

(discretion) Unsatisfactory

65% or more

of a teacher’s

DCAS student

growth targets

are met.

50-64% of a

teacher’s DCAS

student growth

targets are met.

35-49% of a teacher’s

DCAS student growth

targets are met.

Administrator could

upgrade to a “Satisfactory”

rating.

Less than 35%

of a teacher’s

DCAS student

growth targets

are met.

MEASURE B

Growth targets are based on internal assessments

developed by educators or external measures approved by

DDOE. Targets are set at a conference with the

administrator in the fall.

MEASURE C Growth goals are educator-developed and DDOE-approved;

specific to content areas and job assignments.

Exceeds Satisfactory Unsatisfactory

The agreed upon

“exceeds” target is met

or surpassed.

The agreed upon “satisfactory”

target is met or surpassed, but the

“exceeds”

target is not met.

The agreed upon

“satisfactory”

target is not met.

Group 1: Instructors

of >9 students

teaching reading or

math in grades 3-10

A 50%

B 50%

Group 2: Instructors of

>9 students in grades

and subjects other

than DCAS

reading/math for whom

a Measure B is

available

B 50%

C 50%

Group 3: Any educator

who does not meet the

criteria for Group 1 or

Group 2

C 100%

11

12


number of educators with each of the ratings listed above entered in ERS is: 2816 (Measure A), 7599 (Measure B),

7283 (Measure C), and 10,596 (Component V Overall Ratings).

13 6

1 1

55

29

27

48

32

65 72

51

Measure A(n=2816)

Measure B(n=7599)

Measure C(n=7283)

Overall(n=10,596)

% o

f E

du

ca

tors

Rate

d

2013-2014 Component V Measures and Overall Rating Distribution

Unsatisfactory Satisfactory Exceeds

Unsatisfactory 7%

Unsatisfactory with discretion

22%

Satisfactory 39%

Exceeds 32%

Component Measure A Rating 2013-14

Unsatisfactory 28%

Satisfactory 72%

Final Measure A Final Ratings for Educators

Rated "Unsatisfactory with Discretion“ (2013-14)

13


educators with Measure A Ratings is 2,816.

14


educators with “Unsatisfactory with Administrator Discretion” ratings in each district is as follows: District M (14), District

N (24), District O (19), District P (32), District Q (30), District R (102), District S (38), District T (35), District U (11),

District V (114), District W (13), District X (74), District Y (25), and District Z (12).

92%

88%

85%

85%

75%

73%

71%

68%

66%

63%

63%

58%,

42%

29%

District Z

District Y

District X

District W

District V

District U

District T

District S

District R

District Q

District P

District O

District N

District M

% Upgraded with Administrator Discretion

Share of 2013-14 Measure A "Unsatisfactory with Administrator Discretion" ratings upgraded

to "Satisfactory" by District

1 6 3 1 6 6 3

10 4 9 5 1 4 12

6 16

9 16

24

4

17 22 27 23 25 29

23 30 26 31 36 34 27

33 24

33 32

31

95

77 76 73 72 70 69 68 66 65 64 62 62 62 61 60 58 52

44

% o

f Ed

uca

tors

Rat

ed

Distribution of 2013-14

Component V Measure B Ratings by District

Unsatisfactory Satisfactory Exceeds

15


educators with Measure B Ratings entered in ERS ranges from 78 in POLYTECH to 915 in Red Clay School District.

Distribution of Summative Ratings (2013-14)

16


educators with summative ratings entered into ERS is 5,897.

DPAS-II Annual Evaluation—Survey & Focus Groups

2014

Dr. Donald Beers Progress Education Corporation 17

Surveys ◦ 44.8% response rate for teachers (46% 2013) ◦ 44.8% response rate for specialists as well (47% 2013) ◦ 53.4% response rate for administrators (44% 2013)

40 interviews ◦ Expanded to 30 minutes for in-depth conversation ◦ Five districts selected

Caesar Rodney Milford Sussex Technical Cape Henlopen Colonial

Focus Groups ◦ Six focus groups


A grade of “C” was given by a majority of the respondents.

19% of teachers, 25% of specialists, and 32% of administrators reported they were “able to contribute to changes in the DPAS-II system”.

Unannounced observations were reported to have more value for all groups this year.

Walk-through observations provide valuable feedback for 66% of teachers and 76% believe the written feedback is useful.

The scope of unannounced observations is too broad.

Components I-IV add value to the practice of educating students.

The demands of improvement plans on administrators are costly in time and effort.

Technology exists but is not being fully utilized to improve feedback, and streamline the process.

Use data from previous as well as current year for summative year evaluation.

Component V has too great an impact on overall rating.

A majority of administrators would like additional credentialed observers.


42% of administrators believe they have been adequately involved in

improving the DPAS-II system, only 31% of teachers felt teachers have

been adequately involved.

19% of teachers, 25% of specialists, and 32% of administrators reported

they were “able to contribute to changes in the DPAS-II system.”

61% of teacher respondents believe DPAS-II is one of the top five drivers

of student achievement gains.

66% of teachers believe the increased flexibility with unannounced

observation has saved time and 61% believe it has improved feedback

opportunities.

72% of administrators believe the requirement to give criterion-level

ratings in 2013-14 “allowed for meaningful conversations” about growth.

68% of administrators agreed that the DPAS-II system helps drive instructional improvement in their work location.


The majority of teachers gave the DPAS II process a grade of “C”.

The survey shows teachers are split (52% to 48%) on whether the system is fair and equitable.

Interviews and focus groups indicated that many teachers believe DCAS testing and component V carries too much weight in the process - almost half of respondents (teachers) consider the process not fair.

A large majority of teachers hold a positive view of Components I-III.

A large majority of the teachers believe planning and preparation, classroom environment, and instruction are the key components.

Teachers responded (70%) on the survey that DPAS II should not be continued in its current form. Interviews and focus group discussions indicated a continued need for streamlining the process.


The majority believe the evaluation process should be

differentiated based on an educator’s years of experience and

role.

Walk-throughs are beneficial to the practice of teaching

according to 66% of teachers. 76% believe the feedback is

beneficial.

Teachers spoke positively of the attempt at providing flexibility

to the “Student Performance” aspect of DPAS-II by allowing

multiple measures as an example.

Quantitative and qualitative results both supported that

teachers like the feedback they received from various aspects

of the process.


The majority of specialists do not like the DPAS process

because they believe student improvement is not relevant to

their work.

Specialists believe good indicators of performance are

“Planning and Preparation,” “Professional Practice and Delivery

of Service,” “Professional Collaboration and Consultation,

“Professional Responsibilities.”

The website was informative according to the majority of

specialists and resources were adequate.

An overwhelming majority of specialists believe that the

evaluation process should be based on their role.


Administrators made positive statements during interviews and focus groups concerning regulatory changes to DPAS II process.

During interviews and focus groups, administrators voiced concerns with the weighting of component V.

Administrators (over 70%) believe that conferencing is excellent and has contributed to improving practice.

The majority (63%) of administrators responded that DPAS II does positively impact their practice. The qualitative results indicated that the walk-throughs, conferencing, and unannounced observations were the main reasons.

Administrators believe that the evaluation process should be differentiated based on an educator’s years of experience (72%) and role (91%).

Administrators are positive about Bloomboard and the potential to save time and provide timely feedback.

During interviews and focus groups, administrators reported teachers under an improvement plan take 5-10 times the number of hours to manage than other staff.


“Component V” ◦ Overshadows observations and conversations ◦ Application to new or novice teachers ◦ Should directly impact professional development ◦ Review and publicize growth targets

“Improvement Plan” status negatively impacts process ◦ Large time and effort commitment by administrators ◦ Intervention by or placed under qualified professional

Similar to practice teaching; Development Coaches

Expand the use of technology Expand the use of walkthroughs Narrow criteria for unannounced observations Refocus through PD benefits of reflective practice Publicize reforms to DPAS-II


the delaware performance appraisal system-ii (dpas-ii)

Documents