TPE: Housekeeping and Updates

November 13, 2014

Dave Volrath, Planning and Development OfficerAnd TPE Action Team

Agenda

• Appreciation and acknowledgement to LEA partners• Grants

– What’s out there– Status– Current challenges in payment

• Brief review of the MSEB report• Update on CTAC report and SLO-MOU• Update on Convenings and Pipeline• Starting to think about next year’s data collection• Q & A

Status of 3 Grants• Implementation

– $497K still on the table to be claimed– 5 LEAs have claimed nothing– 4 LEA have claimed partial– Only funds encumbered as of September 2014 can be claimed– Likelihood these funds can be kept for LEAs; mechanism not

determined

• iPad– 12 LEAs have not claimed yet– Grants were in excess of the required amount; plan to “replenish”

• Sustaining– Only 8 LEAs have submitted C-125 and narrative– Anticipate it takes at least 3 weeks to create a NOGA

Getting Paid

• Mindset at MSDE is “audit readiness”• You need to provide:

– Your summary invoice to MSDE– The supporting invoices and proof of payment– The supporting invoices must transparently and

precisely map to your request for payment– The AFR screenshot

• For the iPads, this is a breeze. Not so easy for Implementation with multiple parts.

An almost successful approachThis real example almost fits the bill. The invoices are clearly marked.

This Title IIA comment confused Accounting.

Disentangling Invoice 4072 from Title IIA or a clearer narrative would do the trick.

Another example, more problematicHow are these items linked? What is the PM invoice?

Where is the Dell invoice?

Pages and pages of sign-in sheets were provided but not in a way Accountingwas able to understand.

The FICA relates to?

Getting Paid: General Thoughts

• This is not a one-off conversation. This will apply to all RTTT grants.

• Please anticipate that Accounting will only be comfortable with a simple set of documents that “tic and tie.”

• Don’t send 200 pages of sign-in sheets. A one page example is fine, but provide a summary page that cleanly explains the number of units, hourly rate, total bill.

Quick Overview of TPE Ratings

• 43,805 teachers and 1,112 principals• MSDE only provided descriptive statistics although

we did delve into some of the subtleties• Data have gone to WestEd for the inferential analysis• Poverty and minority slides were received with

considerable interest and concern• MSA had a small effect, and more often helped than

harmed• LEAs should have a look at

– Any changes to non-MSA teachers when MSA is removed– Performance of accrued points at rating level transitions

Composition of the State n = 43,805The 5 largest LEAs represent 67% of teacher ratings

Prince

George

's n=8

768

Baltimore

County n=6

248

Baltimore

City n

=5073

Anne Aru

ndel n=5

011

Howard n

=4263

Harford

n=2

316

Charles

n=1643

Carroll

n=1641

Wash

ington n

=1370

Cecil n

=1090

Saint M

ary's

n=0

Calvert

n=9

99

Wico

mico n

=957

Worce

ster n

=640

Allega

ny n=5

55

Queen Anne's

n=5

19

Carolin

e n=3

78

Dorchest

er n=3

22

Talbot n

=295

Garrett

n=2

94

Somers

et n=2

07

Kent n

=154

-2.0%

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

20.0%20.0%

14.3%

11.6% 11.4%

9.7%

5.3%

3.8% 3.7%3.1%

2.5% 2.4% 2.3% 2.2%1.5% 1.3% 1.2% 0.9% 0.7% 0.7% 0.7% 0.5% 0.4%

Summary view of 43,805 teacher ratings

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1229

24719

17857

Highly Effective (40.8%)

Effective (56.4%)

Ineffective (2.8%)

Statewide distribution of teacher ratings by grade span configuration

Elementary n=19170 Middle n=8193 High n=12229 Combined Grades n=4213

All n=438050%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

3.3% 2.5% 2.1% 3.0% 2.8%

52.8% 52.2%59.1%

73.4%

56.4%

43.9% 45.4%38.7%

23.6%

40.8%

Statewide distribution of teacher ratings by LEA size

Large LEAs: Anne Arundel, Baltimore City, Baltimore County, Carroll, Charles, Harford, Howard, Prince George’sMedium LEAs: Calvert, Cecil, Saint Mary’s, Washington, Wicomico, WorcesterSmall LEAs: Allegany, Caroline, Dorchester, Garrett, Kent, Queen Anne’s, Somerset, Talbot

Large LEAs n=34963 Medium LEAs n=6118 Small LEAS n=2724 All n=438050%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

3.2% 0.6% 3.0% 2.8%

56.9%51.7%

60.8% 56.4%

39.9%47.7%

36.2% 40.8%

Statewide distribution of teacher ratings by LEA geographical location

Central LEAs: Anne Arundel, Baltimore City, Baltimore County, Harford, HowardEastern LEAs: Caroline, Cecil, Dorchester, Kent, Queen Anne’s , Somerset, Talbot, Wicomico, WorcesterSouthern LEAs: Calvert, Charles, Prince George’s, Saint Mary’sWestern LEAs: Allegany, Carroll, Garrett, Washington

Western n=3860 Central n=22911 Eastern n=4562 Southern n=12472 All n=438050%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.5% 2.3% 2.1% 4.7% 2.8%

35.5%

50.5%59.7%

72.6%

56.4%

64.1%

47.2%38.2%

22.7%

40.8%

Restoring MSA to models slightly moves teacher ratings toward Effective and has

minimal effect on Ineffective

All State w/o MSA (Official) All State w MSA0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

1228 1282

24719 25743

17857 16779

Delta for MSA teachers: minimum effect on “Ineffective” ratings

86.6% of teachers stay in the same rating category;All 143 “Delta +1” teachers rose from Ineffective to Effective

925 of 980 “Delta -1” teachers went from Highly Effective to Effective

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

10.2% 86.6% 3.2% Pct Delta -1Pct SamePct Delta +1

Schools in the highest quartile for poverty have more ineffective and fewer highly effective teachers than do

schools in the lowest quartile for poverty

Poverty is defined using the method for the Annual APR report: n FARMS/Enrollment sorted into quartiles

High Poverty n=10,899 Middle Range n=22,984 Low Poverty n=9,9220%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

4.6% 2.5% 1.5%

76.0%

55.7%

36.6%

19.4%

41.8%

61.9%

Schools in the highest quartile for minority students have more ineffective, fewer highly effective teachers

than do schools in the lowest quartile for minority

Minority is defined using the method for the Annual APR report: n non-White/Enrollment sorted into quartiles

High Minority n=11,546 Middle Range n=21,528 Low Minority n=10,7310%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

5.4% 2.3% 1.1%

82.5%

47.9% 45.6%

12.1%

49.9% 53.4%

Strand I Schools (meeting all annual indicator targets) have more highly effective teachers than do Strand 5

schools (failing to meet annual indicator targets)

Strands are derived from the 2013 School Progress Index; Data for 42,442 teachers linked to an SPI Strand

Strand 5

Strand 4

Strand 3

Strand 2

Strand 1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

36.1%

35.0%

38.7%

47.7%

51.6%

60.0%

62.3%

58.4%

50.4%

46.7%

4.0%

2.7%

3.0%

1.9%

1.6%

Distribution of OFFICIAL TPE Teacher RatingsMSA Excluded; N=43,805

Charles n

=1643

Allegany n=555

Howard n=4263

Somerset n

=207

Anne Arundel n=5011

Washington n=1370

Wicomico

n=957

Carroll n

=1641

Worceste

r n=640

Calvert

n=999

State Total n=43805

Queen Anne's n=519

Kent n=154

Harford n=2316

Baltimore County n=6248

Baltimore City n=5073

Cecil n=1090

Saint Mary's

n=1062

Dorcheste

r n=322

Talbot n=295

Caroline n=378

Garrett n=294

Prince George's

n=87680%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Composition of the State n = 1,112The 5 largest LEAs represent 61% of principal ratings

Prince

George

's n=2

11

Baltimore

City n

=165

Anne Aru

ndel n=1

20

Baltimore

County n=1

16

Howard n

=71

Harford

n=4

9

Wash

ington n

=44

Carroll

n=43

Charles

n=36

Cecil n

=28

Saint M

ary's

n=26

Calvert

n=2

5

Wico

mico n

=24

Allega

ny n=2

0

Worce

ster n

<20

Queen Anne's

n <2

0

Dorchest

er n <2

0

Garrett

n <2

0

Carolin

e n <2

0

Somers

et n <2

0

Talbot n

<20

Kent n

<20

0.0%

2.0%

4.0%

6.0%

8.0%

10.0%

12.0%

14.0%

16.0%

18.0%

20.0%19.0%

14.8%

10.8%

10.4%

6.4%

4.4%4.0% 3.9%

3.2%2.5% 2.3% 2.2% 2.2% 1.8%

1.3% 1.2% 1.1% 1.1% 0.9% 0.8% 0.7% 0.5%

Statewide distribution of principal ratings by grade span configuration

Elementary n=619 Middle n=162 High n=185 Combined Grades n=146

All n=11120%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

0.8% 1.2% 2.2% 3.4% 1.4%

41.5%50.6%

67.0% 65.8%

50.3%

57.7%48.1%

30.8% 30.8%

48.3%

Schools in the highest quartile for poverty have more ineffective and fewer highly effective principals than do

schools in the lowest quartile for poverty

Poverty is defined using the method for the Annual APR report: n FARMS/Enrollment sorted into quartiles

High Poverty n=314 Middle Range n=579 Low Poverty n=2190%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2.9% 1.2% 0.0%

68.8%

49.2%

26.5%

28.3%

49.6%

73.5%

Schools in the highest quartile for minority students have more ineffective, fewer highly effective principals

than do schools in the lowest quartile for minority

Minority is defined using the method for the Annual APR report: n non-White/Enrollment sorted into quartiles

High Minority n=320 Middle Range n=497 Low Minority n=2950%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

3.4% 0.0% 1.7%

69.7%

38.6%

48.8%

26.9%

61.4%

49.5%

At the Statewide level, distribution of principal ratings are generally consistent across SPI Strands.

Strand 4 schools have both the most highly effective (53.3%) and the most ineffective principals (2.5%)

Strands are derived from the 2013 School Progress Index; Data for 1066 principals linked to an SPI Strand

Strand 5

Strand 4

Strand 3

Strand 2

Strand 1

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

44.1%

53.3%

46.9%

52.2%

44.9%

54.3%

44.3%

51.4%

46.6%

55.1%

1.6%

2.5%

1.7%

1.2%

0.0%

Distribution of OFFICIAL TPE Principal RatingsMSA Excluded; N=1,112

Charles n

=36

Howard n=71

Somerset N

<10

Baltimore County n=166

Harford n=49

Anne Arundel n=120

State n=1,112

Wicomico

n=24

Washington n=44

Saint Mary's n

=26

Garrett n=12

Cecil n=28

Carroll n

=43

Prince Georges n

=211

Talbot n<10

Caroline n=10

Baltimore City n-165

Calvert n=25

Queen Anne's n=13

Worceste

r n=14

Allegany n=20

Dorcheste

r n=12

Kent n<10

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

The TPE Team was very cautious and made no “pronouncements”

The Team suggested:• Actual differences in teacher and

principal performance• Differences in LEA evaluation model

performance• Precision in fitting cut scores

SLO Headlines

• “Real Progress in Maryland” reaffirmed what we already know– It’s a heavy lift– Penetrating the classroom is hard– Managing all the logistics is hard, and good

systems are critical– The closer folks are to the work, the better they

feel about it– SLO investments are a good place for Sustaining

dollars

Update on CTAC/SLOs

• MSDE is working with CTAC to annotate SLOs to create an LEA resource. Samples should represent:

–Various grades and subjects–Assessed and non-assessed areas–Teacher AND Principal samples–Aligned to the Quality Rating Rubric

• No sample will be identified by LEA

Update on SLO Collaboration MOU

• Fall convenings• Overview of feedback• December focus on communication• Plans to close the Quality Control loop for PY 5

Next Year’s Data Collection

• There will be no MSA strand to collect• The APR variables we discussed last year

festered with USDE again: eligible for tenure, retained, promoted, compensation

• Headwinds behind interest to link teachers to preparatory programs or strands

• MSEB interest in how 1st/2nd year teachers fare one year later

Contacts

Dave Volrath, Planning and Development Officer David.Volrath@maryland.gov , 410 767 0504

Ben Feldman, TPE TeamBen.Feldman@maryland.gov , 410 767 0142

Today’s data release on: LEA/School Teacher-Principal Evaluations.