ohio’s experience with ayp

Ohio’s Experience with AYP

Presentation to the American Educational Research Association

Mitchell D. Chester

April 12, 2004

2

Pre-Implementation Concerns

• Over-identification

• Accountability for students with disabilities

• Volatility of results

3

Presentation Outline

• Ohio’s accountability system

• 2002-03 AYP results

• Measures employed to increase validity and reliability

• Conclusions

4

Ohio’s Accountability System

• School and district rating criteria

• 2002-03 results

5

Ohio Rating Criteria

94% to 100%

(17 or 18 for districts)

94% to 100%(17 or 18 for districts)

75% to 93%(14 to 16 for districts)






Met AYPandor

Excellent

Performance Indicators

Missed AYPor

Missed AYP

Missed AYP

Missed AYP

and

and

and

Missed AYP

Met AYP

and

and

and

and

or

or

Federal AYP Requirements

Value Added -- Once grades 3 to 8 reading and

math assessments are implemented and multiple

years of data available, Ohio will incorporate a measure of individual student grade-to-grade achievement gains

to help determine school building and district des

Performance Index Score Growth Calculation

100 to 120

100 to 120

90 to 99 Met AYP

Effective

or

or

or

90 to 99

80 to 89

70 to 79

0 to 89Continuous

Improvement

or

Temporary Growth Calculation -- Districts and schools will move from

Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c)

most recent year'

and

Academic Watch

Academic Emergency and 0 to 69

6


94% to 100%











most recent year'

and

Academic Watch


Effective

or

or

or

90 to 99

80 to 89

70 to 79

0 to 89Continuous

Improvement

or

or

or







100 to 120

100 to 120

90 to 99 Met AYP

Missed AYP

Met AYP

and

and

and

and

Missed AYP

Missed AYP

Missed AYP

and

and

and

Met AYPandor

Excellent


Missed AYPor

7


94% to 100%











most recent year'

and

Academic Watch


Effective

or

or

or

90 to 99

80 to 89

70 to 79

0 to 89Continuous

Improvement

or

or

or







100 to 120

100 to 120

90 to 99 Met AYP

Missed AYP

Met AYP

and

and

and

and

Missed AYP

Missed AYP

Missed AYP

and

and

and

Met AYPandor

Excellent


Missed AYPor

8


94% to 100%











most recent year'

and

Academic Watch


Effective

or

or

or

90 to 99

80 to 89

70 to 79

0 to 89Continuous

Improvement

or

or

or







100 to 120

100 to 120

90 to 99 Met AYP

Missed AYP

Met AYP

and

and

and

and

Missed AYP

Missed AYP

Missed AYP

and

and

and

Met AYPandor

Excellent


Missed AYPor

9


94% to 100%










Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c) most recent year's gain of at

least 3 points.

and

Academic Watch


Effective

or

or

or

90 to 99

80 to 89

70 to 79

0 to 89Continuous

Improvement

or

or

or





to help determine school building and district

designations.


100 to 120

100 to 120

90 to 99 Met AYP

Missed AYP

Met AYP

and

and

and

and

Missed AYP

Missed AYP

Missed AYP

and

and

and

Met AYPandor

Excellent


Missed AYPor

10

District Designations

Total

Excellent 85 85 0

Effective 178 149 29

Continuous Improvement

278 255 19 3 5

Academic Watch

52 36 15 1

Academic Emergency

16 16

Not Rated 4


Only

Performance Index

Performance Index Growth

Meeting AYP

11

School Designations

Total

Excellent 633 621 12

Effective 765 496 269


1,242 360 532 131 806

Academic Watch

237 125 77 39

Academic Emergency

338 338

Not Rated 500

Performance Index

Performance Index Growth

Meeting AYPPerformance

Indicators Only

12

2002-03 AYP Results

• AYP within rating categories

• School Improvement within rating categories

• Disentangling the AYP categories

13

AYP within Ohio’s Rating Categories

Number Percent Number PercentIn School

Improvement

Excellent 85 18 21.2% 633 36 5.7% 0

Effective 178 65 36.5% 765 45 5.9% 0


278 166 59.7% 1,242 152 12.2% 35

Academic Watch

52 52 100.0% 237 237 100.0% 36

Academic Emergency

16 16 100.0% 338 338 100.0% 117

Not Rated 4 0 0.0% 500 0 0.0% 3

TOTAL 609 317 52.1% 3,215 808 25.1% 191

SchoolsDistrictsMissing AYP Missing AYP

Total Total

14



Improvement

Excellent 85 18 21.2% 633 36 5.7% 0

Effective 178 65 36.5% 765 45 5.9% 0


278 166 59.7% 1,242 152 12.2% 35

Academic Watch

52 52 100.0% 237 237 100.0% 36

Academic Emergency

16 16 100.0% 338 338 100.0% 117

Not Rated 4 0 0.0% 500 0 0.0% 3

TOTAL 609 317 52.1% 3,215 808 25.1% 191


Total Total

15



Improvement

Excellent 85 18 21.2% 633 36 5.7% 0

Effective 178 65 36.5% 765 45 5.9% 0


278 166 59.7% 1,242 152 12.2% 35

Academic Watch

52 52 100.0% 237 237 100.0% 36

Academic Emergency

16 16 100.0% 338 338 100.0% 117

Not Rated 4 0 0.0% 500 0 0.0% 3

TOTAL 609 317 52.1% 3,215 808 25.1% 191


Total Total

16



Improvement

Excellent 85 18 21.2% 633 36 5.7% 0

Effective 178 65 36.5% 765 45 5.9% 0


278 166 59.7% 1,242 152 12.2% 35

Academic Watch

52 52 100.0% 237 237 100.0% 36

Academic Emergency

16 16 100.0% 338 338 100.0% 117

Not Rated 4 0 0.0% 500 0 0.0% 3

TOTAL 609 317 52.1% 3,215 808 25.1% 191


Total Total

17



Improvement

Excellent 85 18 21.2% 633 36 5.7% 0

Effective 178 65 36.5% 765 45 5.9% 0


278 166 59.7% 1,242 152 12.2% 35

Academic Watch

52 52 100.0% 237 237 100.0% 36

Academic Emergency

16 16 100.0% 338 338 100.0% 117

Not Rated 4 0 0.0% 500 0 0.0% 3

TOTAL 609 317 52.1% 3,215 808 25.1% 191


Total Total

18

School Improvement within Ohio’s Rating Categories


Improvement

Excellent 85 18 21.2% 633 36 5.7% 0

Effective 178 65 36.5% 765 45 5.9% 0


278 166 59.7% 1,242 152 12.2% 35

Academic Watch

52 52 100.0% 237 237 100.0% 36

Academic Emergency

16 16 100.0% 338 338 100.0% 117

Not Rated 4 0 0.0% 500 0 0.0% 3

TOTAL 609 317 52.1% 3,215 808 25.1% 191

Missing AYP Missing AYPTotal Total

SchoolsDistricts

19

AYP CategoriesDistricts School Buildings

Total Missing AYP 317 808

Met Both Reading & MathProficiency

9 71

Missed Both Reading & MathProficiency

197 344

Missed Reading Proficiency, MetMath

39 66

Missed Math Proficiency, MetReading

72 327

Met Participation 281 655

Missed Participation 36 153

Missed ONLY Participation 9 31

Missed Attendance 1 65

Missed Graduation 6 27

Missed ONLY Attendance and/orGraduation

0 38

20




9 71


197 344


39 66


72 327







0 38

21




9 71


197 344


39 66


72 327







0 38

22




9 71


197 344


39 66


72 327







0 38

23

AYP & Students with Disabilities

Total Missing AYP

Missed Solely

Because of SWD

Schools 808 42 / 5.2%

Districts 317 180 / 56.8%

24

AYP & Limited English Proficient Students

Total Missing AYP

Missed Solely

Because of LEP

Schools 808 1 / 0.1%

Districts 317 2 / 0.6%

25

Measures Employed to Increase Validity and Reliability

• Safe Harbor

• Averaging

• Other

26

AYP: Impact of Safe Harbor

Total Meeting AYP

Met Because of Safe Harbor

Schools 2,407 52 / 2.2%

Districts 292 19 / 6.5%

27

AYP: Impact of Averaging

Total Meeting AYP

Met Because of Averaging

Schools 2,407 116 / 4.8%

Districts 292 16 / 5.5%

28

Other Measures

• Tests of statistical significance

• Confidence intervals

• Minimum N

29

Conclusions

• 2002-03 AYP impact was lowest of pre-implementation estimates

• Participation was not an issue

• Need better understanding of false positives / false negatives

30

Conclusions (continued)

• Volatility of results needs careful attention

• Consequential validity is the “main event”

ohio’s experience with ayp

Documents