ohio’s experience with ayp
Post on 04-Jan-2016
72 Views
Preview:
DESCRIPTION
TRANSCRIPT
Ohio’s Experience with AYP
Presentation to the American Educational Research Association
Mitchell D. Chester
April 12, 2004
2
Pre-Implementation Concerns
• Over-identification
• Accountability for students with disabilities
• Volatility of results
3
Presentation Outline
• Ohio’s accountability system
• 2002-03 AYP results
• Measures employed to increase validity and reliability
• Conclusions
4
Ohio’s Accountability System
• School and district rating criteria
• 2002-03 results
5
Ohio Rating Criteria
94% to 100%
(17 or 18 for districts)
94% to 100%(17 or 18 for districts)
75% to 93%(14 to 16 for districts)
75% to 100%(14 to 16 for districts)
0% to 74%(0 to 13 for districts)
50% to 74%(9 to 13 for districts)
31% to 49%(6 to 8 for districts)
0% to 31%(0 to 5 for districts)
Met AYPandor
Excellent
Performance Indicators
Missed AYPor
Missed AYP
Missed AYP
Missed AYP
and
and
and
Missed AYP
Met AYP
and
and
and
and
or
or
Federal AYP Requirements
Value Added -- Once grades 3 to 8 reading and
math assessments are implemented and multiple
years of data available, Ohio will incorporate a measure of individual student grade-to-grade achievement gains
to help determine school building and district des
Performance Index Score Growth Calculation
100 to 120
100 to 120
90 to 99 Met AYP
Effective
or
or
or
90 to 99
80 to 89
70 to 79
0 to 89Continuous
Improvement
or
Temporary Growth Calculation -- Districts and schools will move from
Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c)
most recent year'
and
Academic Watch
Academic Emergency and 0 to 69
6
Ohio Rating Criteria
94% to 100%
(17 or 18 for districts)
94% to 100%(17 or 18 for districts)
75% to 93%(14 to 16 for districts)
75% to 100%(14 to 16 for districts)
0% to 74%(0 to 13 for districts)
50% to 74%(9 to 13 for districts)
31% to 49%(6 to 8 for districts)
0% to 31%(0 to 5 for districts)
Temporary Growth Calculation -- Districts and schools will move from
Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c)
most recent year'
and
Academic Watch
Academic Emergency and 0 to 69
Effective
or
or
or
90 to 99
80 to 89
70 to 79
0 to 89Continuous
Improvement
or
or
or
Federal AYP Requirements
Value Added -- Once grades 3 to 8 reading and
math assessments are implemented and multiple
years of data available, Ohio will incorporate a measure of individual student grade-to-grade achievement gains
to help determine school building and district des
Performance Index Score Growth Calculation
100 to 120
100 to 120
90 to 99 Met AYP
Missed AYP
Met AYP
and
and
and
and
Missed AYP
Missed AYP
Missed AYP
and
and
and
Met AYPandor
Excellent
Performance Indicators
Missed AYPor
7
Ohio Rating Criteria
94% to 100%
(17 or 18 for districts)
94% to 100%(17 or 18 for districts)
75% to 93%(14 to 16 for districts)
75% to 100%(14 to 16 for districts)
0% to 74%(0 to 13 for districts)
50% to 74%(9 to 13 for districts)
31% to 49%(6 to 8 for districts)
0% to 31%(0 to 5 for districts)
Temporary Growth Calculation -- Districts and schools will move from
Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c)
most recent year'
and
Academic Watch
Academic Emergency and 0 to 69
Effective
or
or
or
90 to 99
80 to 89
70 to 79
0 to 89Continuous
Improvement
or
or
or
Federal AYP Requirements
Value Added -- Once grades 3 to 8 reading and
math assessments are implemented and multiple
years of data available, Ohio will incorporate a measure of individual student grade-to-grade achievement gains
to help determine school building and district des
Performance Index Score Growth Calculation
100 to 120
100 to 120
90 to 99 Met AYP
Missed AYP
Met AYP
and
and
and
and
Missed AYP
Missed AYP
Missed AYP
and
and
and
Met AYPandor
Excellent
Performance Indicators
Missed AYPor
8
Ohio Rating Criteria
94% to 100%
(17 or 18 for districts)
94% to 100%(17 or 18 for districts)
75% to 93%(14 to 16 for districts)
75% to 100%(14 to 16 for districts)
0% to 74%(0 to 13 for districts)
50% to 74%(9 to 13 for districts)
31% to 49%(6 to 8 for districts)
0% to 31%(0 to 5 for districts)
Temporary Growth Calculation -- Districts and schools will move from
Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c)
most recent year'
and
Academic Watch
Academic Emergency and 0 to 69
Effective
or
or
or
90 to 99
80 to 89
70 to 79
0 to 89Continuous
Improvement
or
or
or
Federal AYP Requirements
Value Added -- Once grades 3 to 8 reading and
math assessments are implemented and multiple
years of data available, Ohio will incorporate a measure of individual student grade-to-grade achievement gains
to help determine school building and district des
Performance Index Score Growth Calculation
100 to 120
100 to 120
90 to 99 Met AYP
Missed AYP
Met AYP
and
and
and
and
Missed AYP
Missed AYP
Missed AYP
and
and
and
Met AYPandor
Excellent
Performance Indicators
Missed AYPor
9
Ohio Rating Criteria
94% to 100%
(17 or 18 for districts)
94% to 100%(17 or 18 for districts)
75% to 93%(14 to 16 for districts)
75% to 100%(14 to 16 for districts)
0% to 74%(0 to 13 for districts)
50% to 74%(9 to 13 for districts)
31% to 49%(6 to 8 for districts)
0% to 31%(0 to 5 for districts)
Temporary Growth Calculation -- Districts and schools will move from
Emergency to Watch or from Watch to Improvement if: (a) improved performance index score each of past two years, (b) total two-year gain of at least 10 points, and (c) most recent year's gain of at
least 3 points.
and
Academic Watch
Academic Emergency and 0 to 69
Effective
or
or
or
90 to 99
80 to 89
70 to 79
0 to 89Continuous
Improvement
or
or
or
Federal AYP Requirements
Value Added -- Once grades 3 to 8 reading and
math assessments are implemented and multiple
years of data available, Ohio will incorporate a measure of individual student grade-to-grade achievement gains
to help determine school building and district
designations.
Performance Index Score Growth Calculation
100 to 120
100 to 120
90 to 99 Met AYP
Missed AYP
Met AYP
and
and
and
and
Missed AYP
Missed AYP
Missed AYP
and
and
and
Met AYPandor
Excellent
Performance Indicators
Missed AYPor
10
District Designations
Total
Excellent 85 85 0
Effective 178 149 29
Continuous Improvement
278 255 19 3 5
Academic Watch
52 36 15 1
Academic Emergency
16 16
Not Rated 4
Performance Indicators
Only
Performance Index
Performance Index Growth
Meeting AYP
11
School Designations
Total
Excellent 633 621 12
Effective 765 496 269
Continuous Improvement
1,242 360 532 131 806
Academic Watch
237 125 77 39
Academic Emergency
338 338
Not Rated 500
Performance Index
Performance Index Growth
Meeting AYPPerformance
Indicators Only
12
2002-03 AYP Results
• AYP within rating categories
• School Improvement within rating categories
• Disentangling the AYP categories
13
AYP within Ohio’s Rating Categories
Number Percent Number PercentIn School
Improvement
Excellent 85 18 21.2% 633 36 5.7% 0
Effective 178 65 36.5% 765 45 5.9% 0
Continuous Improvement
278 166 59.7% 1,242 152 12.2% 35
Academic Watch
52 52 100.0% 237 237 100.0% 36
Academic Emergency
16 16 100.0% 338 338 100.0% 117
Not Rated 4 0 0.0% 500 0 0.0% 3
TOTAL 609 317 52.1% 3,215 808 25.1% 191
SchoolsDistrictsMissing AYP Missing AYP
Total Total
14
AYP within Ohio’s Rating Categories
Number Percent Number PercentIn School
Improvement
Excellent 85 18 21.2% 633 36 5.7% 0
Effective 178 65 36.5% 765 45 5.9% 0
Continuous Improvement
278 166 59.7% 1,242 152 12.2% 35
Academic Watch
52 52 100.0% 237 237 100.0% 36
Academic Emergency
16 16 100.0% 338 338 100.0% 117
Not Rated 4 0 0.0% 500 0 0.0% 3
TOTAL 609 317 52.1% 3,215 808 25.1% 191
SchoolsDistrictsMissing AYP Missing AYP
Total Total
15
AYP within Ohio’s Rating Categories
Number Percent Number PercentIn School
Improvement
Excellent 85 18 21.2% 633 36 5.7% 0
Effective 178 65 36.5% 765 45 5.9% 0
Continuous Improvement
278 166 59.7% 1,242 152 12.2% 35
Academic Watch
52 52 100.0% 237 237 100.0% 36
Academic Emergency
16 16 100.0% 338 338 100.0% 117
Not Rated 4 0 0.0% 500 0 0.0% 3
TOTAL 609 317 52.1% 3,215 808 25.1% 191
SchoolsDistrictsMissing AYP Missing AYP
Total Total
16
AYP within Ohio’s Rating Categories
Number Percent Number PercentIn School
Improvement
Excellent 85 18 21.2% 633 36 5.7% 0
Effective 178 65 36.5% 765 45 5.9% 0
Continuous Improvement
278 166 59.7% 1,242 152 12.2% 35
Academic Watch
52 52 100.0% 237 237 100.0% 36
Academic Emergency
16 16 100.0% 338 338 100.0% 117
Not Rated 4 0 0.0% 500 0 0.0% 3
TOTAL 609 317 52.1% 3,215 808 25.1% 191
SchoolsDistrictsMissing AYP Missing AYP
Total Total
17
AYP within Ohio’s Rating Categories
Number Percent Number PercentIn School
Improvement
Excellent 85 18 21.2% 633 36 5.7% 0
Effective 178 65 36.5% 765 45 5.9% 0
Continuous Improvement
278 166 59.7% 1,242 152 12.2% 35
Academic Watch
52 52 100.0% 237 237 100.0% 36
Academic Emergency
16 16 100.0% 338 338 100.0% 117
Not Rated 4 0 0.0% 500 0 0.0% 3
TOTAL 609 317 52.1% 3,215 808 25.1% 191
SchoolsDistrictsMissing AYP Missing AYP
Total Total
18
School Improvement within Ohio’s Rating Categories
Number Percent Number PercentIn School
Improvement
Excellent 85 18 21.2% 633 36 5.7% 0
Effective 178 65 36.5% 765 45 5.9% 0
Continuous Improvement
278 166 59.7% 1,242 152 12.2% 35
Academic Watch
52 52 100.0% 237 237 100.0% 36
Academic Emergency
16 16 100.0% 338 338 100.0% 117
Not Rated 4 0 0.0% 500 0 0.0% 3
TOTAL 609 317 52.1% 3,215 808 25.1% 191
Missing AYP Missing AYPTotal Total
SchoolsDistricts
19
AYP CategoriesDistricts School Buildings
Total Missing AYP 317 808
Met Both Reading & MathProficiency
9 71
Missed Both Reading & MathProficiency
197 344
Missed Reading Proficiency, MetMath
39 66
Missed Math Proficiency, MetReading
72 327
Met Participation 281 655
Missed Participation 36 153
Missed ONLY Participation 9 31
Missed Attendance 1 65
Missed Graduation 6 27
Missed ONLY Attendance and/orGraduation
0 38
20
AYP CategoriesDistricts School Buildings
Total Missing AYP 317 808
Met Both Reading & MathProficiency
9 71
Missed Both Reading & MathProficiency
197 344
Missed Reading Proficiency, MetMath
39 66
Missed Math Proficiency, MetReading
72 327
Met Participation 281 655
Missed Participation 36 153
Missed ONLY Participation 9 31
Missed Attendance 1 65
Missed Graduation 6 27
Missed ONLY Attendance and/orGraduation
0 38
21
AYP CategoriesDistricts School Buildings
Total Missing AYP 317 808
Met Both Reading & MathProficiency
9 71
Missed Both Reading & MathProficiency
197 344
Missed Reading Proficiency, MetMath
39 66
Missed Math Proficiency, MetReading
72 327
Met Participation 281 655
Missed Participation 36 153
Missed ONLY Participation 9 31
Missed Attendance 1 65
Missed Graduation 6 27
Missed ONLY Attendance and/orGraduation
0 38
22
AYP CategoriesDistricts School Buildings
Total Missing AYP 317 808
Met Both Reading & MathProficiency
9 71
Missed Both Reading & MathProficiency
197 344
Missed Reading Proficiency, MetMath
39 66
Missed Math Proficiency, MetReading
72 327
Met Participation 281 655
Missed Participation 36 153
Missed ONLY Participation 9 31
Missed Attendance 1 65
Missed Graduation 6 27
Missed ONLY Attendance and/orGraduation
0 38
23
AYP & Students with Disabilities
Total Missing AYP
Missed Solely
Because of SWD
Schools 808 42 / 5.2%
Districts 317 180 / 56.8%
24
AYP & Limited English Proficient Students
Total Missing AYP
Missed Solely
Because of LEP
Schools 808 1 / 0.1%
Districts 317 2 / 0.6%
25
Measures Employed to Increase Validity and Reliability
• Safe Harbor
• Averaging
• Other
26
AYP: Impact of Safe Harbor
Total Meeting AYP
Met Because of Safe Harbor
Schools 2,407 52 / 2.2%
Districts 292 19 / 6.5%
27
AYP: Impact of Averaging
Total Meeting AYP
Met Because of Averaging
Schools 2,407 116 / 4.8%
Districts 292 16 / 5.5%
28
Other Measures
• Tests of statistical significance
• Confidence intervals
• Minimum N
29
Conclusions
• 2002-03 AYP impact was lowest of pre-implementation estimates
• Participation was not an issue
• Need better understanding of false positives / false negatives
30
Conclusions (continued)
• Volatility of results needs careful attention
• Consequential validity is the “main event”
top related