next-generation accountability designing a differentiated accountability system for michigan...
TRANSCRIPT
![Page 1: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/1.jpg)
NEXT-GENERATION ACCOUNTABILITYDesigning a Differentiated Accountability System for Michigan
Presentation to the Michigan Educational Research AssociationNovember 22, 2011
![Page 2: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/2.jpg)
The Challenge
• Design a school accountability system that:• Sets a high proficiency standard (where proficiency is
based on career and college ready standards) AND• Rewards schools for achieving growth with students,
regardless of starting point• Moves Michigan toward a higher level of preparation for
career and college• Fair and equitably applied
![Page 3: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/3.jpg)
WHERE ARE WE NOW?Assessing Michigan’s Current Situation
![Page 4: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/4.jpg)
College going rates
• Statewide: • 71% of 2008-2009 graduates enrolled in an IHE• 73% of those who enroll earn at least one year’s worth
of credits• Gives a total of 52% of 2008-09 graduates who earned
at least one year’s worth of credits
• By individual school:• Median = 63%• 25th percentile: 40%• 75th percentile: 75%
![Page 5: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/5.jpg)
0.5
11.
52
2.5
Den
sity
0 .2 .4 .6 .8 1
Percentage of Graduates Enrolled at an IHE
![Page 6: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/6.jpg)
![Page 7: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/7.jpg)
![Page 8: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/8.jpg)
Relationship between new cut scores and college going0
2040
6080
100
Per
cen
t E
nro
lled
in a
n IH
E
0 20 40 60 80 100
2008 MME Math Percent Proficient (based on new cut scores)
![Page 9: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/9.jpg)
Takeaways
• Michigan students are going to college• Even if students are not proficient on new cut scores on the MME, they are enrolling in college.
Question: Will those students be successful? Will they pursue challenging majors?
![Page 10: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/10.jpg)
Achievement Gap
• Since 2001, schools have been held accountable on overall student performance… AND the performance of the nine traditional subgroups
• Put the focus on achievement of all students, as defined by demographic characteristics
• Caveat: IF you had a sufficient number of students!
![Page 11: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/11.jpg)
Economically disadvantaged gaps: Math
![Page 12: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/12.jpg)
Ethnicity gaps: Math
![Page 13: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/13.jpg)
Economically disadvantaged gaps: Reading
![Page 14: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/14.jpg)
Ethnicity gaps: Reading
![Page 15: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/15.jpg)
Career and College Readiness in Our Schools: Math
0.0
1.0
2.0
3.0
4
Den
sity
0 20 40 60 80 100mmemath2011
50th percentile 95th percentile
![Page 16: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/16.jpg)
CCR: Reading0
.01
.02
.03
Den
sity
0 20 40 60 80 100mmeread2011
50th percentile 95th percentile
![Page 17: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/17.jpg)
Tension: Is it really important that our students be career and college ready?
• Evidence points to the importance of higher education.
• Competitive job market, especially in states like Michigan.
• Fast-growing occupations (health care, technology) require higher education/specific training
• Educational inflation
![Page 18: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/18.jpg)
NEXT GENERATION ACCOUNTABILITYCharting a New Path
![Page 19: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/19.jpg)
Necessary Components
• Focused consequences and interventions for schools most in need
• Achievement gap• Differentiated accountability = differentiated interventions
• Fair and equitable• Ambitious AND achievable goals
![Page 20: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/20.jpg)
Focused Consequences and Interventions
• Priority Schools• Lowest 5% of the Top to Bottom list• Priority schools = PLA Schools• Aligns federal and state accountability• Priority schools must enter a three year cycle of school
improvement, with the most highly targeted interventions
![Page 21: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/21.jpg)
PLA Schools: Anecdotal Evidence
• Two “cohorts” of PLA schools: 2010 and 2011.• 2010 schools: first year of implementation• 2011 schools: planning• 2010 schools:
• About half experienced an increase in percent proficient and increased their improvement rate
• More meaningful data in another 1-2 years
![Page 22: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/22.jpg)
PLA Schools: Anecdotal Evidence
• What are they saying? • ERA Unit doing PLA data visits• Being named a PLA school was extremely difficult• However, it has fundamentally altered the way the
schools are approaching achievement• Pushing a fundamental redesign• Impetus to address crucial issues• Innovative strategies (i.e. flipped learning)
![Page 23: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/23.jpg)
Achievement Gap as Central Focus
• Achievement gaps have not closed to the extent that we need
• Proposal:• Focus on the bottom 30% of students, regardless of
demographic, not the traditional subgroups• Puts the attention firmly on the lowest achieving
students• By improving that group, increase school’s overall
achievement, and improvement rate
![Page 24: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/24.jpg)
Achievement Gap
• Pros• All schools have a subgroup• At least 700 schools have no subgroup under AYP
traditional subgroups• Unmask low performance in high performing subgroups• Asks that all schools consider their lowest performing
students• Schools cannot mask low-performance with overall high
performance
![Page 25: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/25.jpg)
Achievement Gap
• Cons:• Concern that we will lose focus on demographic
subgroups.• In the lowest 30% subgroup—approximately 70% of
that group are also a member of one or more traditional subgroups.
• High-achieving schools do not like it• People think that “lowering the ceiling on our highest
achieving students” will help the schools
![Page 26: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/26.jpg)
Achievement Gap: Focus Schools
• Need to identify the schools with the largest achievement gaps.
• Using the bottom 30% subgroup, would rank the bottom 10% in terms of largest gap.
• Using traditional subgroups in a ranking (normative) setting is complicated:• Not all schools have a subgroup• Comparing schools with the same subgroup• Unfairly focuses on students with disabilities• Still allows for “masking”
![Page 27: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/27.jpg)
Achievement Gap: Our Belief
• If Michigan is serious about raising the achievement of ALL students, then the bottom 30% is the correct way to go.
• Distributing accountability to traditionally high achieving schools and asking them to achieve those same results with all students is appropriate.
• Michigan cannot leave students behind any longer.
![Page 28: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/28.jpg)
Differentiated Accountability For All Schools
• Priority schools = 5% and Focus Schools = 10%--so what about the other 85%?
• Need a more nuanced system than pass/fail AYP.• Need to integrate performance for all students, bottom 30%, and all subjects (not just reading and mathematics)
![Page 29: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/29.jpg)
Use this system to set a proficiency goal with improvement• Proficiency target = AMO
• Set for each school as the increase in percent proficient necessary for that school to reach the overall target proficiency
• Improvement target• If school does not meet proficiency target, can meet an
improvement target• Set as the increase in percent proficient demonstrated
by a high-improvement school in the base year
![Page 30: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/30.jpg)
Example with Data
• End proficiency target: 85% • School is at 10% proficient now• Need to improve 75% in 10 years, or 7.5% per year.
• Proficiency target in year 1: 17.5% proficient• If does not meet it, must have improved by 3.5% (which is improvement rate for school at the 90th percentile in base year)
![Page 31: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/31.jpg)
Bottom 30% Subgroup as Accountable Subgroup
• Only one “accountable” subgroup now (still report on nine traditional subgroups)
• Need to meet a proficiency target for the bottom 30% subgroup (unlikely…) OR the improvement target
![Page 32: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/32.jpg)
Notes on this system
• Offers differentiated AMOs by school• Keeps a clear proficiency target in the system• Proficiency target is actually an improvement target as well
• Many of the increases demanded of schools will be greater than we have historically seen, so need the improvement (safe harbor) target
![Page 33: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/33.jpg)
Questions
• What is an “ambitious and attainable” end goal? 100%? 85%? 70%? How do you determine this?
• Should meeting the target based on improvement be equivalent to meeting it based on straight proficiency?
• Should we reset each year?
![Page 34: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/34.jpg)
Participation
• Necessary to keep a firm participation target in the system
• Schools will begin to “game” on who they assess if no clear participation target
![Page 35: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/35.jpg)
“Green” SchoolSchool Name: ABC Schools
Reward
Proficienc
y Improvement Participation Overall Math All Students Yes -- Yes
Bottom 30% No Yes Reading All Students Yes -- Yes
Bottom 30% Yes -- Writing All Students Yes -- Yes
Bottom 30% No Yes Science All Students Yes -- Yes
Bottom 30% No Yes Social Studies All Students Yes -- Yes
Bottom 30% Yes -- Rate Improvement OverallGraduation Rate Yes -- OverallCompliance Educator Evaluations
![Page 36: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/36.jpg)
“Yellow” SchoolSchool Name: XYZ Schools
Proficienc
y Improvement Participation Overall Math All Students No Yes Yes
Bottom 30% No Yes Reading All Students Yes -- Yes
Bottom 30% Yes -- Writing All Students No Yes Yes
Bottom 30% No Yes Science All Students Yes -- Yes
Bottom 30% No Yes Social Studies All Students Yes -- Yes
Bottom 30% Yes -- Rate Improvement OverallGraduation Rate Yes -- OverallCompliance Educator Evaluations
![Page 37: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/37.jpg)
“Red” SchoolSchool Name: MNO Schools
Priority
Proficiency Improvement Participation Overall Math All Students No Yes Yes
Bottom 30% No No Reading All Students No No Yes
Bottom 30% No No Writing All Students No Yes No
Bottom 30% No Yes Science All Students Yes -- Yes
Bottom 30% No No Social Studies All Students No No Yes
Bottom 30% No No Rate Improvement OverallGraduation Rate No Yes OverallCompliance Educator Evaluations
![Page 38: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/38.jpg)
Rules for Colors
• Need to be green on all indicators• This makes “green” a more rare indicator; it means that there are no areas of concern
• To be red, need to be red on all five academic indicators; makes it a more rare indicator
• Yellow—largest category—can have some red, some green; is indicative of “intervention” needed; use colors within to target
• Final color is not the key determiner for consequences; priority/focus status is more critical
![Page 39: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/39.jpg)
Questions?
• How to determine the final colors?• Balancing public desire for “one” rating with internal knowledge that “one” rating is difficult.
• Other indicators that should be included?
![Page 40: NEXT-GENERATION ACCOUNTABILITY Designing a Differentiated Accountability System for Michigan Presentation to the Michigan Educational Research Association](https://reader035.vdocuments.site/reader035/viewer/2022062719/56649ee55503460f94bf44bd/html5/thumbnails/40.jpg)
Contact Information
• Venessa A. Keesler, Ph.D.• Evaluation, Research and Accountability• Office of Psychometrics, Accountability, Research and
Evaluation (OPARE)• [email protected]