secondary and postsecondary indicators perkins v methodology … · 2019. 9. 6. · simulated rates...
TRANSCRIPT
Sept. 6, 2019
Secondary and Postsecondary Indicators
Perkins V Methodology for Setting Performance LevelsPresenters:Tabatha Ries-Miller, Minnesota StateKari-Ann Ediger, Minnesota Department of Education
2
Introduction
3
• “State-determined levels of performance”
• Set by the State
– consultation and public comment
• All 4-years set in the 2020 state plan
• Requires “meaningful progress” yearly
• Set using “valid and reliable data”
• Revisions before 3rd year
• Revise at the end of any year if unanticipated circumstances, or changes in data or approaches
• Still required to meet 90%
Perkins V Transformation
4
• WIOA /Alternative Regression Model
– Isolate “unique impact” of program
– Time consuming –data collection and processing
– Complicated and difficult to explain
• Trend Extrapolation
– Presumes continued “normal” growth – business as usual
• Statistical Significance (Recommended)
– Evaluates if differences are large enough that we can conclude they are not just random variation
Exploring Methodologies
5
What is the relationship between state and local performance levels?
At what rate should we expect to see improvement?
How much of an increase is “meaningful progress” and also reasonably achievable?
Standard Deviation Method: Factors to Consider
6
4-year State Performance Level Methodology
7
Historical trend data
Cohort 2011-2013 2012-2014 2013-2015 2014-2016 2015-2017 2016-2018
Actual 45.9% 46.4% 47.4% 49.3% 49.7% 49.9%
Historical test
2013-2015(Baseline)
2017-2019Perf. Level
2017-2019Actual
2017-2019 90% Required
47.4% 50.7% 50.1% 45.6%
How much to increase? Exploring Historical Data for Reasonability
Standard Deviation = 1.6% X 2 = 3.3%
Example 2P1 (Earned postsecondary credential)
8
Baseline: 13.9%
Standard Deviation: 0.4%
4-year level: 0.8% increase over baseline
2024 Level: 14.7%
Baseline: 50.1%
Standard Deviation: 1.4%
4-year level: 2.8% increase over baseline
2024 Level: 52.9%
Baseline: 90.8%
Standard Deviation: 1%
4-year level: 2% increase over baseline
2024 Level: 92.9%
1P1: Postsecondary Retention and Placement
2P1: Earned Recognized Postsecondary Credential
3P1: Nontraditional Program Enrollment
Post-Secondary Indicators: Two Standard Deviations
9
12.5%
25%
50%
0%
25%
50%
75%
100%
0%
40%
70%
90%100%
Baseline Year 1 Year 2 Year 3 Year 4
Perc
ent
of
targ
et
Conservative Linear Aggressive
Three options:1. Conservative start with
more growth in grant year 3 and 4
2. Linear growth – equal growth each year
3. Aggressive start with more growth in grant year 1 and 2
Recommend: Conservative Start
How fast to grow? Exploring Trend Options
• Postsecondary cohort model – data lag
• Any change takes time to realize
• We can adjust methodology after 2nd year if needed before
Possible adjustment
10
All 4-year performance levels are set using two standard deviations. The standard deviation was calculated using simulated rates for 2012-2017 entry cohorts.
Recommended Postsecondary State Levels of Performance
Indicator Baseline Year
BaselineRate
4-year Increase
Year 1 Year 2 Year 3 Year 4
1P1: Postsecondary Retention and Placement
2016-2018
90.85% 2.05% 91.10% 91.36% 91.87% 92.90%
2P1: Earned Recognized Postsecondary Credential
2017-2019
50.06% 2.80% 50.41% 50.76% 51.47% 52.87%
3P1: Nontraditional Program Enrollment
2017-2019
13.87% 0.82% 13.98% 14.08% 14.28% 14.70%
11
The 4-year performance level for 1S1 and 1S2 are set using a two standard deviation increase over the baseline, while 2S1 and 2S2 was set using a one standard deviation increase. The standard deviation for 1S1 was based on 2011-2018 data, while 1S2, 2S1, and 2S2 was based on 2015-2018 data.
Recommended Secondary State Levels of Performance
Indicator Baseline Year
BaselineRate
4-year Increase
Year 1 Year 2 Year 3 Year 4
1S1: Four-year Graduation Rate
2018 91.50% 2.42% 91.80% 92.11% 92.71% 93.93%
1S2: Extended-yearGraduation Rate
2018 93.06% 1.70% 93.27% 93.48% 93.91% 94.75%
2S1: Academic Proficiency: Reading/Language Arts
2018 56.06% 2.35% 56.35% 56.65% 57.23% 58.41%
2S2: Academic Proficiency: Mathematics
2018 45.25% 1.19% 45.40% 45.55% 45.84% 46.44%
12
The 4-year performance level for 4S1 is set using a two standard deviation increase over the baseline, 5S2 was set using a one standard deviation increase and 5S4 was set using a ½ standard deviation. The standard deviation for 4S1, 5S2, and 5S4 were based on 2015-2018 data.
Recommended Secondary State Levels of Performance
Indicator Baseline Year
BaselineRate
4-year Increase
Year 1 Year 2 Year 3 Year 4
3S1: Post-Program Placement
TBD
4S1: Non-traditional Program Concentration
2018 37.57% 1.57 % 37.77% 37.96% 38.35% 39.14%
5S2: Program Quality: Postsecondary Credit
2018 38.94% 2.35% 39.23% 39.53% 40.11% 41.29%
5S4: Program Quality: TSA/IRC
2018 67.35% 2.60% 67.68% 68.00% 68.65% 69.95%
13
Questions or Feedback?
State Levels of Performance
14
Local Level of Performance Methodology
15
• Meaningful progress every year
• Aggregate must at least equal state performance level
• Take into account differences among locals
• Reasonable based on historical differences
• Acknowledge that not all locals will reach state determined performance level
– Some local’s 2017 performance was lower than 2013, so they would not have met 0% increase
Local Methodological Parameters
16
Local Levels: Equal to Opportunity to Improve
Successful Outcomes
State’s Total Concentrators
Opportunity to Improve State’s Total
Opportunity to Improve
Each Consortium’s Opportunity
State’s 4-year Increase (E.g. 2%
Increase)
Consortium’s 4-year increase
17
Measure State Baseline
Recommended Methodology
State 4-year Increase
State 4-year Perf. Level
Ranges of Local Rate Increases
Range of Local N Increases
Estimate of % Locals Success (90%)
1P1: Postsecondary Retention and Placement
90.85% 2.05% 92.90% 0.84% to 6.42%
0 to 26 97%
2P1: Earned Recognized Postsecondary Credential
50.06% 2.80% 52.87% 1.20% to 4.28%
0 to 49 83%
3P1: Nontraditional Program Enrollment
13.87% 0.82% 14.70% 0.66% to 0.89%
0 to 11 83%
How do the numbers shake out?Postsecondary Indicators
18
Measure State Baseline
Recommended Methodology
State 4-year Increase
State 4-year Perf. Level
Ranges of Local Rate Increases
Range of Local N Increases
Estimate of % Locals Success (90%)
1S1: Four-year Graduation Rate
91.50% 2.42% 93.93% 0.73% to 5.18%
1 to 89 100%
1S2: Extended-yearGraduation Rate
93.06% 1.70% 94.75% 0.82% to 5.20%
3 to 73 100%
2S1: Academic Proficiency: Reading/Language Arts
56.06% 2.35% 58.41% 1.75% to 3.41%
2 to 75 62%
2S2: Academic Proficiency: Mathematics
45.25% 1.19% 46.44% 0.97% to 1.64%
1 to 38 73%
How do the numbers shake out?Secondary Indicators
19
Measure State Baseline
Recommended Methodology
State 4-year Increase
State 4-year Perf. Level
Ranges of Local Rate Increases
Range of Local N Increases
Estimate of % Locals Success (90%)
3S1: Post-Program Placement
TBD
4S1: Non-traditional Program Concentration
37.57% 1.57 % 39.14% 1.17% to 2.34%
2 to 52 88%
5S2: Program Quality: Postsecondary Credit
38.94% 2.35% 41.29% 1.29% to 3.83%
3 to 125
88%
5S4: Program Quality: TSA/IRC
67.35% 2.60% 69.95% 0% to 6.97%
0 to 25 62%
How do the numbers shake out?Secondary Indicators
20
Questions or Feedback?
Local Levels of Performance
21
Secondary
• Kari-Ann Ediger
651-582-8269
Postsecondary
• Tabatha Ries-Miller
651-201-1848
Contact Us with Comments or Questions
22
MINNESOTA STATE AND MINNESOTA DEPARTMENT OF EDUCATION ARE AFFIRMATIVE ACTION, EQUAL OPPORTUNITY EMPLOYERS AND EDUCATORS.
30 East 7th Street, Suite 350St. Paul, MN 55101-7804
www.minnstate.edu/system/cte
1500 Highway 36 WestRoseville, MN 55113
www.education.state.mn.us/MDE/dse/cte