sierra college erik cooper. a brief history the rp conference 2013 steps project sierra college ...
TRANSCRIPT
Sierra CollegeERIK COOPER
A Brief History
The RP Conference 2013 STEPS Project Sierra College
Paul Neal & Craig Kelly
Working with the English Faculty
Logistic Regression Prototyping with Excel
The Tool
http://www.sierracollege.edu/admissions/assessment/index.php
http://bit.ly/SierraAssess
Early Results
Term Placement Success Rate
Fall 2010 All 68.2%Fall 2011 All 71.7%Fall 2012 All 73.3%Fall 2013 All 70.3%Fall 2014 Accuplacer 73.3%
HS Transcripts 78.9%Other 70.6%All 72.7%
Next Steps
Adjusting the cut score
English A placement
“Non-cognitive measures”
Math Placement
LONG BEACH CITY COLLEGEANDREW FUENMAYOR
LBCC’s Promise Pathways: Background
Promise Pathways is a first year experience program for students matriculating directly from high school
– Alternative assessment using multiple measures– Prescriptive scheduling emphasizing full-time
enrollment and early completion of basic skills courses– Priority registration– Achievement coaches and other pilot experiments
Alternative Assessment
• Longitudinal Analysis of LBUSD students revealed high school performance dramatically predicts success in college courses.
• Traditional Placement Mechanisms ignored high school performance and relied only on Standardized Assessment Exam (Accuplacer).
• New Assessment Model was devised which leveraged the combined predictive utility of multiple measures of student achievement.
Drastic Transformation of Remediation
Transfer Level Math Transfer Level English0%
10%
20%
30%
40%
50%
60%
70%
9%13%
9%14%
31%
59%
29%
37%32%
39%
F2011 LBUSD F2012 Promise PathwaysAccuplacer Only
F2012 Promise Pathways with Multiple Measures
F2013 Pathways F2014 Pathways
Performance in Transfer Level
Transfer Level Math Transfer Level English0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
49%
67%
49%
79%
Fall 2014 General Population Fall 2014 Promise Pathways
Performance by Qualification in Transfer Level English
ENGL1 Performance0%
10%
20%
30%
40%
50%
60%
70%
80%
56%
73%69%
Accuplacer Only (N: 79) Alternative and Accuplacer (N: 116) Alternative Only (N: 748)
Performance by Qualification in Transfer Level Math
ENGL1 Performance0%
10%
20%
30%
40%
50%
60% 57% 57%
47%
Accuplacer Only (N: 174) Alternative and Accuplacer (N: 114) Alternative Only (N: 466)
Achievement of Milestones in Two Years by Ethnicity: Fall 2011 LBUSD
Successfully Completed Transfer Math Successfully Completed Transfer English0%
10%
20%
30%
40%
50%
60%
70%
4%
13%
21%24%
12%
25%18%
34%
Black Asian Hispanic White
Achievement of Milestones in Two Years by Ethnicity: Promise Pathways Cohort 1
Successfully Completed Transfer Math Successfully Completed Transfer English0%
10%
20%
30%
40%
50%
60%
70%
12%
39%
26%
58%
21%
51%
36%
64%
Black Asian Hispanic White
Predictive Early Momentum Points, Using PSM to Compare Mid Term Outcomes
Intent to Complete Behavioral Intent to Transfer0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
81%
39%
85%
60%
Matched Cohort (n=785) Promise Pathways (n=785)
Significance can be quickly shared with simple t-test
Outcome t-statistic significance
Transfer Math Success 3.69 .000
Transfer English Success 11.63 .000
Achieve Intent to Complete 2.55 .011
Achieve Behavioral Intent to Transfer 8.35 .000
New Challenges Revealed
• When capable students are being placed into remediation, institution will overestimate overall success of remediation program.
• Once capable students are removed, it became very apparent that remediation was working much worse than we thought.
Bringing it all together
Implementation challenges• The missing fourth presentation
• Change management• Assessment and matriculation• Enrollment management• Support services• Faculty and staff professional development and
communication• English, Math, Counseling
Second order effects• Examination of effectiveness existing
developmental education• Enhancement of capacity for evaluation of
interventions• Reflection on long-standing processes• Galvanization of innovation
Why does this matter?
Level of first attempt, Fall 2007 students (by levels below transfer of first attempt)
Math English0%
5%
10%
15%
20%
25%
30%
35%
40%
45%
25%
39%
20%
30%27%
20%18%
9%10%
2%
Transfer 1 2 3 4
Percentage completion of transfer-level course (by level of first attempt)
Math English0%
10%
20%
30%
40%
50%
60%
70%60%
63%
18% 19%
8%11%
4% 6%2%
5%
Transfer 1 2 3 4
Among completers, average year of completion of transfer-level course (by level of first attempt)
Math English0.0
1.0
2.0
3.0
4.0
5.0
6.0
2.0 2.1
3.73.4
4.03.5
4.0 4.1
4.9
3.8
Transfer 1 2 3 4
Among completers, distribution of completions by F2007 first-time students
Math English0%
5%
10%
15%
20%
25%
30%
35%
40% 37% 36%
32%
37%
2% 3%
29%
24%
Placed in Transfer Level, First YearPlaced in Transfer Level, next 5 yearsPlaced below transfer level, first yearPlaced below transfer level, next five years
Among completers, distribution of completions by F2007 first-time students
Math English0%
5%
10%
15%
20%
25%
30%
35%
40% 37% 36%
32%
37%
31%
27%
Placed in Transfer Level, First YearPlaced in Transfer Level, next 5 yearsEveryone else
Middle of road statewide projection: MMAP
Math English0%
10%
20%
30%
40%
50%
60%
70%
29%
48%
41%
63%
Traditional PlacementMultiple Measures
Perc
en
t p
laced
at
tran
sfe
r le
vel
For much, much more, on MMAP join us tomorrow at 10:10 in the beautiful Fresno room.
What that means: all studentsMath: Irvine English: Anaheim
What that means: all studentsMath: Irvine English: Anaheim
What that means each year: first time studentsMath: AT&T Park English: Dodger Stadium
What that means each year: first time studentsMath: AT&T Park English: Dodger Stadium
What does that mean for students?
• Average reduction of development education of 1-2 semesters/student• Direct costs to student
• Direct costs to state
• Potential reduction in opportunity costs of college
• Fewer students met at our front doors to be told they don’t belong in college
What might this mean for all of us?
• The worst financial recession in our lifetimes took a million people out of the workforce for a year or more in California, causing suffering on an epic scale.
• What is the scale of the human and economic impacts of 1-2 semesters per student for two million students….
Contact information• John Hetts• [email protected], 916-498-8980
• Janet Fulks• [email protected]
• Erik Cooper• [email protected]
• Andrew Fuenmayor• [email protected]