the hitchhiker’s guide to guided pathways€¦ · • research base, predictive analytics,...

The Hitchhiker’s Guide to Guided PathwaysRedesigning Community CollegesBakersfield College

Craig HaywardJohn HettsTerrence Willett

http://bit.ly/MMAPPathways 1

2

EngagementDo you know your mascots?

3

• Guided pathways: Meaningful metaphors

• Clear the brush & set the stage

• Fire-tested Pathways Practices• Multiple Measures

• Acceleration• English acceleration & curricular redesign

• Statistics pathways

• Co-requisite acceleration

• Q&A

• Academic Momentum & Academic Velocity

• Acceleration

• Institutional throughput rate

• Pathways completion cost

• Academic Momentum & Academic Velocity• Momentum helps to transition, persist or cross a divide

• Velocity: Progress along a pathway over time toward a goal

• Guided pathway: A clear sequence of courses leading to a degree or certificate

• Lesson of the Basic Skills Cohort Progress Tracker

• Structure affects progression

• The design of a model pathway calls for a “light” touch• It’s not just about earning units, it’s about earning the right units

Importance of institutional throughput rate

Importance of institutional throughput rate

Clearing the brush/building the trailhead

• Successful pathways through college requires trailheads that are clear and easy to access.

• Important to:• Clear unnecessary obstacles

• Beware barriers to entry

• Maintain trailhead

• Build from existing (and functional) pathways

Trying to avoid trailheads that feel like

Dimrill Stairs and the Bridge of Khazad Dum

Grand Staircase of Hogwarts

Want to build trailheads like:

• Limpy Creek Trailhead• Well-integrated with point of

access

• Clear and direct path to trail

• Ample maps, guidance, and information

• Multiple types of optional support structures available for those who need it

Clearing the brushLessons from Long Beach Promise Pathways

• Examined predictive utility of wide range of high school achievement data for predicting:

• How students are assessed and placed

• How students perform in those classes

• (and alignment between them)

Alignment in English

* p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10

1.34x

.00

.30**

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

CST ELA (z) Eng Grade(12)

GPA (other)

Ord

inal

Re

gre

ssio

n C

oe

ffic

ien

ts

Predicting Placement

.17*

.37***

.88x

0.0

0.2

0.4

0.6

0.8

1.0

CST ELA (z) Eng Grade(12)

GPA (other)

Logi

stic

Re

gre

ssio

n C

oe

ffic

ien

ts

Predicting Performance

Alignment in Math

* p <.05 **, p <.01, *** p<.001, x = p< 1 x 10-10

.75x

.20

.000.0

0.2

0.4

0.6

0.8

1.0

CST Math (z) Last MathGrade

HSGPA

Ord

inal

Re

gre

ssio

n C

oe

ffic

ien

ts

Predicting Placement

.20*.25**

.73x

0.0

0.2

0.4

0.6

0.8

1.0

CST Math (z) Last MathGrade

HSGPA

Logi

stic

Re

gre

ssio

n C

oef

fici

en

ts

Predicting Performance

• Assessment should predict how students will perform at our colleges

• Instead:

• Previous standardized tests predict later standardized tests

• Previous classroom performance predicts later classroom performance

• More information tells us more about student capacity than less information

Re-imagined student capacity

• Starting in Fall 2012, students from LBUSD were provided an alternative assessment

• (now 6 districts covering >30 high schools and growing) )

• Reverse engineered analysis to place students using:• Overall HSGPA• Last high school course in discipline• Grade in last course in discipline• Last standardized test in discipline (and level)

• Placed students in highest course where predicted success rate higher than average success rate for that course.

• Built semester plans with those placements and courses pre-populated

Implementing Multiple Measures Placement:LBCC Transfer-level Placement Rates

11%7%

13%9%

14%9%

60%

31%

0%

10%

20%

30%

40%

50%

60%

70%

Transfer Level English Transfer Level Math

F2011 First time students

F2011 LBUSD

F2012 Promise Pathways -Accuplacer Only

F2012 Promise Pathways -Multiple Measures

Not just opening the gates:Success rates in transfer-level courses by entry type

64%

55%62%

51%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

English Math

Cohort One, F2012

Non-Pathways Promise Pathways

Neither of these differences approach significance, p >.30

67%

49%

79%

49%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

English Math

Cohort Three: F2014

Non-Pathways Promise Pathways

English difference, p < .001

Equity impact LBCC: F2011 Baseline Equity Gaps for 2-year rates of achievement

4%

13%2%

15%12%

25%

3%

32%

21%24%

1%

33%

18%

34%

6%

41%

0%

10%

20%

30%

40%

50%

60%

70%

Transfer Math SuccessfulCompletion

Transfer English SuccessfulCompletion

English 3 SuccessfulCompletion

Behavioral Intent toTransfer

F11 African Americans F11 Hispanic F11 Asian F11 White

Equity impact LBCC: F2012 2-year rates of achievement

12%

39%

18%

42%

21%

51%

17%

52%

26%

58%

23%

59%

36%

64%

28%

66%

0%

10%

20%

30%

40%

50%

60%

70%

Transfer Math SuccessfulCompletion

Transfer English SuccessfulCompletion

English 3 Success Behavioral Intent toTransfer

F12 African American F12 Hispanic F12 Asian F12 White

Multiple MeasuresSTEPS to MMAP

𝑦 = 𝑓 𝑥

• 2008: Hewlett Foundation funded study of high school to college transition with

CalPASS statewide data set indicating predictive utility of high school data

http://bit.ly/WIllett2008

• 2011: Long Beach City College utilizes CalPASS data to redesign placement and

develop replication infrastructure http://www.lbcc.edu/PromisePathways/

• 2014: Student Transcript Enhanced Placement System (STEPS) replication of

LBCC research with 12 additional colleges http://bit.ly/RPSTEPS

• 2014: Bakersfield College and Sierra College began similar implementation

http://bit.ly/RPMMEarly

• 2014: MMAP Statewide Research & local replications: http://bit.ly/MMAP2015

• 2015: MMAP Pilot colleges: http://bit.ly/MMAPPilot

• Examination of HS achievement for predictors of successful completion of English & math

• Focus on predictive validity (success in course) and improving completion of sequence or throughput

• Integration with the Common Assessment Initiative

• Statewide support• Research base, predictive analytics, decision tree models• Pilot colleges and faculty/staff engagement

• Webinars, convenings/summits, professional development• K-12 outreach and data population• Data warehouse and tool development

• http://bit.ly/MMAP2015

• Colleges continue to join the project and enthusiastically inquire about participating

• 41 pilot colleges now committed, 8 more at various stages of exploration, representing more than:

• >900,000 community college students

• >40% of community college students statewide

• >8% of all community college students nationally

• 11 had pilots in place in Fall 2015

• 10 additional colleges are already matching for Spring 2016

• English• Cumulative HS GPA• Grade in last HS English

• C+ or better in AP English class

• Score on English CST

• Non-remedial status in HS English

• Math• Cumulative HS GPA• Enrollment and grades in Geometry, Algebra II,

Trigonometry, Pre-calculus, Statistics, Calculus

• Taking a more challenging CST

• Score on math CST• Delay*

English Level Rule

Transfer HS 12 GPA >= 2.6

One level below Transfer

HS 12 GPA >= 2.2

AND

HS 12 English course

GP >= 1.8

Math Level Rule

College Algebra

HS GPA >=3.2 OR

HS GPA >=2.9

AND

Pre-Calculus C or better

Intermediate Algebra

HS 12 GPA >=2.9 OR

HS 12 GPA >=2.5

AND

Algebra II CST >= 302

38%31%

61%

42%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

English(n=103,510)

Math(n=143,253)

Pe

rce

nt

Tran

sfe

r Le

vel P

lace

me

nt Current Disjunctive MM

62%

72%

62%71%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Transfer-level Math Transfer-Level English

Succ

ess

ful c

om

ple

tio

n o

f tr

ansf

er-

leve

l co

urs

e

Historic success rate Projected success rate

24%30%

41%

53%

40%

51%

73% 74%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Afr Am Latino Asian White

Transfer Level English Placement

Current Disjunctive MM

15%21%

41%

51%

22%

32%

53%

65%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Afr Am Latino Asian White

Transfer Level Math Placement

Current Disjunctive MM

AccelerationResults from the California Acceleration Project

Hayward & Willett, 2014 http://bit.ly/CAPEval

• Acceleration effects were large and robust

• Acceleration worked for students of all backgrounds

• Acceleration worked for students at all placement levels

• Implementation Mattered™

35

36

1.51.2

2.3

4.5

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

All English CAPpathways

Low-accelerationEnglish pathways

High-accelerationEnglish pathways

All Math CAPpathways

Acceleration Odds Ratio (Effect Size) for English CAP Colleges

17%21%

25%30%

22%27%

32%38%

0%

20%

40%

60%

80%

100%

Starting Place4 or More

Levels Below

Starting Place3 Levels Below

Starting Place2 Levels Below

Starting Place1 Level Below

Estim

ate

d P

erc

ent

of S

tudents

Successfu

lly

Com

ple

ting T

ransfe

r-Level C

ours

e

in S

equence

English Current Level

Comparison Accelerated

Marginal means for the percentage of students completing transfer-level English for

accelerated and comparison sequences by current level. McFadden’s pseudo-R2 = 0.15

Regression Estimated Effects – Not Raw Throughputs

6%10%

15%23%21%

30%

41%

53%

0%

20%

40%

60%

80%

100%

Starting Place4 or More

Levels Below

Starting Place3 Levels Below

Starting Place2 Levels Below

Starting Place1 Level Below

Estim

ate

d P

erc

ent

of S

tudents

Successfu

lly

Com

ple

ting T

ransfe

r-Level C

ours

e

in S

equence

Math Starting Place

Comparison Accelerated

Marginal means for the percentage of students completing transfer-level math for accelerated

and comparison sequences by current level. McFadden’s pseudo-R2 = 0.14

Regression Estimated Effects – Not Raw Throughputs

48%

69%

58%

70%

23%

60%

0%

10%

20%

30%

40%

50%

60%

70%

80%

WR 201 & 301 EXP 389

Throughput in traditional English sequence vs. accelerated: IVC fall 2012 - fall 2014

Overall rate Asian Americans African Americans

Co-requisite AccelerationCompelling results from across the country

• Coleman, 2015 http://bit.ly/2015ALP

• CCA, 2016 http://bit.ly/CCACoreq

• For students placed one level below in English, the Accelerated Learning Program (ALP) model involves:

• Enrollment directly in college-level English (mainstreamed)

• Concurrent enrollment in just-in-time companion developmental English course taught by same instructor

• Coleman (2015) reviewed the results of four early implementers outside CCBC at or near institutional scale

• CCA (2016) reviews results of corequisite efforts at or near statewide scale

Coleman, 2015: Completion of College-Level English

36% 34% 37%

50%

78% 78%

62%

78%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

College 1 College 2 College 3 College 4

Pe

rce

nt

succ

ess

fully

co

mp

leti

ng

tr

ansf

er

leve

l

Baseline ALP Model

Among those enrolled in one-level below course.

Coleman, 2015: Completion of College-Level English

25%29%

46%42%

37%

55%

70%66%

76%80% 82%

76%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

College 1 College 2 College 4

Pe

rce

nt

succ

ess

fully

co

mp

leti

ng

tr

ansf

er

leve

l

Baseline B/H Baseline W ALP B/H ALP W

College 3 Omitted from chart due to small sample size (14)for B/H

CCA, 2016: Gateway course completion at at public two-year colleges

22%

14% 12%

63% 61%

0%

10%

20%

30%

40%

50%

60%

70%

NationalAverage

West Virginia Tennessee

Successful Completion of Transfer-Level Course: Math

Pre-reform (2 years) Co-Requisite (1 semester)

22%

37%31%

68%64%

0%

10%

20%

30%

40%

50%

60%

70%

NationalAverage

West Virginia Tennessee

Successful Completion of Transfer-Level Course: English

Pre-reform (2 years) Co-Requisite (1 semester)

Among students enrolling in remediation.

Increasing Access to Transfer-Level Courses• Henson & Hern, 2014 http://bit.ly/LetThemIn

• Kalamkarian, Raufman, & Edgecombe, 2015 http://bit.ly/Kalamkarian2015

• Rodriguez, 2014 http://bit.ly/Rodriguez2014

Natural experiment at Butte College

• In 2011, switched from one placement test to another

• Old test/cut scores:• 23% of incoming students

“college ready” in English

• New test/cut scores:• 48% of incoming students

“college ready” in English

8%

17%

13%

23%23%

35%

27%

37%

0%

5%

10%

15%

20%

25%

30%

35%

40%

AfricanAmerican

AsianAmerican

Hispanic White

Pe

rce

nt

succ

ess

fully

co

mp

leti

ng

tra

nsf

er

leve

l in

fir

st y

ear

F2010 F2012

Developmental Math Reform – Virginia Community College System

• Intentionally increased percentage assigned to college-level math

• (Also, introduced new assessment instrument, redesigned remedial math into modular setup, increased alignment of math to educational goals)

19%

8%

43%

18%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

Placement into CollegeMath

Completion of College Mathin 1 year

Pre-Reform, Fall 2010 Post-Reform, Fall 2012

VCCS Combination of Increased Access and Corequisite Expansion

25%37%

3%

11%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Pre-reform, F2010 Post-reform, F2013

Completion of College English in first year

College English Co-RequisiteCollege English

43%58%

10%

23%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Pre-reform, F2010 Post-reform, F2013

Placement into College English

College English Co-RequisiteCollege English

• Two to five times transfer-level course completion

• Comparable or higher success rates

• Works across demographic groups & placement levels

• Tremendous equity implications

• Evidence-based assessment, placement and redesign

of development education provides a true on-ramp

into college programs and college-level work

ICOE

• These strategies save students 1-2 semesters of developmental education on average

• Direct costs• $200-$250 per course for student (~$50/unit +books!)• $800-$1000 per course for state (~$200/unit NR fees)

• Opportunity costs even higher• Median 2012 salary of “some college” is ~$30,000/year• Students don’t lose first or median year, they lose either:

• their last year of salary or• the opportunity to retire earlier.

Opportunity to change the future of the California Community Colleges

Sense of Scale• According to the BLS, the

Great Recession of 2008 took ~1,000,000 out of the California workforce for a year or more.

• 2.4 million California community college students have lost up to an additional year of time out of the workforce and/or have become less likely to complete their education

Fierce Urgency of Now• ~500,000 new community college

students in California every year

• “We are now faced with the fact that tomorrow is today. We are confronted with the fierce urgency of now. In this unfolding conundrum of life and history, there "is" such a thing as being too late. This is no time for apathy or complacency. This is a time for vigorous and positive action.”

• Dr. Martin Luther King, Jr.

Thank you!

Terrence WillettThe RP Group twillett@rpgroup.org

Craig HaywardThe RP Groupchayward@rpgroup.org

Mallory NewellThe RP Groupnewellmallory@deanza.edu

John HettsEducational Results Partnershipjhetts@edresults.org(714-380-2678)

Ken SoreyEducational Results Partnershipken@edresults.org

Daniel LamoreeEducational Results Partnershipdlamoree@edresults.org