cal-pass plus - home - assignment of students into ... · web viewnote: in scff, the way unique...

Methodology Changes for Student Success Metrics2.6 Release, May 2020

Table of ContentsMETHODOLOGY CHANGES FOR STUDENT SUCCESS METRICS 2.6 RELEASE, MAY 2020...................................................................................................1

ASSIGNMENT OF STUDENTS INTO STUDENT JOURNEYS...............................................................................................................................................2

DRILLDOWNS.............................................................................................................................................................................................................3

STUDENT JOURNEY METRICS......................................................................................................................................................................................4

SUCCESSFUL ENROLLMENT........................................................................................................................................................................................5

LEARNING PROGRESS METRICS..................................................................................................................................................................................6

MOMENTUM METRICS.................................................................................................................................................................................................8

SUCCESS METRICS...................................................................................................................................................................................................10

EMPLOYMENT AND EARNINGS METRICS.....................................................................................................................................................................18

1Methodology Changes for Student Success Metrics. | 2.6 Release, April 2020

Assignment of Students into Student JourneysMetric Issue 2019 Release 2020 Release Alignment or Reason for

ChangeAdjustments

Students Placement in Student Journeys

Students are only in-cluded in one student journey at the selected college in the selected academic year.

Students can be included in multiple student journeys at the selected college in the selected academic year if they have multiple goals in the most recent term or ex-hibit course taking pattern in the selected year that would place them in Short-Term Career or Adult Ed.

Allows colleges to view outcomes based on either goals or course-taking behavior

Allows colleges to view a student’s outcomes in multiple categories in cases where there is conflicting in-formation from different institu-tions

In 2019, a hierarchy was established to place students into student journeys. 1) stu-dent was placed into a student journey based on formal or informal goal at a col-lege in the selected academic year 2) if there was conflicting information in the last term for goal, then degree/transfer trumped short-term career which trumped adult ed/ESL for student journey placement 3) For any student without a goal, course taking behavior was considered to see if the stu-dent should be placed in either short-term career or adult ed/ESL, and short-term ca-reer trumped if both behaviors were dis-played 4) if neither of those course taking patterns were exhibited, then the student was placed in undecided/other

In 2020 SSM, a student can be placed in de-gree/transfer, short-term career or adult ed/ESL student journey based on goal in the last term OR based on course taking behav-ior in the selected academic year at the se-lected college

DrilldownsMetric Issue 2019 Release 2020 Release Alignment or

Reason for Change Adjustments Impacts

Perkins Economically Disadvantaged

Students considered economically disadvantaged using the Perkins defi-nition.

Same Brief Definition Better alignment with Chancellor’s Office ap-proved definition

See All Student Journey

Removed fee waivers not based on economic need

Higher values statewide but varies across colleges


Metric Issue 2019 Release 2020 Release Alignment or Reason for Change

Adjustments Impacts

Added CalWORKS data el-ements

Students with Disabili-ties

Students who were ever flagged as disabled at a California community college

Same Brief DefinitionAdded in new MIS data variables


Added in newer MIS data variables for autism and ADHD to Primary Disabil-ity data element

Higher values statewide and for nearly all colleges

Lesbian/Gay/Bisexual/Transgender

Students who ever identified as les-bian/gay/bisexual/transgender in their college application

Students who identified as lesbian/gay/bisexual/transgender in their college application for the selected college or at their selected college in the selected year

Better alignment with orig-inal intent of the metric and Added in new MIS data elements


Added in two new data el-ements for transgender and sexual orientation with data available for 2017-18 and 2018-19: SB36 Student Transgen-der and SB 37 Student Sexual Orientation

Limited timeframe and location of LGBT status to the selected institution and selected year, rather than any time

Lower values

First GenerationStudents who ever reported that none of their parents/guardians at-tended college

Same Brief Definition Better alignment with orig-inal intent of the metric

Excluded students whose parents reported that they have some college but no de-gree

Lower values


Student Journey MetricsMetric Issue 2019 Release 2020 Release Alignment or


All Student Journey

All students enrolled in the given year, excluding students who were concurrently enrolled in K12 that same year

Same Brief Defini-tion

Better alignment across dashboards in Launch-Board

In 2019, unique students were identified using a hybrid derived key that included Student ID (SB00), last name, first name, date of birth, and gender

In 2020, unique students are identified using last name, first name, date of birth, and gender, with enrollment confirmed using SB00

Higher val-ues

Adult Education/ESL Student Journey

Students who enrolled in the se-lected year with a goal of building foundational literacy, quantitative, and English-language skills


Alignment with Adult Edu-cation Pipeline 2020 re-lease


See assignment of students into student journeys

Added TOP code 493060 Elementary Education (Grades 1-8) to Adult Basic Education or ABE

Added TOP code 493062 Secondary Education (Grades 9-12) and G.E.D to Adult Secondary Edu-cation or ASE

Higher val-ues

Short-Term Career Education Student Journey

Students who enrolled in the se-lected year with a goal of building skills to enter or advance in their ca-reers


See All Student Journey See All Student Journey


Higher val-ues for about 75% of colleges

Degree/Transfer Stu-dent Journey

Students who enrolled in the se-lected year with a goal of earning a two-year and/or four-year degree

Same Brief Defini-tion See All Student Journey



Slightly lower values

Undecided/Other Student Journey

Students who enrolled in the se-lected year with a goal of life-long learning, fulfilling requirements of a four-year institution while already enrolled in a four-year institution, or those undecided as to their goal


Alignment with Adult Edu-cation Pipeline 2020 re-lease


See assignment of students into student journeysHigher val-ues for over 80% of col-leges


Successful EnrollmentMetric Issue 2019 Release 2020 Release Alignment or Reason

for Change Adjustments Impact

Successful Enrollment in the Same Commu-nity College ANDSuccessful Enrollment in any Community Col-lege

Among all applicants in the previous or selected year, the proportion who en-rolled in the same (or any) community college in the selected year

Among all applicants who in -dicated an intent to enroll in the selected year, the propor-tion who enrolled in the same (or any) community college in the selected year

Better aligned to original in-tent of the metric

The denominator now includes stu-dents who indicated an intent to ap-ply in the selected academic year, based on information provided in OPEN CCCApply

Students from international CCCAp-ply are included

Lower values

Learning Progress MetricsMetric Issue 2019 Release 2020 Release Alignment or Reason

for Change Adjustments Impacts

Skills Gains

Among all students, the per-centage who had one or more skills gains, measured by advancing one or more CB21 levels or by improving one or more educational functioning levels in the se-lected year

Same Brief Definition

Better aligned to original in-tent of the metric and align-ment with Adult Education Pipeline 2020 release


Denominator has been changed to only include students who took a basic skills course(s): [SXD2 IN (S, B)]

Skills gains must be earned in the same discipline area (Math or English)

Denominators are lower for all colleges, and values are higher for some while percentages are higher for most

In 2017-18, Statewide per-centage went from 4% to 13%

Course Success RateAmong enrollments by stu-dents, the course success rate in the selected year

Same Brief Definition Better aligned to Datamart


Denominator has been updated to align with Datamart by ex-cluding additional grades (EW, MW, RD, SP, UG, and UD)

Slightly lower

Completed Transfer Level Math and Eng-lish

Among all students, the pro-portion who completed both transfer-level math and English in their first aca-demic year of credit enroll-ment within the district

Same Brief Definition Alignment with 2019-20 SCFF and better alignment across dashboards in LaunchBoard


To align to the 2019-20 SCFF, the First File is no longer used to determine the first-time a student had a credit enrollment as a non-special admit student

Values and denominators are higher, and percent-ages are lower

In 2017-18, Statewide per-centage went from 8% to



Adjustments Impacts

Note: In the Student Centered Funding Formula (SCFF), the way unique students are identi-fied is slightly different. In SCFF, Student ID (SB00) counts one student with multiple Stu-dent ID Statuses (SB01) as mul-tiple students. In SSM, these counts are deduplicated.

7%

At the district level, the number of students attain-ing the metric is slightly lower on SSM when com-paring to SCFF

Completed Transfer Level Math

Among all students, the pro-portion who completed transfer-level math in their first academic year of credit enrollment within the dis-trict

Same Brief DefinitionAlignment with 2019-20 SCFF and better alignment across dashboards in LaunchBoard


See Completed Transfer Level Math and English

Values and denominators are higher. Percentages are higher for about one-third of the colleges

In 2017-18, the statewide percentage remained con-stant at 11%

Completed Transfer Level English

Among all students, the pro-portion who completed transfer-level English in their first academic year of credit enrollment within the district

Same Brief DefinitionAlignment with 2019-20 SCFF and better alignment across dashboards in LaunchBoard


See Completed Transfer Level Math and English

Values and denominators are higher, and percent-ages are lower

In 2017-18, the statewide percentage went from 27% to 21%


Momentum MetricsMetric Issue 2019 Release 2020 Release Alignment or Reason

for Change Adjustments Impacts

Completed a Level of Education

Among all students who were enrolled in noncredit adult basic education or ESL in the selected year, the proportion who com-pleted one or more levels of adult education by tran-sitioning to adult sec-ondary education in the se-lected year or subsequent year


Alignment with Adult Educa-tion Pipeline 2020 release


In the numerator, added TOP code 493062 Secondary Education (Grades 9-12) and G.E.D has been added to Adult Secondary Education

In the denominator, added TOP code 496060 Grades 1-8) has been added to Adult Basic Education for the denomi-nator

Aligned definition for noncredit stu-dents to Adult Education April 2020 re-lease

At the college level, the im-pact to values, denomina-tors and percentages is mixed

In 2017-18, Statewide per-centage went from 5% to 9%

Completed a Non-credit Workforce Milestone

Among all students with a noncredit enrollment in the selected year, the pro-portion who completed a noncredit career education workforce preparation course or had 48 or more contact hours in a non-credit career education workforce preparation course in the selected year

Among all students with a noncredit en-rollment in the se-lected year, the pro-portion who com-pleted a noncredit career education or workforce prepara-tion course or had 48 or more contact hours in a noncredit career education course(s) or a work-force preparation course(s) in the se-lected year

Better aligned to original in-tent of the metric and align-ment with Adult Education Pipeline 2020 release


Included completing a workforce preparation course in a non-CTE TOP code

Denominator only includes noncredit students who have an enrollment in a CTE TOP code or in a workforce prepa-ration course

Note: This metric is not aligned to a similar metric in the Strong Workforce Program (SWP) dashboard because positive attendance hours were not summed across courses for the numer-ator in SWP. Instead students were only included if they had 48 or more hours in at least one workforce prep or non-credit CTE course in the selected year. Similarly, hours were not summed across courses to meet the 12 hour threshold in the denominator.

Values are higher and de-nominators are lower; therefore, percentages are higher except for a few col-leges


Percentages in SSM are lower for two-thirds of the colleges than those shown in the Strong Workforce Program dashboard



Adjustments Impacts

Earned 9+ Career Education Units

Among all students, the proportion who success-fully completed nine or more career education se-mester units in the se-lected year within the dis-trict


Better aligned to original in-tent of the metric


Changed methodology to count the number of units in the district rather than at the college level

Note: In SCFF, the way unique students are identified is slightly different. In SCFF, Student ID (SB00) counts one stu-dent with multiple Student ID Statuses (SB01) as multiple students. However, in SSM these counts are deduplicated. In addition, in SCFF special admit stu-dents have not been omitted from this metric

Quarter units for colleges on a quarter system have been converted to semes-ter units

At the college level, per-centages are higher except for a few colleges

Figures are much higher for colleges in multi-col-lege districts


At the district level, SSM is slightly lower than SCFF

Successfully Com-pleted Unit Thresh-olds in the Fall AND Successfully Com-pleted Unit Thresh-olds in the Selected Year

Among all students, the proportion who success-fully completed degree-ap-plicable semester unit thresholds in the fall term of the selected year OR in the selected year

Same Brief Defini-tion Unchanged


At the regional and statewide levels, quarter units have been converted to semester units, unless the user selects a college on quarter units

For colleges on the quarter sys-tem, values are lower

Retained from Fall to Spring at the Same CollegeANDRetained from Fall to Spring at Any Col-lege

Among all students, the proportion retained from fall to spring at the same college OR any college in the selected year, exclud-ing students who com-pleted an award or trans-ferred to a postsecondary institution

Same Brief Defini-tion Unchanged See All Student Journey

Denominators are lower

At the college level, per-centages are higher for about 80% of the colleges for retention at the se-lected college and for about 95% of the colleges for retention at any college


Success MetricsMetric or Issue

2019 Release 2020 Release Alignment or Reason for Change

Adjustments Impacts

Export of Data Set

For all award attain-ment figures, the de-nominator provided in the export included those who were en-rolled in the same or prior year. However, only students enrolled in the same year were included in the metric shown on the dash-board (which is dis-played as a count and not a rate).

The denominator only includes stu-dents enrolled in the same year as earning an award.

Alignment with SCFF Change to the metric definition dictionary and

export file


Values are the same

Transitioned to Postsecondary

Among all students, the proportion who took a noncredit adult basic educa-tion, adult sec-ondary education, or ESL course and subsequently en-rolled in a noncredit career education course or any col-lege level credit course in the se-lected or subse-quent year

Among all stu-dents, the number of students who took a noncredit adult basic educa-tion, noncredi t adult secondary education, or non -credit ESL course and subsequently enrolled in either a noncredit career education course or any college level credit course in the selected or subse-quent year

Better aligned to origi-nal intent of the metric and alignment with Adult Education Pipeline 2020 release


Added TOP code 493060 Elementary Education (Grades 1-8) to Adult Basic Education

Added TOP code 493062 Secondary Education (Grades 9-12) and GED to Adult Secondary Edu-cation

Only counted transition for the first time

Denominators are lower for about two-third of the colleges, and values are lower for about three-quarters

In 2017-18, the statewide per-centage went from 25% to 13% with lower percentages for about two-thirds of the col-leges


Metric or Issue


Adjustments Impacts

Students Who Earned a Non-credit Certificate

Among all students, the number who earned a noncredit certificate and had an enrollment in the selected year at the selected college

Among all stu-dents, the number who earned a non-credit certificate and had an enroll-ment in the se-lected year in the district

Alignment with SCFF


See Export of Data Set

Adjusted denominator to require enrollment at the district instead of the college

Note: Noncredit certificates are not included in SCFF


Counts are higher

Students Who Earned a Chancel-lor’s Office Ap-proved Certificate

Among all students, the number who earned a Chancel-lor’s Office ap-proved credit certifi-cate and had an en-rollment in the se-lected year at the selected college

Among all stu-dents, the number who earned a Chancellor’s Office approved credit certificate and had an enrollment in the selected year in the district

Alignment with SCFF




Added new data variables for Chancellor’s Of-fice approved credit certificates for 8 to 16 units and 16 to 30 units.

Note: In SCFF, 8 to 16 unit certificates are not included

Note: In SCFF, the way unique students are identified is slightly different. In SCFF, Student ID (SB00) counts one student with multiple Stu-dent ID Statuses (SB01) as multiple students. However, in SSM these counts are deduplicated. In addition, in SCFF special admit students have not been omitted from this metric

Note: In SCFF, students are counted only in the highest award category. In SSM, they are counted in each award category.


At the district level, counts will vary when comparing to SCFF

Students Who Earned an Asso-ciate Degree

Among all students, the number who earned an AA or AS degree (not for

Among all stu-dents, the number who earned an AA or AS degree (not

Alignment with SCFF See All Student Journey


Adjusted denominator to require enrollment at


At the district level, counts


Metric or Issue


Adjustments Impacts

transfer) and had an enrollment in the selected year at the selected college

for transfer) and had an enrollment in the selected year in the district

the district instead of the college


Note: In SCFF, students are counted only in the highest award category. In SSM, they are counted in each award category.

will vary when comparing to SCFF

Students Who Earned an Asso-ciate Degree for Transfer

Among all students, the number who earned an associate degree for transfer and had an enroll-ment in the selected year at the selected college

Among all stu-dents, the number who earned an as-sociate degree for transfer and had an enrollment in the selected year in the district

Alignment with SCFF





Note: in SCFF, students are counted only in the highest award category. In SSM, they are counted in each award category.


At the district level, counts will vary when comparing to SCFF

Students Who Earned a Commu-nity College Bach-elor’s Degree

Among all students, the number who earned a community college bachelor’s degree and had an enrollment in the

Among all stu-dents, the number who earned a com-munity college bachelor’s degree and had an enroll-

Alignment with SCFF See All Student Journey



Note: In SCFF, the way unique students are


At the district level, counts will vary when comparing to


Metric or Issue


Adjustments Impacts

selected year at the selected college

ment in the se-lected year in the district

identified is slightly different. In SCFF, Student ID (SB00) counts one student with multiple Stu-dent ID Statuses (SB01) as multiple students. However, in SSM these counts are deduplicated. In addition, in SCFF special admit students have not been omitted from this metric

Note: in SCFF, students are counted only in the highest award category. In SSM, they are counted in each award category.

SCFF

Students Who At-tained the Vision Goal Completion Definition

Among all students, the unduplicated count of students who earned one or more of the follow-ing: Chancellor’s Office approved cer-tificate, associate degree, and/or CCC baccalaureate de-gree, and had an en-rollment in the se-lected year at the selected college

Among all stu-dents, the undupli-cated count of stu-dents who earned one or more of the following: Chancel-lor’s Office ap-proved certificate, associate degree, and/or CCC bac-calaureate degree, and had an enroll-ment in the se-lected year in the district

Alignment with SCFF




Added new data variables for Chancellor’s Of-fice approved credit certificates for 8 to 16 units and 16 to 30 units

Note: In SCFF, 8 to 16 unit certificates are not included

Note: In SCFF, the way unique students are identified is slightly different. In SCFF, Student ID (SB00) counts one student with multiple Stu-dent ID Statuses (SB01) as multiple students. However, in SSM these counts are dedupli-cated.


In 2017-18 at the college level, values are higher for about two-thirds of the colleges

Students Who At-tained Appren-ticeship Journey Status

Among all students, the number who at-tained apprentice-ship journey status in the selected year

Among all stu-dents, the number of students who at-tained apprentice-ship journey status in the selected year and who were en -

Better aligned to origi-nal intent of metric


Added requirement that a student be enrolled at a California community college during the year when the student started the apprentice-ship program

Removed requirement that a student has to be

Values are lower

In 2017-18, the statewide val-ues went down by 25%


Metric or Issue


Adjustments Impacts

rolled at any com -munity college in the year that they started the appren -ticeship program

enrolled in the year that the student attained apprenticeship journey status

Students Who Transferred to a Four-Year

No Metric

Among students who earned 12 or more units at any time and at any college and who exited the commu-nity college system in the prior year and who enrolled in a four-year insti-tution in the se-lected year

Provide overall transfer figure



Note: In 2019-20 SCFF, students must earn 12 or more units in the district in the selected aca-demic year prior to transfer. In SSM, students must earn 12 or more units at any time and at any college before transfer.

At the district level, values are much higher in SSM than SCFF

Students Who Transferred to a UC/CSU

Among all students who completed at least 12 units at any community college at any time, the number who trans-ferred to a CSU or UC Institution

Among all students who completed at least 12 units at any community college at any time and exited the community college system , the num-ber who trans-ferred to a CSU or UC Institution



Excludes students who did not exit the commu-nity college system

Denominators are much lower

Values are slightly lower

Students Who Transferred to an In-State Private

Among all students who completed at least 12 units at any

Among all students who completed at least 12 units at





Values are


Metric or Issue


Adjustments Impacts

Four-Year Institu-tion

community college at any time, the number who trans-ferred to an in-state private four-year in-stitution

any community college at any time and exited the community college system , the num-ber who trans-ferred an in-state private four-year institution

slightly lower

In 2017-18, the statewide val-ues went down by 3%

Students Who Transferred to an Out-of-State Four-Year Institu-tion

Among all students who completed at least 12 units at any community college at any time, the number who trans-ferred to an out-of-state four-year insti-tution

Among all students who completed at least 12 units at any community college at any time and exited the community college system , the num-ber who trans-ferred an out-of-state four-year in-stitution




Note: The data received from MIS was problem-atic, so the Chancellor’s Office is looking into this file.


Values are slightly lower

In 2017-18, the statewide val-ues went down by 9% and for prior years, the percentage dif-ferences are much higher.

Average Num-ber of Units At-tained by All As-sociate Degree Earners ANDAverage Num-ber of Units At-tained by ADT EarnersANDAverage Num-ber of Units At-

Among all students who earned an asso-ciate degree (all, ADT, not ADT) and who were enrolled in the selected year, the average number of units earned in the California com-munity college sys-tem Among all stu-dents who had com-




Added “D” grades

Converted quarter units for colleges on a quar-ter system to semester units

Denominator in-creased, and values in-creased, except for colleges on the quarter sys-tem

In 2017-18, the statewide value was five units higher


Metric or Issue


Adjustments Impacts

tained by AA/AS Earners

pleted at least 60 units at any commu-nity college

Employment and Earnings MetricsMetric 2019 Release 2020 Release Alignment or


Unemployed Stu-dents Who Be-came Employed

Among all students who ex-ited the community college system and did not transfer to any postsecondary insti-tution, the proportion of students who were unem-ployed and became em-ployed after exiting college

Same brief defi-nition

Better aligned to origi-nal intent of the metric


Updated exiter definition to only exclude students if they enrolled in any postsecondary institution from 07/01 of the selected year to 06/30 of the subsequent year, rather than if they ever reenrolled

Denominators and values are higher for all colleges, except in 2016-17 where they are higher for half of the colleges

At the college level, per-centages are higher for about half of the colleges

In 2016-17, the statewide percentage is the same, but the prior two years de-creased by 8%

Students Who Earned a Students with a Job Closely Related to Their Field of Study

Among all students who re-sponded to the CTE Out-comes Survey and did not transfer to any postsec-ondary institution, the pro-portion who reported that they are working in a job very closely or closely re-lated to their field of study

Same Brief Defi-nition



Changed the methodology for excluding students who trans-ferred from using self-reported data from the CTE Outcomes Survey to using the Transfer Bucket

Updated exiter definition to

Denominators are lower for about 80% of colleges in 2015-16

At the college level, per-centages decreased for three-fourths of the col-leges in 2015-16 and half of the colleges in 2014-15


Metric 2019 Release 2020 Release Alignment or Reason for Change

Adjustments Impacts

only exclude students if they enrolled in any postsecondary institution from 07/01 of the selected year to 06/30 of the subsequent year, rather than if they ever reenrolled

In 2015-16, the statewide percentage decreased by 3%

Median Annual Earnings

Among all students who ex-ited the community college system and who did not transfer to any postsec-ondary institution, median earnings following the aca-demic year of exit





Adjusted values for inflation.

Note: SCFF does not adjust for inflation because it only looks at values for a single year.

Denominators are higher

At the college level, earn-ings are higher for 90% of the colleges in 2014-15, 78% of the colleges in 2015-16, and 47% of the colleges in 2016-17

At the statewide level, me-dian annual earnings were 4% higher in 2014-15, 2% higher in 2015-16, and 1% lower in 2016-17

Median Change in Earnings

Among all students who ex-ited the community college system and who did not transfer to any postsec-ondary institution, median change in earnings be-tween the second quarter prior to the academic year of entry and the second quarter after the academic year of exit from the last college attended




Changed methodology to in-clude students who were not enrolled in all terms of a given academic year and then had an enrollment in any term of the subsequent academic year, rather than an absence of three or more semesters

Updated exiter definition to only exclude students if they enrolled in any postsecondary institution from 07/01 of the

Denominators are much higher

Median percentage changes in earnings are much lower except for a few colleges

In 2016-17, change in statewide median earnings decreased from 51% to 31%


Metric 2019 Release 2020 Release Alignment or Reason for Change

Adjustments Impacts

selected year to 06/30 of the subsequent year, rather than if they ever reenrolled

Attained the Liv-ing Wage

Among all students who ex-ited the community college system and who did not transfer to any postsec-ondary institution, the pro-portion who attained the district county living wage for a single adult measured immediately following the academic year of exit




Adjusted values for inflation


Note: In SCFF, the way unique students are identified is slightly different. In SCFF, Stu-dent ID (SB00) counts one stu-dent with multiple Student ID Statuses (SB01) as multiple students. However, in SSM these counts are deduplicated. In addi-tion, in SCFF special admit stu-dents have not been omitted from this metric


At the district level, counts will vary when comparing to SCFF, because numbers may be slightly lower due to deduplication or higher because of how awards are counted Wages are higher

At the college level, the val-ues and percentages are higher for all colleges ex-cept a few that are slightly lower

At the statewide level, fig-ures are 5% higher in 2014-15, 3% higher in 2015-16, and 1% higher in 2016-17

At the district level com-pared to SCFF, SSM is higher


cal-pass plus - home - assignment of students into ... · web viewnote: in scff, the way unique...

Documents