2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010

Maps to Success: Using Data

• Richard Mahon, Curriculum Committee• Beth Smith, Grossmont College• Greg Stoup, Cañda College

o Key reference: Data 101

2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010

Maps to Success: Using Data

Accreditation standards require that colleges assess the curriculum at the course, program, and degree levels. What data would allow the curriculum committee to contribute to broader campus conversations about student learning, and how can the curriculum development and approval process help guide faculty toward the thoughtful use of data in evaluating and improving their own effectiveness?

Does your curriculum committee concern itself with data?

2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010

Maps to Success: Using Data

Curriculum, data and institutional processes…

• Program Review• Student Learning Outcome Assessment• Student Equity• Enrollment Management

2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010

Maps to Success: Using Data

System Data: Datamart http://www.cccco.edu/ChancellorsOffice/Divisions/TechResearchInfo/MIS/DataMartandReports/tabid/282/Default.aspx)

ARCC Report:

CalPASS - http://www.cal-pass.org/

•NCES National Center for Education Statistics http://nces.ed.gov/

•CPEC California Postsecondary Commission http://www.cpec.ca.gov/

Local Data?

2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010

Maps to Success: Using Data

Data for what? • Prerequisites• Labor Market Analysis• Student performance• Student completions (certificates, degrees, transfer)

The Challenges with using college data to

identify strategic interventions

To the outsider, researchers can have an intimidating comfort zoneThe T-Test Procedure Statistics

Lower CL Upper CL Lower CL Upper CLVariable: female N Mean Mean Mean Std Dev Std Dev Std Dev Std Err

Write 0 91 47.975 50.121 52.267 8.9947 10.305 12.066 1.0803Write 1 109 53.447 54.991 56.535 7.1786 8.1337 9.3843 0.7791Write-Write Diff (1-2) -7.442 -4.87 -2.298 8.3622 9.1846 10.188 1.3042

T-Tests

Variable Method Variances DF t Value Pr > |t|

Write Pooled Equal 198 -3.73 0.0002Write Satterthwaite Unequal 170 -3.66 0.0003

Equality of Variances

Variable Method Num DF Den DF F Value Pr > F

Write Folded F 90 108 1.61 0.0187

The wrong format creates dissonance between your message and your audience.

Packaging the message

Consider a simple pathway assessment Basic Skills

Student

Successful Transfer

Learning Community

Hypothesis: Learning Communities improve the transfer success of Basic Skills students

We pull data from our database to test the hypothesis ….

Successful Transfer

Complete SEP

Take Summer Courses

Learning CommunityEnroll Full-

Time

Basic Skills Student

Enroll in MESA

We learn that we need more info to tell the real story

Successful Transfer

Faculty Letter of Recommendation

Library Orientation

Good impression

from Campus

Visit

Talk to Univ. Rep

Nurture Faculty Relationships

Meet with Outreach Professional

Attend Lecture Series

Learning Community

Complete SEP

Take Summer Courses

Participate in Student Govt

Hallway chat with

professor

Basic Skills Student

Enroll in MESA

Enroll Full-Time

Text books in stock

Becomes student tutor

Join Student Club

Placement Test Prep

Faculty suggest outside reading

Faculty recommends MESA

Planning advice from FA Office

But the reality is…

Examples of Program Level Data that can

drive curricular discussion

Cohort Description Headcount

One year Fall-to-Fall Student

Persistence

Two year Fall-to-Fall Student

Persistence FTES

One year Fall-to-Fall

FTES Persistence

Two year Fall-to-Fall

FTES Persistence

Concurrent Enrollment 585 29.5% 15.0% 142.6 43.8% 26.0%

ESL-Only Student 975 32.9% 17.9% 293.5 38.1% 19.9%

Basic Skills Student 1,062 51.0% 35.3% 509.9 48.4% 30.7%

Career-Tech Directed Student 1,328 35.9% 21.3% 304.9 41.8% 21.8%

Transfer Directed Student 2,249 38.2% 23.8% 570.7 40.0% 19.7%

Ed Development Student 606 45.6% 31.9% 117.4 44.6% 26.5%

College Overall 6,805 39.1% 24.4% 2,098 42.4% 23.4%

There is significant variation in persistence across the six segments.

Segment Persistence

College Average 39.1%

College Average 24.4%

One Year Fall-to-Fall

Two Year Fall-to-Fall

Segment Success & Retention

College Average 70.0%

College Average 84.5%

Success Rates

Retention Rates

Can we build on this information?

An estimate of the resource usage patterns of each segment (low, medium, high) might help us develop strategies to more effectively deploy rare resources.

Cohort DescriptionPct of FTES

Fall-to-Fall Persistence

Percent Single Course Takers Six Year Growth Pattern

Resource Usage

Concurrent Enrollment 8.6% 29.5% 68% Accelerating Growing ?

ESL-Only Student 14.3% 32.9% 59% Mild Growth with some volatility ?

Basic Skills 15.6% 51.0% 14% Modest increase with some volatility ?

Career Directed 19.5% 35.9% 58% Several years of steady growth ?

Transfer Student 33.0% 38.2% 40% Slight recovery after extended decline ?

Ed Development 8.9% 45.6% 69% Flattening after steep decline ?

College Overall 100% 39.1% 47% Strong growth following a steep decline ?

ESL Students have low persistence in part because many navigate long course sequences.

ESL / ENGL SequenceEntry Point for Student

Cohort Metric Level 0 Level 1 Level 2 Level 3 Level 4 ESL 400*ENGL 100

Level 0# 1,881 889 443 227 160 42 16% 47.3% 23.6% 12.1% 8.5% 2.2% 0.9%

Level 1# 2,722 1,424 817 602 167 70% 52.3% 29.9% 22.1% 6.1% 2.6%

Level 2# 2,555 1,425 912 312 147% 55.8% 35.7% 12.2% 5.8%

Level 3# 2,173 1,248 489 241% 57.4% 22.5% 11.1%

Level 4# 1,815 806 403% 44.4% 22.2%

ESL Persistence by Student Entry Level(Tracking Period: Fall 2000 – Spring 2008)

* ESL400 category also includes ENGL 836

Persistence patterns generally follow a 50% Rule

Students Taking a Single Course

Students Taking Multiple Courses

In any given term roughly half of our students are Single Course Takers

Why is this important?

• Single Course Takers have a more tenuous connection to the college• Single Course Takers have low persistence• Returning students that become single course takers are 80% less likely to transfer or obtain a degree†

*Persistence Rate = 50.3% *Persistence Rate = 29.8%

*Includes only the Base Segment population of Basic Skills, Career Directed & Transfer Directed Students. †Adjustments were made to account for students whose course load dropped in the final term before transfer or receiving a degree.

Headcounts of Single Course Takers & Percentage of Segment Population

Single Course Takers are not uniformly distributed across the Student Segments

68%

59%

14%

58%

40%

69%

College Average = 47%

INTERMEDIATE ALGEBRA

Sequence Completion Rates by Initial Course Placement

TRANSFER LEVEL MATH

PRE ALGEBRA

ELEMENTARY ALGEBRA

INTERMEDIATE ALGEBRA

Initial Placeme

nt3 years

3.6%

19.1%

47.6%

4 years

5.4%

20.6%

49.2%

5 years

6.1%

22.4%

49.2%

PRE ALGEBRA ELEMENTARY ALGEBRA

2 years

2.3%

15.5%

43.4%

Percent of Students Completing the Algebra Sequence within 2 to 5 Years

Basic Skills Curriculum Sequence

Course Pass % = 55% Course Pass % = 55% Course Pass % = 50% Course Pass % = 75%

Profile of Developmental English & Reading

TRANSFER LEVEL

ENGLISH

Basic Composition

Developmental Reading

Writing Development

Reading Strategies

Course Pass % = 58%

Course Pass % = 58%

Course Pass % = 64%

Course Pass % = 69%

Course Pass % = 67%

ENGL 826

Initial Placeme

nt3 years

23.9%

24.8%

4 years

27.2%

29.9%

5 years

27.7%

30.1%

2 years

20.3%

22.1%

Percent of Students Completing the Basic Skills Sequence within 2 to 5 Years

READ 826

Examples of often overused metrics that, in isolation, can lead to

false conclusions

What are the two measures most widely used by CCCs to assess progress?

• Success Rate• Retention Rate

Consider the multitude of changes over this period:

- faculty/staff turnover- program successes/failures- changing student demographics - budget contractions/expansion- leadership turnover

and yet these performance metrics remained relatively stable.

What to do when you reach the limits of what data can answer

You start off with a large number of

options

The data helped shrink the number of options but

there is still more than one choice

Trust your

intuition and

choose !

( You’ve talked with your colleagues &

reflected on the data )

Handout: An illustration of a collaborative dialogue converging on action

Collaborative Model of Institutional Research

Faculty/Staff

Researcher

Joint Activity

Primary Responsibility

Key Features:

• Dialog-rich• Jointly-driven processes• Priority on the development of the data story

Design Charrette Joint Reflection on Findings

Story Development

Validate Message

Share the Story

1. An attitude of wisdom (knowing what you don’t know)

2. Commitment to framing issues with data

3. Commitment to hearing & telling the truth

4. Adoption of an experimental mindset

5. Oriented toward action

What characterizes a learning organization?

Source: Jeffrey Pfeffer, Professor of Organizational Behavior at the Graduate School of Business, Stanford University.

Nurturing a Culture of Inquiry

DISCUSSION

If you remember only one thing

Make sure you have a room, populated with reflective thinkers, that meet regularly, to discuss data and ask questions

Nurturing a Culture of Inquiry

THANK YOU