© 2011 center for community college student engagement tennessee community colleges statewide...

© 2011 Center for Community College Student Engagement

Tennessee Community Colleges

Statewide WorkshopOctober 30, 2014


Misha TurnerAssociate Director, College RelationsCenter for Community College Student Engagement

Mike BohligSenior Research AssociateCenter for Community College Student Engagement

Courtney AdkinsAssistant DirectorCenter for Community College Student Engagement

Center for Community College Student EngagementProgram in Higher Education LeadershipThe University of Texas at Austin

Introductions


Who is in the room today?

Have you ever seen CCSSE results?

Have you ever logged into the online reporting system?

Have you formed a workgroup, discussed CCSSE and other data, and used that data to inform decisions to change something at your college or on your campus?


Agenda

Student Voices

Student Engagement and Success

Data Narrative Exercise

Looking at Your Data Across Time and Review of the Online Reporting System

Diving Into Your 2014 CCSSE Data

Promising Practices to Strengthen Student Success


Student Voices Video


What did you hear?

About “front door” experiences?

About teaching and learning?

About support for students?

About what makes a difference for students?


Defining Student Engagement


What is Student Engagement?

…the amount of time and energy students invest in meaningful educational practices

…the institutional practices and student behaviors that are highly correlated with student learning and retention


Center for Community College Student Engagement

CCSSE (& CCFSSE)SENSE

Established surveys:

Assess the quality of their work

Identify and grow successful educational practices

Identify areas in which to improve

Provide context: a data-derived picture of institution

Shift the focus to institutional locus of control

CCSSE and SENSE are tools designed to help colleges:

Qualitative work


What is the relationship between student engagement and student success?

How do we know this?


I have a goal!

Sources: CCSSE 2014

Tennessee Students

Certificate:

Associate Degree:

Transfer to 4-year:

47%

85%

79%


Reality Check

Horn, L., & Skomsvold, P. (2011). Community college student outcomes: 1994–2009 (NCES 2012-253). Retrieved from http://nces.ed.gov/pubs2012/2012253.pdf

ACT, Inc. (2010). What works in student retention? 4th national survey: Community colleges report. Retrieved from http://www.act.org/research/policymakers/pdf/droptables/CommunityColleges.pdf

• Fifty-four percent of students who enter community colleges earn a certificate, a degree, or are still enrolled six years later.

• Nearly half of all community college students entering in the fall term drop out before the second fall term begins.


15%but….

74%

ATD Data Notes (2008)


One thing we KNOW about community college student engagement…

It’s unlikely to happen by accident.

It has to happen

by design.


Student Success: What We Know Matters


What We Know Matters

In focus groups with students, what do they typically report as the most important factor in keeping them in school and persisting toward their goals?

Relationships

Connections matter


High Expectations Matter

How often have you worked harder than you thought you could to meet an

instructor’s standards or expectations? Item #4p

Never Sometimes Often/Very Often

8.0% 35.7% 56.3%

Source: 2014 CCSSE data


High Expectations Matter

Expectations may not be as high as they need to be…

How often have you come to class without completing readings or assignments? Item #4e

Never Sometimes Often/Very Often

35.8% 52.4% 11.8%



Very/Somewhat

Peer or other tutoring 73.7%

Skill labs (writing, math, etc.) 77.5%

Rarely/Never

Peer or other tutoring 48.1%

Skill labs (writing, math, etc.) 34.3%

How important are the following services?

How often do you use the following services?


High Support Matters


TN students who…

Talked about career plans with an instructor or advisor (CCSSE Item 4m): 74.6%

Never: 25.4%

Discussed grades or assignments with an instructor (CCSSE Item 4l): 92.6%

Never: 7.5%

Inescapable Engagement Matters



Plan to take classes at this college again (CCSSE Item 20)


14%

6%

65%

15%

I will accomplish my goal(s) this termI have no current plans to returnWithin the next 12 monthsUncertain

High Support MattersTN students who…


How can we make engagement

inescapable?


Make it Mandatory

How do students feel about “MANDATORY” ?

a. Frightened

b. Appreciative

c. Disgruntled

d. Rebellious

e. Depressed

Students want our

guidance…

Even though they

complain about it.

Key Question: Does “mandatory” really mean mandatory?


Benchmarks and Benchmarking


Benchmarking for Excellence

The most important comparison: where you are now, compared with where you want to be.


Center Benchmarks of Effective Educational Practice

Groups of conceptually-related items

Standardized to a national mean of 50

Address key areas of student engagement

Provide a way for colleges to compare their own performance with other groups of colleges (across your consortium and other colleges like you) and across student groups


CCSSE Benchmarks

Active and Collaborative LearningStudent EffortAcademic ChallengeStudent Faculty InteractionSupport for Learners


2014 CCSSE Benchmark Scores for Tennessee Community Colleges

48.8

52.1

50.3

51.9

48.550



Disaggregating Benchmark DataTennessee:Enrollment Status (P/T vs. F/T)

P/T F/T

Active and Collaborative Learning 44.8 53.3

Student Effort 48.9 55.6

Academic Challenge 47.3 53.9

Student-Faculty Interaction 48.9 55.4

Support for Learners 46.1 51.2

56% of Tennessee community

college students are enrolled part time

Source: 2014 CCSSE data, 2012 IPEDS data


What about never?

Full-time students might have more opportunity to engage with other students and their instructors, but…how do we explain never?

Never worked with other students on projects during class (CCSSE Item 4f): 15%

Part-time: 18.5%

Never worked with classmates outside of class to prepare a class assignments (CCSSE Item 4g): 38.5%

Part-time: 45.4%



Disaggregating Benchmark DataTennessee: Developmental Status

Dev Non-Dev

Active and Collaborative Learning 50.0 47.2

Student Effort 56.9 45.9

Academic Challenge 51.7 48.3

Student-Faculty Interaction 54.0 49.2

Support for Learners 51.9 44.6



More engaged, but …

Approximately 62% of U.S. community college students take at least one developmental education* course within six years of their initial enrollment.

At some colleges, the percentage is even higher.

For too many students, traditional developmental education is a terminal roadblock to success.

A CCRC study of 250,000 community college students found that only 20% of students referred to developmental math and 37% of students referred to developmental reading go on to pass the relevant entry-level or "gatekeeper" college course. **

*Source: RTI International. (n.d.). The completion arch: Measuring community college student success (Participation in developmental courses: United States). Retrieved from College Board website: http://completionarch.collegeboard.org/placement/participation-in-developmental-courses/participation-in-developmental-courses-us

**Source: Bailey, T., Jeong, D. W., & Cho, S. W. (2009). Referral, enrollment, and completion in developmental education sequences in community colleges (CCRC Working Paper No. 15). Retrieved from Community College Research Center website: http://ccrc.tc.columbia.edu/media/k2/attachments/referral-enrollment-completion-developmental.pdf


OUTCOME

1 Completion of at Least One Developmental Education Course With a Grade of C or Better

ACCELERATED OR FAST-TRACK DEVELOPMENTAL EDUCATION

Source: CCSSE-linked student record data

OUTCOMES

CCSSE developmental students

who reported participating in were to successfully complete

accelerated or fast-track developmental education during their first academic term

2.40 timesmore likely

a developmental English courseN=1,219p<0.0001

than were students who did not report participating in accelerated or fast-track developmental education during their first academic term.


OUTCOME

1

16%

84%

Completion of at Least One Developmental Education Course With a Grade of C or Better

ACCELERATED OR FAST-TRACK DEVELOPMENTAL EDUCATION

Source: CCSSE-linked student record data

PARTICIPATION

CCSSE developmental students (N=1,219)

Yes(n=195)

No(n=1,024)


Data Narrative Exercise

Step One: Open Envelope #1

Look at the information inside. Does this data point surprise you? What do you think is driving this data?

Step Two: Open Envelope #2

Look at the information inside. How is this data point related to the one in Envelope #1?

Step Three: Open Envelope #3

Look at the information inside. How is this data point related to the ones in Envelopes #1 and #2?

Step Four

What story do the data in these three envelopes tell?


Looking at Your CCSSE Data Across Time


What sort of data are we talking about?Benchmarks – scores on high level concepts to get you into the results

Means – place responses on a scale to allow comparison

Frequencies – give you details about the actual responses/behaviors


The Data – Where To Begin

At least three administrations since 2005 Benchmarks

• CCSSE website: Standardized benchmark scores are not appropriate for longitudinal analysis.

• Standardized scores re-standardized every year based on 3-year cohort

• Raw benchmark scores – available in download data file. • Scale: 0 - 1


Standardized vs. Raw Benchmark Scores

Standardized

Benchmark 2006 2014%

Change

Active and Collaborative Learning

50.741 48.764 -3.895

Student Effort 50.913 52.096 2.324

Academic Challenge 50.819 50.288 -1.045

Student-Faculty Interaction 51.646 51.892 0.476

Support for Learners 49.631 48.525 -2.228

Raw

2006 2014%

Change

0.3588 0.3664 2.113

0.4576 0.4831 5.5650.5539 0.5827 5.189

0.3724 0.4398 18.106

0.4011 0.4394 9.546


Tennessee State-Wide Raw Benchmark Scores 2006, 2009, 2011, and 2014

2006 2009 2011 2014

ACTCOLL 0.3588131 0.362355 0.3651788 0.3663956

STUEFF 0.4576204 0.4692714 0.4832194 0.4830853

ACCHALL 0.5539408 0.5669929 0.5799882 0.5826857

STUFAC 0.3724023 0.3983581 0.4164189 0.4398292

SUPPORT 0.4011079 0.4178661 0.4324272 0.4393995

0.33

0.38

0.43

0.48

0.53

0.58

Ra

w B

en

ch

ma

rk S

co

re


What’s Driving Benchmark Scores:

Student-Faculty Interaction Benchmark Items – Means

2006 2009 2011 2014 % Diff

4kUsed email to communicate with an instructor 2.44 2.72 2.89 2.98 22.228

4lDiscussed grades or assignments with an instructor 2.52 2.57 2.62 2.67 6.046

4mTalked about career plans with an instructor or advisor 1.98 2.03 2.06 2.15 8.565

4nDiscussed ideas from your readings or classes with instructors outside of class

1.71 1.74 1.76 1.82 6.450

4oReceived prompt feedback (written or oral) from instructors on your performance

2.67 2.72 2.75 2.81 5.273

4qWorked with instructors on activities other than coursework 1.38 1.38 1.40 1.48 7.298


What’s Driving Benchmark Scores:Student-Faculty Interaction Benchmark Items - Means

2006 2009 2011 2014

4k. EMAIL 2.4358168 2.7157811 2.8874838 2.9772545

4l. FACGRADE 2.5176751 2.5714679 2.624084 2.6698943

4m. FACPLANS 1.9804817 2.0256748 2.0573525 2.1501189

4n. FACIDEAS 1.7101274 1.7395515 1.7638023 1.8204276

4o. FACFEED 2.6709448 2.7236351 2.7461102 2.8117774

4q. FACOTH 1.3797627 1.3841151 1.4000202 1.4804625

1.3

1.8

2.3

2.8

Ite

m M

ea

n S

co

re



Student-Faculty Interaction Benchmark Items - Frequencies

Percentage reported NEVER doing

2006 2009 2011 2014 % Diff

4kUsed email to communicate with an instructor 19.97 10.33 5.96 4.03 -79.820

4lDiscussed grades or assignments with an instructor 26.01 27.3 22.87 23.82 -8.420

4mTalked about career plans with an instructor or advisor 25.97 27.49 24.3 22.25 -14.324

4nDiscussed ideas from your readings or classes with instructors outside of class

24.79 26.34 24.56 24.31 -1.936

4oReceived prompt feedback (written or oral) from instructors on your performance

26.24 27.88 23.3 22.59 -13.910

4qWorked with instructors on activities other than coursework 24.36 26.63 24.9 24.12 -0.985



Percentage Never Doing Activities on the

Student-Faculty Interaction Benchmark

2006 2009 2011 2014

4k. EMAIL 19.97 10.33 5.96 4.03

4l. FACGRADE 26.01 27.3 22.87 23.82

4m. FACPLANS 25.97 27.49 24.3 22.25

4n. FACIDEAS 24.79 26.34 24.56 24.31

4o. FACFEED 26.24 27.88 23.3 22.59

4q. FACOTH 24.36 26.63 24.9 24.12

3

8

13

18

23

28

Pe

rce

nt

Ne

ve

r D

oin

g A

cti

vit

y


Item Differences by Subgroup Selected Student-Faculty Interaction Items

2006 2009 2011 2014

4k. EMAIL FT 53.1 64.53 70.64 72.64

4k. EMAIL PT 35.74 48.63 56.62 63.51

Series3 NaN NaN NaN NaN

4m. FACPLANS FT 27.86 31.5 31.3 34.3

4m. FACPLANS PT 19.22 19.85 20.26 25.78

20

30

40

50

60

70

Pe

rce

nt

Re

sp

on

din

g

Oft

en

or

Ve

ry O

fte

n


A quick look at the CCSSE online reporting system.

Standard Reports

Custom Reports

Online tutorials for Online Reporting System:http://www.ccsse.org/tools/tutorials.cfm


Q & A


LUNCH


Diving Into Your Data


Report Out


Targeting Areas for IMPROVEMENT

Student Success Data Team analyzed trends from three sets of CCSSE data

Focused on two areas needing improvement:

Student Effort & Academic Challenge

Selected specific items which faculty could address

Planned faculty presentation at in-service

Butler Community College (KS)


STUDENT EFFORT ITEMS

In the current school year, about how often have you done each of the following?

Prepared two or more drafts of a paper or assignment before turning it in. Butler Mean 2.5

Worked on a paper or project that required integrating ideas or information from various sources. Butler Mean 2.78

Came to class without completing readings or assignments. Butler Mean 1.91



ACADEMIC CHALLENGE ITEMS

During the current school year, how much has your coursework at this college emphasized the following mental activities?

Analyzing the basic elements of an idea, experience or theory (Butler Mean 2.8)

Synthesizing and organizing ideas, information, or experiences in new ways (Butler Mean 2.69)

Making judgments about the value or soundness of information, arguments, or methods (Butler Mean 2.54)

Applying theories or concepts to practical problems or in new situations (Butler Mean 2.61)

Using information you have read or heard to perform a new skill (Butler Mean 2.64)



TWO ACTION QUESTIONS

In the coming semester what will you do to:

Encourage students to spend more time in preparing for class?

In the coming semester what will you do to:

Encourage students to engage more often in mental activities that promote higher learning?



Examples of ACTIONS IMPLEMENTED

Accounting: Increase faculty emphasis on time management in supervision of students’ major project

Marketing: Make the group marketing projects into service learning projects benefiting local organizations

Animal Science: Implement more proactive faculty intervention with at-risk online students

Digital Illustration: Implement more detailed formative assessment of student work during creation of projects

Art Appreciation: Implement common essential reading assignments attached to formative assessments

Biology: Require student to meet with instructor during office hours at least once during first six weeks of class


NEXT STEPS in Fall 2014

Analyzing data from Spring 2014 implementations

Continuing, modifying, or expanding implementations


To learn more, contact Karla Fisher, VP of Academic Affairs, at [email protected]

mailto:[email protected]


Promising Practices for Student Success


CCCSE Special Study on Promising Practices

What is it? Online Institutional Survey (CCIS) Special-focus items on CCSSE Items added to CCFSSE Special-focus module on SENSE

Lots of data


Late RegistrationDuring the current term at this college, I completed registration before the first class session(s). (CCSSE Promising Practices, Item #1)


Yes, all courses

Mostly Partly No, not any

93%

5% 2% 1%

89%

6% 2% 2%

TN 2012-2014 PP Respondents

Percentages may not total 100% due to rounding


OrientationThe ONE response that best describes my experience with orientation when I first came to this college is... (CCSSE Promising Practices, Item #2)


16%

40%

6%

19% 19%

12%

43%

8%

19% 18%




First-Year ExperienceDuring my first term at this college, I participated in a structured experience for new students... (CCSSE Promising Practices, Item #3)


Yes, first term Yes, first term AND at least one

other term

Yes, but NOT during first term

No

19%

2% 3%

76%

24%

3% 4%

69%




Learning CommunityDuring my first semester at this college, I enrolled in an organized learning community... (CCSSE Promising Practices, Item #4)


other term

Yes, but not first term

No

6% 3% 2%

89%

8% 4% 3%

86%


Source: 2014 CCSSE data Percentages may not total 100% due to rounding


Student Success Course



other term

Yes, but not first term

No

15%

2% 4%

79%

18%

3% 5%

74%


During my first semester/quarter at this college, I enrolled in a student success course (such as a student development, extended orientation, study skills, student life skills, or college success course). (CCSSE Promising Practices, Item #5)



Do these practices make a difference?

Series of Center Reports on High-Impact Practices


2014 Tennessee CCSSE Benchmark Scores by timely registration

49.1

52.651.1

52.5

48.7

45.4

49.9

44.5

49.9

45.9

Registered for all classes on time Did not register for all classes on time



2014 Tennessee CCSSE Benchmark Scores by Orientation

50.2

54.7

52.354.1

51.6

46.6

48.647.7

49.3

43.3

Participated in orientation Did not participate in orientation



2014 Tennessee CCSSE Benchmark Scores by First-Year Experience

54.0

58.0

55.2

57.655.6

47.5

50.949.5

50.9

46.5

Participated in FYE Did not participate in FYE



2014 Tennessee CCSSE Benchmark Scores by Learning Community

59.1 59.8

55.1

62.160.0

47.9

51.650.2

51.3

47.3

Participated in LC Did not participate in LC



2014 Tennessee CCSSE Benchmark Scores by Student Success Course

54.8

59.0

54.8

58.757.7

47.6

50.949.8

50.9

46.6

Student Success Course No Student Success Course



Relatively small numbers of students are experiencing high

impact practices, but for those who do, we consistently see higher

levels of engagement.


So now what?

Let’s look a bit deeper. Are we implementing high-impact practices

for students who need them the most?


Subgroup Analysis

25% of TN Developmental Students reported experiencing a student success course during their first

term.

8% of TN non-dev students reported experiencing a student success course during their first term.


2014 Tennessee CCSSE Benchmark Scores For Student Success Course Participation – By Dev Status

55.9

60.9

55.1

60.1 59.4

47.9

56.1

51.252.5

49.451.1 51.1

53.554.5

51

47.145.6

48.249.2

44.1

Dev: SSC Dev: No SSC Non-Dev: SCC Non-Dev: No SCC



Does a student success course differentially affect developmental education students vs

non-developmental education students?


Subgroup Analysis

9% of TN First Generation students reported experiencing an organized

learning community

7% of TN Not-First Generation students reported experiencing an

organized learning community.


2014 Tennessee CCSSE Benchmark Scores For Learning Community Participation – By Generation Status

58.1

63.3

58.0

61.7

65.2

47.8

53.851.8 51.2

49.2

60.7

57.656.4

63.3

55.7

47.849.6 49.6

51.1

45.5

1st Gen: LC 1st Gen: No LC Not 1st Gen: LC Not 1st Gen: No LC



Does participating in an organized learning community differentially

affect first gen vs not first gen students?


What other points of data do you need to explore to make this more informative?

• Success rates between the student types• Success rates between the student types based on

having the course or not• Other demographic information for the student

groups• Whether student in the student success course

groups were receiving other services• Etc…..


Effective educational practice…

Quality of design matters. Scale matters. Intensity matters.


Promising Practices Discussion Questions-Part One

Which of these practices are mandatory at your college? For whom?

What practices started small and have been brought to scale?

What are the challenges that get in the way of implementing promising practices? Bringing them to scale?


Spend a few minutes reviewing your Promising Practices data

On the Standard Reports page: http://www.ccsse.org/members/reports/2014/reports.cfm

CCSSE Special-Focus Items - Promising Practices

http://www.ccsse.org/members/reports/2013/reports.cfm

http://www.ccsse.org/members/reports/2013/reports.cfm


On the Custom Report Requests page, choose CCSSE Special-Focus Items as your instrument

For Option One, choose to make comparisons within your college

For Option Two, choose a breakout group

Repeat with a different breakout group of interest

Find your reports on the Completed Report Requests page

Diving Deeper


What do your data tell you about the practices? Who is participating in them? By developmental status, enrollment status, other targeted group?

Based on your review of the promising practices data, what might your first steps around promising practices be?

Promising Practices Discussion Questions-Part Two


Q and AThanks for joining us!

© 2011 center for community college student engagement tennessee community colleges statewide...

Documents

community college students

student success

student retention

student learning

student behaviors

college relationscenter

community colleges report

energy students