bidle

1

Student Learning Outcomes Assessment (SLOA) at the Program and Course Levels:Comprehensive Projects That Work Across All Disciplines

AFACCT – January 2008Carroll Community College

T. Bidle, C. Dove, M. Jozik, V. Stein, N. Thorpe

2

Challenges to Student Learning Outcomes Assessment

1. Learning Outcomes vs. Content Objectives

2. Program vs. Course level Outcomes3. Assessment across all sections and

instructors4. In-house vs. benchmarked assessment5. Data collection, analysis, storage6. Closing the loop to improve instruction

3

Levels of Student Learning Outcomes & Assessment

Program

Course

Institutional

Middle States Commission on Higher Education (2003), Student Learning Assessment: Options and Resources, Philadelphia, PA 19104.

4

Program Level SLOA

• Definition of an HCC program – “a sequence of study that results in student achievement of specific learning outcomes”

• Program areas include:- General education- Developmental education- Transfer education- Career education

5

AS in Chemistry ProgramGeneral Education 32Program Requirements Gen CHM I and II (8) Organic CHM I and II 8 Calculus 1 4 Principles/ Physics I and II 10Recommended Electives 10

Total Credits 64

6

Chemistry Program

• Does the two year chemistry transfer program prepare students for successful upper division work?

7

How well does the program transfer to upper division institutions?

HCC Towson Shippensburg Hood Shepherd UMCPCHM 103 (4)General Chemistry I

CHEM 110 (4)Gen. Chem. I

CHM121 & 125 Chemical Bonding & Lab3

CHEM 101 (4)General Chemistry I

CHEM 207 & 207L (4) General Chemistry & Lab I

CHEM 143 (5)General and Analytical Chemistry6

CHM 104 (4)General Chemistry II

CHEM 111 (4)Gen. Chem. II

CHM122 & 126 Chemical Dynamics & Lab3

CHEM 102 (4)General Chemistry II

CHEM 209 & 209L (4)General Chemistry & Lab II

CHEM 153 (3)General and Inorganic Chemistry6

CHM 203 (4)Organic Chemistry I

CHEM 331 (5)Organic Chem. I1

CHM 221 & 225 (4) Mod. Org. Chem I & Lab

CHEM 209 (4)Organic Chemistry I

CHEM 315 & 315L (4)Organic Chemistry & Lab I5

CHEM 237 (4) Principles of Organic Chemistry I6

CHM 204 (4)Organic Chemistry II

CHEM 332 (5)1

Organic Chem. II1CHM 222 & 226 (4) Mod. Org. Chem II & Lab

CHEM 210 (4)Organic Chemistry II

CHEM 316 &316L (4)Organic Chemistry & Lab II5

CHEM 247 (4)Principles of Organic Chemistry II6

MAT 203 (4)Calculus I

MATH 211 (3) Calculus for Applications orMATH 273 (4)Calculus I


MATH 201 (4)Calculus I


MATH 140 (4)Calculus I

Add MAT 205 (4)As a recommended elective (Calc II)

MAT 212 (4)Calculus II

MATH 202 (4)Calculus II

MAT 208 (4)Calculus II

MATH 141 (4)Calculus II

PHY 203 (5)Principles of Physics I

PHYS 211 (4)Gen. Physics I (non-calculus based)2

PHY 205 (4)Intermediate Physics I

PHYS 101 (4)4

General Physics IPHYS 201 & 201L (4)College Physics I orGen. Physics I and Lab

Course Transfer Equivalencies Handout

Incomplete / see handout for complete chart

8

What are Program Outcomes?• Expectations for students when they leave the

classroom and are working in their careers or in subsequent coursework

• Results of accomplishing course learning outcomes which build on each other

• Involve the mastery, integration, and application of content and skills of many courses

• Must be measurable• More difficult to assess than giving final exam • “Career” programs may have a certification or licensure

exam

9

How are Program Outcomes Assessed?• Survey students completing all or most of the course

sequence in a program• Assess skills at the course level as the program is in

progress• Maintain portfolios of student work• Monitor success and persistence of students in the

course sequences within a program• Follow-up with transfer institutions that articulate with

many program completers• Follow-up with employers• Benchmark with nationally normed exams

10

Chemistry Program Outcomes1. Apply quantitative thinking process and reasoning

skills in upper division academic work

2. Communicate chemical concepts to varied audiences

3. Use mathematical tools essential to success in upper division chemistry courses

4. Design an appropriate response to a research problem-- appropriate lab equipment-- experimental design (scientific method)

5. Solve problems collaboratively

6. Use software and hardware to analyze experimental data

11

Survey of Chemistry Graduates(chemistry program assessment)

• Survey instrument developed based on Program Outcomes (handout)

• Criteria for selection expanded to include all students who completed 200 level program requirements for degree even if they did not graduate

• 72 surveys were mailed in Fall 2006• 14 surveys were returned and completed

12

4.1

3.7

4.2

3.9

4.2

3.4

0 1 2 3 4 5

Thinking/Reasoning

Communication

Mathematical Tools

Experimental Design

Collaborative TeamWork

Data Analysis

Chemistry Survey Results(chemistry program assessment)

13

CAAP Scientific Reasoning 2005 Chemistry Students

(Benchmarking)

65.2 66.4

84 86

0

20

40

60

80

100

CHM 101 CHM 104 CHM 204 PHY 202/204

Natio

nal P

erce

ntile

Mean

n=12 n=24 n=8 n=8

14

Pre-Allied Health “Program”

• Prerequisites from Math and Science Division– Anatomy and Physiology I & II– Math 101– Biology 099– Chemistry 101

15

Pre-Allied Health Program

• Do pre-allied health prerequisites prepare students for success in allied health career programs?

16

Evaluation of Pre-Allied Health Program

Success in BIO 099, A&P I and A&P IIFA 2000 SP 2007

72.9

59.5

45.7

79.2 75.9

0102030405060708090

ABC 099 ABC 103 ABC 104

% su

cces

sful

at e

ach

leve

l

With 099Without 099

17


35 34 31

238

127

48

335

207

107

050

100150200250300350400

ASN RAD LPN

Started in 099 Directly into 103 Admitted to Program

Students Admitted to Allied Health ProgramsFA 2000 SP 2007

18

Mathematics Courses Completed by A&P Students 2000-2005


40

20

4051.6

15.1

46.858

90.9

0102030405060708090

100

MAT 098 MAT 099 MAT 100 MAT 101

% C

ompl

etin

g Ea

ch M

ath

Cour

se

Needed BIO 099 Passed P.T. without BIO 099

19

BIO 099 Course Redesign

• Revised outcomes to include – Reading skills– Problem solving– Math skills

• Increase credits from 2 to 3 hours• Placement exam revised to reflect

outcome changes

20

Pre-Allied Health Program Outcomes

• Successful completion of Anatomy and Physiology I and II

• Successful admission into Allied Health Program

21

HCC Science Course Projects:

Program

Course

Institutional

Middle States Commission on Higher Education (2003), Student Learning Assessment: Options and Resources, Philadelphia, PA 19104.

22

HCC Science Course Projects:

• General Chemistry I and II

• Intro. To College Chemistry

• Biology for Allied Health

• Anatomy & Physiology I and II

23

The Program:Course Matrix To Integrate Outcomes (handout)

Program Outcomes

Course #1Outcomes

Course #2Outcomes

Course #3Outcomes

Course #4Outcomes

1. Reasoning skills2. Communicate concepts3. Use math tools4. Experimental design5. Collaboration

6. Use software for data analysis

24

Assessment of Skills and Content(chemistry course assessment)

• Common Rubrics in Related Courses– Dimensional analysis problems (handout)

– Laboratory reports (handout)

– Chemical Equations (in progress)

• Common Final Exams– Introductory Chemistry, General Chemistry I and II

25

Definitions

Active Students

= enrollees

Students registered at the census date (including those that withdrew

after the census date)

Completers Active students – withdrawals

Retention Rate

Completers active students

Success RateStudents earning A, B, C

active students

Completer success rate

Students earning A, B, Ccompleters

26

Definitions, cont’d

Persistence Progression through successive courses in a course sequence

Assessment Measurement of student learning of expected learning outcomes

Benchmark Assessment

Aggregate data from peer institutions or normed

national tests

27

General Chemistry I (CHM 103) Retention and Success (Fa 05 Fa 07)

92.582.5

89.288.1

65.474.2

81.362.5

76.9

0102030405060708090

100

Retention Success Completer Success

% s

tude

nts

Spring students Fall students Summer studentsN=40 N=189 N=18

28

American Chemical Society (ACS) Standardized General Chemistry Exams CHM 103 (Gen Chemistry, I)

35.2 36.2

56.4

43.437.6

42.638.2 41

0102030405060

n= 52 n=14

n=5

n=42 n=21n = 7 n = 53

29

CHM 103 (Gen. Chem. I) Fall 05 – Fall 07

A

B

C

D

F

Course Grade as a function of ACS Exam Percentiles: F05 --> F07

0

1

2

3

4

5

0 20 40 60 80 100

ACS Exam Percentiles

Cou

rse

Gra

de

30

89.1

64.171.9

92.3

76.983.3

0

20

40

60

80

100

Retention Success Completer Success

% s

tude

nts

Spring students Fall studentsN=64 N=26

General Chemistry II (CHM 104) Retention and Success (SP 06 Fa 07)

31

30.927.6

44.5

32.3

39.1

0

5

10

15

20

25

30

35

40

45

50

Spring 2006 Fall 2006 Spring 2007 Fall 2007 ACSBenchmark

American Chemical Society (ACS) Standardized General Chemistry Exam

CHM 104 (Gen Chemistry, II)

n=32 n=8n=18

n =16

32

CHM 104 (Gen. Chem. II) Spring 06 – Fall 07

A

B

C

D

F

Course Grade as a function of ACS Exam Percentiles

0

1

2

3

4

5

0 10 20 30 40 50 60 70 80 90 100

ACS Exam percentiles

Cou

rse

Gra

de

33

Persistence from General Chemistry I (CHM 103) II (CHM 104)

71

17 15

131

87

64

0

20

40

60

80

100

120

140

Successful 103 Attempted 104 Successful 104

No.

of S

tude

nts

66.4%

23.9% 48.9%21.1%

CHM 103 in Spring or SummerCHM 104 completed in Fall

CHM 103 in FallCHM 104 completed the following Spring

34

CHM 101: Introductory College Chemistry

• Assessment started Fall 2007• Rubric for grading dimensional

analysis problems• Common final exam

35

CHM 101 (Intro. Chem.) Common Final ExamCHM 101 Fall 2007

0

1

2

3

4

5

0 5 10 15 20 25 30 35 40 45 50

Common Exam Score

Cou

rse

Gra

de

A

B

C

D

F

36

Anatomy and Physiology I and II

• High Enrollment Course• Prerequisites

– Placement Test – BIO 099 for those who fail the placement test– H.S. Chemistry or Introductory College

Chemistry

37

A&P Project Design Overview

1. Based on the HAPS* learning outcomes

2. Develop a common cumulative final exam to assess student learning at the end of A&P I

3. Administer exam to all students completing A&P I and analyze student and section data

4. Administer national HAPS exam to all A&P II completers5. Use data to modify course and increase student success6. Generate Database of students’ exam grades, course

grades, success in other prerequisite courses

* Human Anatomy & Physiology Society (HAPS)

38

Using the Student Database

• Identify gaps in student learning• Modify course components• Science Learning Center supplements• Laboratory activities• Examples of using the data

39

HAPS Exam Scores vs. HCC 1st Semester Exam

40

Student Groups Average + Standard Deviation

Average of 2-yr and 4-yr colleges (n=2430)

53.46 + 11.88

4-yr colleges (n= 964) 57.53 + 12.10

2-yr colleges (n=1466) 50.80 + 10.45

HCC students (n=294) 54.01 + 12.5*

HAPS 2-Semester Comprehensive ExamComparison with Other Colleges

(benchmarking)

*HCC Retention Rate = 71%

41

The Science Team

• Cindy Dove• Veronica Stein• Nancy Thorpe• Judith Peisen• Terri Bidle• Robin Thomas

• Maria Jozik• Richard Campbell• Bernard Murphy• David Karstaedt• James Stemmle• Elaine Ashby

Contact information: [email protected]@hagerstowncc.edu 301-790-2800 extension 268

mailto:[email protected]

mailto:[email protected]

bidle

Business