Where Have We Been? Where Are We Going? Using Student Surveys to

Assess and Improve Literature Courses

Kelly Douglass, PhD

Asst. Professor, English

Riverside Community College

Riverside Community College District

Three campus district First day enrollment for Fall 2007 district

wide: 29,371 (http://academic.rcc.edu/enrollmentdata/)

49% of students taking an English class report educational goal of BA (with/without AA)

English Discipline

Common curriculum across district 39 full time faculty members; 136 part-

time faculty members Discipline offers 44 courses; 19 are

literature courses 13 literature courses offered in Spring

2006 (across district)

Why a student survey?

Intimidation about collaborative assessment of different courses

Most other course offerings had been or were being formally assessed by Spring 2006

Body of research indicating reliability

Factors for validity and reliability of student self-reporting

1. The information requested is known to the participants

2. The questions are phrased clearly and unambiguously

3. The questions refer to recent activities

4. Participants think the questions merit a serious and thoughtful response

5. The questions do not threaten, embarrass, or violate the privacy of the participate or encourage the participant to respond in socially desirable ways

Qtd. in William E. Knight et. al. “Student Self-Reported Gains Attributed to College Attendance: Comparing Two-Year and Four-Year Institutions”

Original Literature Review in G. D. Kuh’s “Using Student and Alumni Surveys for Accountability in Higher Education” in J. C. Burke (Ed.), The Many Faces of Accountability (2005) and The National Survey of Student Engagement: Conceptual Framework and Overview of Psychometric Properties (2001)

Student Learning Outcomesfor Literature Courses

1. Recognize and appreciate some of the distinctive features of the major writers, literary works, movements, trends, and genres in ________ literature from _____ to _____.

2. Explain how this literature both reflects and shapes the literary history of ______ with an awareness of how differences of culture, gender, and other social markers may shape a writer’s interactions with this history.

3. Relate literary works from different ______ authors and time periods to one another and synthesize ideas that connect them into a tradition.

4. Employ college-level methods of literary analysis to reading and interpreting _____ literature and literature in general.

5. Demonstrate critical thinking and writing skills through the process of constructing responses to, interpretations of, and arguments about literature.

Pre/Post Test General Questions

1. How many English classes have you taken in college? (include composition courses like 60AB, 50, 1AB)

1. What interests/interested you about this class?

1. What do you most want to get out of the class? / What is the most important thing you gained from this class?

Survey Questions and Corresponding SLOS

4. How confident are you about your writing skills? (SLO 5)

5. How confident are you about your ability to analyze literature (SLO 4)

6. How would you rate your knowledge of the following: a.) Major writers and literary

works of period and culture, b.) Literary movements and

trends of period and culture c.) Literary genres (SLO 1)

Survey Questions and Corresponding SLOs

7. When you read, are you able to relate what you read to outside circumstances of the period about which you’re reading such as history, culture, and/or politics? (SLO 2)

8. When you read, are you able to relate what you read to your own personal circumstances or philosophies? (SLO 5)

What interests/ed you about this class?

0

10

20

30

40

50

60

PRE (141respondents)POST (95respondents)

Req. for transfer / major / program / career / grad units; Eng. major

Subject area interest

Interest in Prof.

Gen. reading pleasure

% scale

Desire to see lit. in new ways; analyti-cal skills

Wtg. skills

Schedule No reply

Other

What do you most want to get out of this class? (PRE)What is the most important thing you have gained from this class? (POST)

0

5

10

15

20

25

30

35

40

45

PRE (141respondents)POST (95respondents)

% scale

A good grade / units for transfer / applies to career

More knowledge of / exposure to subject area

Increased ability to analyze / understand lit.

More writing practice / skills

Pleasure / Enjoyment

No reply

Other

Data for Questions 4 - 8

Question 8 Findings:A Poorly Written Question

Measures a skill that is not directly taught

Does not effectively represent SLO; narrow definition of skill through personalization

Question 4: How confident are you about your writing skills?

“Significant insignificance” in survey results

Writing not primary focus of literature courses

Students more aware of benefit/growth in content knowledge component than writing component

The class made them aware of writing skills as an area for improvement or interest

Responses to Study: Teaching Practices

More tools to help students “develop interpretive independence”

Reconsideration of writing assignments, particularly more that are better aligned with SLOs

In response to student desire for more traditional academic content: More occasional “straight lectures,” particularly

on theoretical structures A few more canonical authors

Depends very much on what is already being done in the classroom; reactions were very subjective

Responses to Study: SLOs

SLO 5: “Demonstrate critical thinking and writing skills through the process of constructing responses to, interpretations of, and arguments about literature.”Subpoint 3: “Support premises with effectively

integrated, relevant, thoughtful, and sufficient evidence drawn (as appropriate) from literary texts and the writer’s prior experience and knowledge.

Subpoint 5: “Think creatively within and beyond literary studies, making some connections between the literary work, and the intellectual and cultural forces that shape individuals’ lives.”

Responses to Study: Assessment and Future Projects

Instructor responses to assessment: Interesting information, but some difficulty

translating into classroom change

Helpful for refining exactly the kind of assessment to do in individual classrooms

Encouragement to create a writing based writing-SLO related assessment plan for the literature classes

Responses to Study: Assessment and Future Projects

07-08 Assessment Plan: Writing based General prompt for all literature courses Will measure portions of two SLOs:

Employ college-level methods of literary analysis to reading and interpreting _____ literature and literature in general.

Demonstrate critical thinking and writing skills through the process of constructing responses to, interpretations of, and arguments about literature.