TRANSITIONING TO A CULTURE OF EVIDENCE:

SUSTAINABLE ASSESSMENT PRACTICES

Roundtable Discussion Organized by Assessment Subcommittee

2018 UWSP Teaching ConferenceJanuary 18, 2018

Invited Speakers and Assessment Subcommittee Members ■ Mark Tolstedt, Professor of Media Studies, Division of Communication,

College of Fine Arts and Communciation ■ David Barry, Assistant Professor of Sociology, Department of Sociology and

Social Work, Colleges of Letters and Science ■ Paula DeHart, Professor of Education, School of Education, College of

Professional Studies ■ AS Members: Maggie Bohm-Jordan, Dave Dettman (Co-Chair), Paul Doruska,

Becca Franzen, Dan Graf (Co-Chair), Matt Groshek, Jennifer Huffman, Vera Klekovkina, Cuiting Li, Tom Salek, Jasia Steinmetz

■ Moderator: Vera Klekovkina (Assessment Coordinator)

Our Goals ■ Describe some sustainable assessment

practices developed on our campus

■ Give a quick preview of Campus Labs –an electronic assessment management system

■ Invite all participants to share or imagine other sustainable assessment practices that provide tangible and insightful assessment results while reducing the time and effort required

CULTURE OF ASSESSMENT VS. CULTURE OF EVIDENCE

Briefly, what is EVIDENCE?Assessment tools and results

Grades versus Assessment

Scoring rubrics & assessing rubrics

Direct & indirect measure assessment

Surveys & student responses

Informal anecdotes & rough estimations

DIVISION OF COMMUNICATION

SELECT ASSESSMENT PRACTICES:CLICKER QUESTIONS

Mark Tolstedt, Professor of Media Studies, Division of Communication, College of Fine Arts and Communciation

Assessment Methods and Measures:Direct Measures

– COMM 201 research presentations– COMM 400 ePortfolios– COMM 100 v 400 clicker questions– ARTM 480– COMM 400 InterviewStream

COMM 100 v. 400

■ Background. Both COMM 100 Introduction to Communication and COMM 400 Communication Capstone are required of all majors. COMM 100 is designed for first year, transfer and newly declared majors. Students are supposed to take the class before they earn form the 7 credits in Communication. COMM 400 is taken during a student’s last semester. During spring semester 2016, one question for each PLO was asked of both 100 and 400. Clickers were used for questions and answers. There were 100 students in COMM 100 and 60 in COMM 400.

Results■ PLO#1: Question:

Meeting face-to-face after interacting online for an extended period of time is likely to increase uncertainty (True or False)

– i. COMM 100 31% correct– ii. COMM 400 52% correct

■ PLO#2: Question: A theory is an idea that is unproven (True or False)

– i. COMM 100 84% correct– ii. COMM 400 62% correct

■ PLO#3: Question: Uncertainty reduction theory was the foundation for many contemporary theories of uncertainty (True or False)

– i. COMM 100 74% correct– ii. COMM 400 85% correct

■ PLO#4: Question Studying history allows us to assess evidence and interpretations (True or False)

– i. COMM 100 96% correct– ii. COMM 400 100% correct

■ PLO#5: Question The absolutist approach to decision making in any profession means that all decision must be based on the context of the individual situation (True or False)

– i. COMM 100 48% correct– ii. COMM 400 56% correct

COMM 100 vs. 400

48

96

74

84

31

56

100

85

62

52

0 20 40 60 80 100 120

PLO5

PLO4

PLO3

PLO2

PLO1

% giving correct answer

COMM 400 COMM 100

InterviewStream

■ Background. In the COMM 400 class, students are asked to find a job they want to apply for upon graduation, create both a resume and a cover letter for the job and then, using the Interview Stream software, apply for that job. Questions based on the Division’s three emphasis areas are programmed into the software. For assessment purposes, the Communication Assessment Committee identified questions asked that were tied to the PLO’s. 18 student interviews were randomly selected: 7 from the IPO emphasis, 7 from the MS emphasis and 4 from the PR emphasis. The PLO rubric was applied to the identified questions for each student

Results (IPO)■ PLO#2:

– Question: Give me an example of any time when you used tools such as survey data, library research or statistics as important contributors to definition of a specific problem.■ i. Developing: 2■ ii. Developed: 3■ iii. Mastered: 2

– Question: Describe a time when you were proud of your ability to use your mathematical knowledge or research techniques to solve a problem.■ i. Developing: 2■ ii. Developed: 4■ iii. Mastered: 1

PLO#3:– Question:Describe a project or

situation where you demonstrated your analytical abilities.■ i. Developing: 1■ ii. Developed: 2■ iii. Mastered: 4

PLO#5:– Question:Describe a situation in

which you had to draw a conclusion quickly and take speedy action.■ i. Developing: 0■ ii. Developed: 4■ iii. Mastered: 3

Clicker questions

■ Measure students’ knowledge■ Track students’ attendance

srs.campuslabs.com

19651

INDIRECT AND DIRECT MEASURES IN ONE

QUESTIONNAIREDavid Barry, Assistant Professor of Sociology, Department of Sociology

and Social Work, Colleges of Letters and Science

Department of SociologyStudent Questionnaire■ Questionnaire featuring

A) indirect measures that record students’ perception of their learning and their satisfaction with the instruction received and B) direct measure – an essay question – that records students’ understanding of the core concepts in sociology

Examples■ Perception of proficiency (indirect)

7. Statistical skills to conduct research about society.a. Great extentb. Moderate extentc. Small extentd. Not at all

■ Actual proficiency (direct)– (Closed-ended)

26. Which of the following is NOT an example of qualitative data collection?a. observations b. in-depth interviewsc. self-administered surveyd. content analysis

– (Open-ended)b. Based on the scenario you’ve chosen, identify the independent/dependent variables.

Reflection■ Strengths

– Uniform, quick (painless), and great for trend comparisons– Results valuable feedback

■ Difference between student perception and actual knowledge■ Allows for constructive discussion on what content is core to our discipline■ And how/where to make changes

■ Limitations– Not exhaustive due to device size and structure (e.g., methods measures)– Missing pre-test

SHARED SURVEYS : COMMUNITY

EXIT SURVEYS ON PROGRAM LEARNING OUTCOMES

Paula DeHart, Professor of Education, School of Education, College of Professional Studies

Exit Surveys

■ School of Education– World Languages and Literatures

COMMON RUBRICS Vera Klekovkina, Assessment Coordinator and Associate Professor of French,

Department of World Languages and Literatures, College of Letters and Science

PUTTING IT ALL TOGETHER

OUTCOMES MODULE IN CAMPUS LABS

OUTCOMES MODULE

Check Outcome Language & Bloom’s Taxonomy

Course Learning Outcome 1: To augment your vocabulary and improve your oral skills by practicing paragraph-length narrations.

Image source

Check Outcome Language & Bloom’s Taxonomy

Course Learning Outcome 1 (NEW): Students will be able to construct a paragraph-length narration in order to practice their oral skills in the target language.

Image source

Reworking Course Learning Outcomes

Diagnostics, Reflections, Measurement of Learning Outcomes

QUESTIONS, SUGGESTIONS, IDEAS?

■ AS Co-Chairs: Dave Dettman & Dan Graf ■ Vera Klekovkina (Assessment Coordinator) ■ AS Members:

– COFAC: Tom Salek & Matt Groshek– COLS: Maggie Bohm-Jordan, Vera Klekovkina & Dan Graf – CNR: Paul Doruska & Becca Franzen– CPS: Cuiting Li & Jasia Steinmetz – University College: Jennifer Huffman & Dave Dettman

THANK YOU!