Addressing Assessment Fatigue:Generating Student Interest,

Participation, and Ownership in Assessment

Marissa CopeStudentVoice

Nathan LindsayUniversity of North Carolina Wilmington

Overview of Session

•General information on applicable research•Techniques for generating student participation•Coordination of assessment activity and student

ownership•Generating interest: Sharing results with students •Examples from UNCW and other campuses•Discussion questions

Audience participation will be requested throughout the session.

Assessment fatigue among students has been a challenge at our university.

17% 17% 17%17%17%17%

A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable

General Information• Survey response rates have been falling • Difficult to contact people• Refusals to participate increasing

• National survey response rates fell from about 60% in the 1960s to just above 20% in the 1980s

• Nonresponse may not be random

• Two strategies for correcting low response rates:1.Weight the data for nonresponse2.Implement strategies to increase response rates

Research on Survey Fatigue

• Multiple surveys are seen as a burden; would expect response rates to decrease as requests increase

• Time is a major issue

• Example - 1998 study at Air Force Academy:▫ Anecdotal evidence – survey about surveys▫ 97 percent: somewhat over surveyed▫ Should be surveyed only 3-4 times a year▫ Over surveyed: “combination of frequent surveys

perceived as irrelevant”

Implications from Research

• Potential of multiple surveys can reduce response rates

• Non-respondents cite time concerns as reason

• Effects of survey fatigue may be moderated by salience of survey

content

• Number of previous surveys may have an impact on current survey

response

• Survey fatigue may have biggest impact on survey administered

back-to-back

• Feeling of “I have done enough” (reciprocity)

pg. 66

Theories of Survey Response

1. Reasoned action approach• Calculation of costs and benefits• Social exchange: rewards, cost, and trust• Increase rewards• Reduce costs• Establish trust

2. Psychological approach▫Informal decision rules▫Norm of reciprocity▫Norm of social responsibility

Theories of Survey Response

•What this means in action:▫More likely to comply when request appears to be from

a legitimate authority▫Requests for help should be clear and straightforward in

survey invitation▫Emphasize that recipients are part of a select group

Techniques for Generating Student Participation

pro.corbis.com

Technique 1: Multiple Contacts

Pre-notifications:Provides information about the project including data

collection times, incentives, purpose, contact information

Invitation with survey information:Repeat of pre-notification information

Follow-up reminders:Ideally only to non-respondentsRange of 2-5 day intervals

We usually use pre-notifications for our surveys.

17% 17% 17%17%17%17%

A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable

Technique 2: Length

• The longer the survey the higher the perceived cost

• Moderate correlation between length and non-response

• Studies recommend no more than the following:• 22 questions• 13 minutes

• Also consider type of questions: text fields vs. answer choices

Our surveys are generally shorter than 22 questions.

17% 17% 17%17%17%17%

A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable

Technique 3: Incentives

• Research on the effect of incentives on college students is lacking

• Prepaid incentives consistently raise levels of response

• Effect of postpaid incentives is minimal• Payment contingent upon completion =

compensation• 2 effects:• Getting paid removes reciprocity aspect• Expectations for future participation

Incentives for assessments improve response rates at our institution.

17% 17% 17%17%17%17%

A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable

Technique 4: Salience

Salience: How important or relevant a survey topic is to the

survey recipient

• 12-14% increase in response rates for salient surveys

• Difficult to control

• Highlight salience in survey invitations

Technique 5: Statement of Confidentiality

Anonymous vs. confidentialAnonymous: responses are unidentifiableConfidential: can identify but will not share

• Voluntary statement assuring participants of confidence of information ascertained through survey (may be required)

• Reduces perceived cost

• Establishes trust

• Heightens awareness of what may be asked

• Misuse can lead to decrease in response rates

Other Techniques

Technique 6: Request for Help• Follows social responsibility theory• Be careful of wording in contact, especially with

subject lines

Technique 7: Sponsorship• Who is coordinating the survey• Collaborations with external entity may need

elaboration

Technique 8: Deadlines• Scarce/limited opportunity: more valuable• Results are mixed: won’t hurt, may not help

Contact/Invitation Checklist

Should convey the importance of the study

Should include a request for help

Should guarantee confidentiality (as appropriate)

Should provide information on how long it will take to complete the assessment

Should indicate the time period for which the survey will be accepting responses

Should include information on incentives

Timing of Contact/Administration

• Avoid busy times (e.g., finals) or holidays

• Send email/pre-announcement 2-3 days prior to survey mailing

• First half of semester/term may be better if you are surveying in an academic environment

• Mid-week invitations are typically recommended

Web Survey Practices•Possible ways of motivating respondents to

continue:

• Keep design simple

• Make directions clear

• Include questions on the first page

• Break the survey up into multiple pages

• Utilize a progress tool

• Utilize skip patterns so that respondents do not have to see questions that do not apply to them

Generating Interest, Participation, and Ownership in Qualitative

Research•Many of the same principles outlined for

quantitative assessment apply•For sampling, a purposeful sample is often the best

strategy, but captive audiences can also be used • Incentives (t-shirts, food, drinks) are helpful in

recruiting participants•Provide opportunities to discuss positive/negative

feedback•Students can help in data collection, analysis, and

reporting

Ownership: Include Students

When developing and administering the project, reviewing the data, and discussing potential changes/action, think about:

•Resident assistants•Peer mentors•Orientation leaders•Student club/organization leaders•Tutors/Supplemental Instruction leaders•Student employees

Coordination of Assessment Activity

• Overseeing body: person or committee that monitors activity for department, division, or institution

• Assessment calendar for division and institution• Think about what other areas on campus are

doing and collaborate with them• Partnerships between units with related missions and

objectives• Find opportunities to share assessment activities and results

(e.g., “Assessments of the Month”)• Share results widely (internally and externally): Publish an

Assessment Update

Sharing Results • Know your audience• Use drawings/other visuals to represent your assessment

plan • Use graphs/charts/other visuals to represent your findings

- Size/biggest or smallest- How things change over time- What is typical - What is exceptional- How one piece of data is related to another

• Share results back with assessment participants/ publicize assessment results in meaningful ways

Sharing Results with Students at UNCW • Reports posted on the Assessment Website, Other Websites

• See University Learning Center website: http://www.uncw.edu/stuaff/uls/si_leaders.htm

• Results shared with leadership groups (Student Government, Resident Assistants, Campus Entertainment Executive Board, Peer Mentors)

• Normative data on healthy behaviors shared in poster campaign

• “Your Voice Has Been Heard” Marketing Campaign• Space specific flyers within each department (e.g.,

Housing, Union)• Campus Television Station/Student Newspaper• Parent Newsletter

Our institution has effectively shared assessment results with students.

17% 17% 17%17%17%17%

A. Strongly AgreeB. AgreeC. Neither Agree nor DisagreeD. DisagreeE. Strongly DisagreeF. Don’t Know/Not Applicable

Other Examples from UNCW• Career Center Counseling Evaluations and Tutor Evaluations

in the University Learning Center• Web link on a laptop used instead of emailed survey

• Health Center and Housing and Residence Life• Bubble forms and paper/pencil surveys used to capture responses

from the captive audience• Health Survey

• Acronyms removed and incentives bolded in email invitation• Campus Life Study

• High salience and $100 incentives• Sustainability Film Series

• PDA’s used directly after the film

Additional Campus Examples

•University of Michigan - Spring Commencement▫New, temporary location for graduation▫Initial PDA survey followed by online surveys to graduating

students▫High publicity: emails, newspaper, website

•University of Richmond - First-Year Student Intervention▫Email from Vice President of Student Development▫6 questions about experience at UR▫Direct follow-up by UR staff if requested

Discussion: Incentives

•What incentives have you used at your institution?

•How have these affected participation rates?

•What recommendations do you have for using incentives?

Discussion: Sharing Results

•How have you shared results with your students?

•Have you noticed any effects on future participation?

•Are there other “best practices” for sharing results that you can pass along?

Discussion: Other Best Practices

•What other recommendations do you have for…▫generating student interest?▫ increasing student participation?▫enhancing student ownership in assessment?

www.chiuni.ac.uk

Thank you!

Please feel free to contact us with any questions or further ideas regarding these topics!

Marissa CopeAssociate Director of Assessment ProgramsStudentVoicemcope@studentvoice.com

Nathan Lindsay, Ph.D.Director of Student Life AssessmentUniversity of North Carolina Wilmingtonlindsayn@uncw.edu