Conducting Surveys atCornell University

Marin Clarkberg, Associate Director Institutional Research and Planning

Yasamin Miller, DirectorSurvey Research Institute

Who We Are

Institutional Research and Planning (IRP)• Within the Division of Budget and Planning;

goal to inform institutional decision-making• Administers regular, large-scale surveys to

students and other University constituencies

Survey Research Institute (SRI)• Full service survey enterprise at Cornell• Designed, hosted, and analyzed hundreds of

surveys for non-profit, government, corporate, and Cornell clients

Rationales for surveys

Why do a survey?• Increasing call to have “real data” and to

assess processes and outcomes• Looks easy (and inexpensive… and maybe

even fun)

Why not do a survey?1. Lots of data is already available2. Survey fatigue3. Not as easy at it might seem at first4. Serious limitations on what survey data can

actually tell you about processes and outcomes

Why you shouldn’t:1. Data already exists

Everyone has some data…• Has existing data been thoroughly

analyzed and understood?

In addition, the University archives an enormous amount data about students (and other constituencies)• Students’ academic records• IRP Surveys, other surveys

Why you shouldn’t:2. Survey fatigue

Surveys are more common• IRP surveying regularly since 2000• CIT-hosted surveys, e.g. WebSurveyor:

oOver 1600 surveys in 2005-2006

Survey response rates are down• Senior Survey response rates:

61% in 1998; 50% in 2002; 45% in 2006

Student (staff, faculty) time is a university resource

Why you shouldn’t:3. There is such a thing as survey expertise

Question design• Bad questions give you bad data

Instrument design• Respondents often bail out of

unsatisfactory surveys

Sample design• Samples are often adequate to the task• Sampling saves all kinds of resources

Why you shouldn’t:4. Survey research has major limitations

Ability to generalize with survey data a function of response patterns• Respondents may differ in important

ways from nonrespondents

Surveys cannot demonstrate causation

What are the alternatives to surveys?

Ransack existing sources of data• Multiple sources of data help

“triangulate”• If nothing else, you can learn more about

what you don’t know (and thus what remains to be learned)

Look at alternate modes of collecting data• Some questions better addressed with

observation, focus groups, interviews, etc.

So when is a survey appropriate?

Existing data is well-understood• Unanswered questions clearly identified

Scope of survey will minimize the imposition on respondents• As short as possible, asked to as few as possible

Appropriate survey expertise in involved in designing the study• Allow ample time for coordination, consultation,

design and pre-testing

Shared understanding of study limitations• Start small and manage expectations

How to Survey

1. Develop a reasonable timeline2. Define your research questions3. Design the survey instrument

and sampling plan4. Develop a data security plan5. Notify and secure approvals6. Data analysis and reporting

1. Develop a reasonable timeline

Putting questions on the web, collecting responses, and even data analyses are the easy parts

Having established a clarity of task, four to six months is not an unreasonable amount to develop and pre-test a survey instrument

4. Develop a data security plan

Anonymous DataIdentities of respondents are never captured• Fewer data security concerns• Impossible to know who responded (or how often)• Impossible to link survey data with other data

sources

Confidential DataIdentities of respondents are kept, but secured

• Necessary to have very secure file storage• Possible to link survey data to other data of interest

5. Notify and Secure Approvals

IRP Survey CalendarNotification courtesy

Institutional Review BoardIf a project is “research” (i.e. develops or contributes to generalizable knowledge), it needs to be reviewed and approved by the IRB.

Data StewardsStudents: the Office of the Registrar

Student and Academic ServicesSAS-Research Group

Resources on campus

Find the help you need• Institutional Research and Planning (IRP)

o Serves the Universityo Stewards of much existing student survey datao Available to consult with institutional studies

(sampling plans, instrument design and review)

• Survey Research Institute (SRI)o Comprehensive survey services from initial planning to

data analysis and reportingo IRP uses SRI for survey hosting and administration

• Institutional Review Board for Human Participants

o First and last authority on mandated review requirements and processes – ask them.