Sampling issues and Sampling issues and survey design survey design

Leigh Estabrook Leigh Estabrook

AndAnd

Ed LaknerEd Lakner

Fundamental issues – in all Fundamental issues – in all researchresearch

To what questions do you want to know the answer?To what questions do you want to know the answer? What information will be able to help answer those What information will be able to help answer those

questions?questions? How much do you already know? Are you at the How much do you already know? Are you at the

beginning of exploration or beyond?beginning of exploration or beyond? Do you want/need to be able to know/infer from your Do you want/need to be able to know/infer from your

data what is going on over a certain population?data what is going on over a certain population? How will you design your research to best answer those How will you design your research to best answer those

questionsquestions DO YOU HAVE TO GET THAT INFORMATION FROM DO YOU HAVE TO GET THAT INFORMATION FROM

PEOPLE?—are there secondary alternatives?PEOPLE?—are there secondary alternatives?

Survey Research—when what you Survey Research—when what you want to know depends on what want to know depends on what

other people know, think, doother people know, think, do A way to look at the relationship between 2 or A way to look at the relationship between 2 or

more variablesmore variables Part of a tool-kit of methodsPart of a tool-kit of methods Complementary to qualitative researchComplementary to qualitative research Suitable for answering some questions but not Suitable for answering some questions but not

othersothers Valuable when seeking to generalize to a larger Valuable when seeking to generalize to a larger

populationpopulation Possible when questions can be defined and Possible when questions can be defined and

clearclear

Before you begin….Before you begin…. Theory constructionTheory construction

What are your assumptions about how people What are your assumptions about how people behave?behave?

At the base of how we shape our work—the At the base of how we shape our work—the reason for reading theoristsreason for reading theorists

Answering the “so what” questionAnswering the “so what” question Thinking about the logical relationships Thinking about the logical relationships

between variables— what do you need to between variables— what do you need to know to explain what you find out—can know to explain what you find out—can you diagram your studyyou diagram your study

New challengesNew challenges

Increased attention to rights to privacyIncreased attention to rights to privacy

At same time, increased ability to violate At same time, increased ability to violate privacy of recordsprivacy of records

Over use of polling dataOver use of polling data

Counter measures such as no-call lists, Counter measures such as no-call lists, anti-spamanti-spam

Fundamentals of Survey Fundamentals of Survey researchresearch

Universe and SamplingUniverse and Sampling

Forms of data collectionForms of data collection

How to ask and design questionsHow to ask and design questions

Universe and SamplingUniverse and Sampling

Universe definition—what is it? What are Universe definition—what is it? What are the boundaries?the boundaries?

Sampling frame—what you can get often Sampling frame—what you can get often determines universe definition (e.g., do determines universe definition (e.g., do people have telephone, driver’s license, people have telephone, driver’s license, email address)email address)

Two will interactTwo will interact

Sampling and sample size — Sampling and sample size — issues of ALA Salary Surveyissues of ALA Salary Survey

issue of whether to stratify sample into issue of whether to stratify sample into subgroups? Partitioning population subgroups? Partitioning population related to survey objectives; but also the related to survey objectives; but also the size of populationsize of population

If you are generalizing to a larger If you are generalizing to a larger population, need to decide on margin of population, need to decide on margin of error you are willing to tolerate. That error you are willing to tolerate. That determines sample size and also determines sample size and also expected and necessary response rate. expected and necessary response rate.

Forms of data collectionForms of data collection

Telephone—finding numbers; using RDD; Telephone—finding numbers; using RDD; choosing household memberchoosing household member

Postal Mail—finding addresses, getting Postal Mail—finding addresses, getting responseresponse

Face to Face—finding people, choosing Face to Face—finding people, choosing household member; managing costshousehold member; managing costs

Email—access and accuracyEmail—access and accuracy Importance of using multiple modes—e.g., Importance of using multiple modes—e.g.,

Mellon StudyMellon Study

How to ask and design questionsHow to ask and design questions

To obtain high response rate—real To obtain high response rate—real person, certain population groups, types of person, certain population groups, types of delivery (e.g., certified mail)delivery (e.g., certified mail)

Ask questions people Ask questions people can can answeranswer Do they have a reason to knowDo they have a reason to know Are they allowed to answer—Are they allowed to answer— Would they be in jeopardy for doing soWould they be in jeopardy for doing so Will they be embarrassed answering truthfullyWill they be embarrassed answering truthfully

Analysis—beyond scope of todayAnalysis—beyond scope of today

Deriving meaning and understanding from Deriving meaning and understanding from survey findingssurvey findings

Requires variety of statistical approachesRequires variety of statistical approaches Will want to think about your analysis Will want to think about your analysis

BEFORE going into the field with the BEFORE going into the field with the survey—it will tell you whether the survey—it will tell you whether the relationships you expect do, in fact, existrelationships you expect do, in fact, exist

Recommend adding at least 2 semesters Recommend adding at least 2 semesters of stats to your portfolioof stats to your portfolio

Questions and issuesQuestions and issues

Clarifying points that are still unclearClarifying points that are still unclear Thinking about your own projectsThinking about your own projects