sampling. o sampling o sampling - the process of selecting observations o often not possible to...
TRANSCRIPT
CHAPTER 6
Sampling
INTRODUCTIONo Sampling - The process of selecting observations
o Often not possible to collect information from all persons or other units you wish to study
o Often not necessary to collect data from everyone out there
o Allows researcher to make a small subset of observations and then generalize to the rest of the population
OBSERVATION AND SAMPLING
o Polls and other forms of social research rest on observations
o The task of researchers is to select the key aspects to observe (sample)
o Generalizing from a sample to a larger population is called probability sampling and involves random selection
POPULATIONS AND SAMPLING FRAMES
o Findings based on a sample represent the aggregation of elements that compose the sampling frame
o All elements must have equal representation in the frame
SAMPLING FRAME
o That list or quasi list of units composing a population from which a sample is selected
o If the sample is to be representative of the population, it is essential that the sampling frame include all (or nearly all) members of the population
REPRESENTATIVENESS
o Representativeness - Quality of a sample having the same distribution of characteristics as the population from which it was selected
o EPSEM - Equal probability of selection method. A sample design in which each member of a population has the same chance of being selected into the sample
POPULATIONo The theoretically specified aggregation of study
elements
o Study population - Aggregation of elements from which the sample is actually selected
o Element - Unit about which information is collected and that provides the basis of analysis
RANDOM SELECTION
o Each element has an equal chance of selection independent of any other event in the selection process
PARAMETER VS. STATISTIC
o Summary description of a given variable in a population
o Summary description of a variable in a sample
THE LOGIC OF PROBABILITY SAMPLING
o Enables us to generalize findings from observing cases to a larger unobserved population
o Representative - Each member of the population has a known and equal chance of being selected into the sample
o Since we are not completely homogeneous, our sample must reflect – and be representative of – the variations that exist among us
CONSCIOUS AND UNCONSCIOUS SAMPLING BIAS
o What is the proportion of FAU students who have been to an FAU football game?
o Be conscious of bias – When sample is not fully representative of the larger population from which it was selected
o Equal Probability of Selection Method (EPSEM) A sample is representative if its aggregate characteristics
closely match the population’s aggregate characteristics; basis of probability sampling
PROBABILITY THEORY AND SAMPLING DISTRIBUTION
o Sample Element: Who or what are we studying (student)
o Population: Whole group (college freshmen)
o Population Parameter: The value for a given variable in a population
o Sample Statistic: The summary description of a given variable in the sample; we use sample statistics to make estimates or inferences of population parameters
PROBABILITY THEORY AND SAMPLING DISTRIBUTION
o Purpose of sampling: To select a set of elements from a population in such a way that descriptions of those elements (sample statistics) accurately portray the parameters of the total population from which the elements are selected
The key to this process is random selection
o Sampling Distribution: The range of sample statistics we will obtain if we select many samples
FROM SAMPLING DISTRIBUTION TO PARAMETER ESTIMATE
o Sampling Frame: list of elements in our population
o By increasing the number of samples selected and
interviewed increased the range of estimates
provided by the sampling operation
ESTIMATING SAMPLING ERROR
o If many independent random samples are selected from a population, then the sample statistics provided by those samples will be distributed around population parameter in a known way
o Probability theory gives us a formula for estimating how closely the sample statistics are clustered around the true value
Standard Error: A measure of sampling error
• Tells us how sample statistics will be dispersed or clustered around a population parameter
CONFIDENCE LEVELS AND CONFIDENCE INTERVALS
o Two key components of sampling error
o We express the accuracy of our sample statistics in terms of a level of confidence that the statistics fall within a specified interval from the parameter
o The logic of confidence levels and confidence intervals also provides the basis for determining the appropriate sample size for a study
PROBABILITY THEORY & SAMPLING DISTRIBUTION
SUMMED UP
o Random selection permits the researcher to link findings from a sample to the body of probability theory so as to estimate the accuracy of those findings
o All statements of accuracy in sampling must specify both a confidence level and a confidence interval
o The researcher must report that he or she is x percent confident that the population parameter is between two specific values
PROBABILITY SAMPLING: POPULATIONS & SAMPLING FRAMES
o Different types of probability sampling designs can be used alone or in combination for different research purposes
o Key feature of all probability sampling designs: the relationship between populations and sampling frames
Sampling frame: The quasi-list of elements from which a probability sample is selected
TYPES OF PROBABILITY OF PROBABILITY SAMPLING DESIGN
o Simple random sampling (SRS)o Systematic samplingo Stratified samplingo Cluster sampling
SIMPLE RANDOM SAMPLING
o Each element in a sampling frame is assigned a number, choices are then made through random number generation as to which elements will be included in your sample
Forms the basis of probability theory and the statistical tools we use to estimate population parameters, standard error, and confidence intervals
Feasible only with the simplest sampling frame
Not the most accurate method available
A SIMPLE RANDOM SAMPLE
SYSTEMATIC SAMPLING
o Systematic Sampling – Elements in the total list are chosen (systematically) for inclusion in the sample
List of 10,000 elements, we want a sample of 1,000, select every tenth element
Choose first element randomly
Danger: “Periodicity" A periodic arrangement of elements in the list can make systematic sampling unwise
Slightly more accurate than simple random sampling
Arrangement of elements in the list can result in a biased sample
STRATIFIED SAMPLINGo Stratified sampling: Ensures that appropriate
numbers are drawn from homogeneous subsets of that population
Method for obtaining a greater degree of representativeness—decreasing the probable sampling error
o Disproportionate stratified sampling: Way of obtaining sufficient # of rare cases by selecting a disproportionate #
To purposively produce samples that are not representative of a population on some variable
Stratificationo Grouping of units composing a population into
homogenous groups before sampling
o This procedure, which may be used in conjunction with simple random, systematic, or cluster sampling, improves the Representativeness of a sample, at least in terms of the stratification variables
Stratified Samplingo Rather than selecting sample for population at
large, researcher draws from homogenous subsets of the population
o Results in a greater degree of representativeness by decreasing the probable sampling error
A SRATIFIED, SYSTEMATIC SAMPLE WITH A RANDOM START
CLUSTER SAMPLING
o A multistage sampling in which natural groups are sampled initially with the members of each selected group being subsampled afterward.
MULTISTAGE CULUSTER SAMPLING
o Used when it's not possible or practical to create a list of all the elements that compose the target population
o Involves repetition of two basic steps: listing and sampling
o Highly efficient but less accurate
MULTISTAGE CLUSTER SAMPLING
o Compile a stratified group (cluster), sample it, then subsample that set...
o May be used when it is either impossible or impractical to compile an exhaustive list of the elements that compose the target population, (Ex.: All law enforcement officers in the US)
o Involves the repetition of two basic steps:
Listing
Sampling
NATIONAL CRIME VICTIMIZATION SURVEY
o Seeks to represent the nationwide population of persons 12+ living in households (≈ 42K units, 74K occupants in 2004)
o First defined are primary sampling units (PSUs)
Largest are automatically included, smaller ones are stratified by size, population density, reported crimes, and other variables into about 150 strata
o Census enumeration districts are selected (CED)
Clusters of 4 housing units from each CED are selected
BRITISH CRIME SURVEY
o First stage – 289 Parliamentary constituencies, stratified by geographic area and population density
o Two sample points were selected, which were divided into four segments with equal #’s of delivery addresses One of these four segments was selected at random, then
disproportionate sampling was conducted to obtain a greater number of inner-city respondents
Household residents aged 16+ were listed, and one was randomly selected by interviewers (n=37,213 in 2004)
NONPROBABILITY SAMPLING
o Technique in which samples are selected in a way that is not suggested by probability theory
o Reliance on available subjects: Only justified if less risky sampling methods are
not possible Researchers must exercise caution in
generalizing from their data when this method is used
NONPROBABILITY SAMPLINGo Purposive sampling: Selecting a sample on the
basis of your judgment and the purpose of the study
o Quota sampling: Units are selected so that total sample has the same distribution of characteristics as are assumed to exist in the population being studied
o Reliance on available subjects
o Snowball sampling - You interview some individuals, and then ask them to identify others who will participate in the study, who ask others…etc.
Purposive (Judgmental) Samplingo Selecting a sample based on knowledge of a
population, its elements, and the purpose of the study
o Used when field researchers are interested in studying cases that don’t fit into regular patterns of attitudes and behaviors
Snowball Samplingo Appropriate when members of a population
are difficult to locate
o Researcher collects data on members of the target population she can locate, then asks them to help locate other members of that population
Quota Samplingo Begin with a matrix of the populationo Data is collected from people with the
characteristics of a given cell o Each group is assigned a weight appropriate
to their portion of the populationo Data should represent the total population
CHAPTER 7
Survey Research and Other Ways of Asking Questions
INTRODUCTION
o Survey research is perhaps the most frequently used mode of observation in sociology and political science, and surveys are often used in criminal justice research as well
o You have no doubt been a respondent in some sort of survey, and you may have conducted a survey yourself
Survey research is the most frequently used methodo Fasto Cheapo Individual as the unit of analysis (usually)o Cross-sectionalo All types of research (exploration, description,
explanation, application)
Steps in Survey Research
o Target populationo Types of respondento Types of surveyo Develop the questionnaireo Pre/pilot test the instrumento Plan a system for recording answers
TOPICS APPROPRIATE TO SURVEY RESEARCH
o Counting Crime: asking people about victimization counters problems of data collected by police
o Self-Reports: dominant method for studying the etiology of crime
Frequency/type of crimes committed
Prevalence (how many people commit crimes) committed by a broader population
TOPICS APPROPRIATE TO SURVEY RESEARCH
o Perceptions and Attitudes: To learn how people feel about crime and CJ policy
o Targeted Victim Surveys: Used to evaluate policy innovations & program success
o Other Evaluation Uses: e.g., Measuring community attitudes, citizen responses, etc.
Chicago Community Policing Evaluation Consortium
o General Purpose Crime Surveys
GUIDELINES FOR ASKING QUESTIONS
o How questions are asked is the single most important feature of survey research
o Open-Ended: Respondent is asked to provide his or her own answer
o Closed-Ended: Respondent selects an answer from a list
Choices should be exhaustive and mutually exclusive
o Questions and Statements – (Likert scale)
Types of Questionso Open-ended questions
Respondent is asked to provide his or her own answer to the question
o Closed-ended questionsRespondent is asked to select an answer from among a list provided by the researcher
GUIDELINES FOR ASKING QUESTIONS
o Make Items Clear: Avoid ambiguous questions; do not ask “double-barreled” questions
o Short Items are Best: Respondents like to read and answer a question quickly
o Avoid Negative Items: Leads to misinterpretation
o Avoid Biased Items and Terms: Do not ask questions that encourage a certain answer
o Designing Self-Report Items: Use of computer assisted interviewing techniques
QUESTIONNAIRE CONSTRUCTION
o General questionnaire format – critical, must be laid out properly – uncluttered Be aware of issues with ordering items Include instructions for the questionnaire Pretest all or part of the questionnaire
o Contingency Questions: Relevant only to some respondents – answered only based on their previous response
o Matrix Questions: Same set of answer categories used by multiple questions
Guidelines for Questionnaire Construction Be aware of issues with ordering items. Include instructions for the
questionnaire. Pretest all or part of the questionnaire.
Contingency Questiono Survey question intended only for some
respondents, determined by their response to some other questions
Contingency Question Format
Matrix Question Format
ORDERING QUESTIONS IN A QUESTIONNAIRE
o Ordering may affect the answers given
o Estimate the effect of question order
o Perhaps devise more than one version
o Begin with most interesting questions
o End with duller, demographic data
This is opposite for in-person interview surveys
MAIL SURVEYo Costs o Warning letterso Consents o Follow up mailings o Postageo Response rate
MAIL SURVEY:RESPONSE RATE
o Number of people participating in a survey divided by the number selected in the sample
o Acceptable response rate 50% - adequate for analysis and reporting 60% - good 70% - very good
SELF-ADMINISTERED QUESTIONNAIRES
o Can be home-delivered
Researcher delivers questionnaire to home of sample respondent, explains the study, and then comes back later
o Mailed (sent and returned) survey is most common
Researchers must reduce the trouble it takes to return a questionnaire
WARNING MAILINGS & COVER LETTERS
o Used to increase response rates
o Warning Mailings: “Address correction requested” card sent out to determine incorrect addresses and to “warn” residents to expect questionnaire in mail
o Cover Letters: Detail why survey is being conducted, why respondent was selected, why is it important to complete questionnaire
Include institutional affiliation or sponsorship
OTHER ASPECTS OF SELF-ADMINISTERED QUESTIONNAIRES
o Monitoring returns: Pay close attention to the response rate, assign #’s serially
o Follow-up mailings: Nonrespondents can be sent a letter, or a letter and another questionnaire; timing
o Acceptable response rates: 50%? 60%? 70%? We would rather have a lack of response bias than a high
response rate?
COMPUTER-BASED SELF-ADMINISTRATION
o Via Fax, Email, Web Site/Page
o Issues
Representativeness
Mixed in with, or mistaken for, spam
Requires access to Web
Sampling frame?
IN-PERSON INTERVIEW SURVEYS
o Typically achieve higher response rates than mail surveys (80-85% is considered good)
o Demeanor and appearance of interviewer should be appropriate; interviewer should be familiar with questionnaire and ask questions precisely
o When more than one interviewer administers, efforts must be coordinated and controlled
o Practice interviewing
GUIDELINES FOR INTERVIEW SURVEY
o Dress in a similar manner to the people who will be interviewed.
o Study and become familiar with the questionnaire.
o Follow question wording exactly.o Record responses exactly.o Probe for responses when necessary.
TRAINING FOR INTERVIERSo Discussion of general guidelines and
procedures.o Specify how to handle difficult or
confusing situations.o Conduct demonstration interviews.o Conduct “real” interviews.
COMPUTER-ASSISTED INTERVIEWS
o Reported success in enhancing confidentiality
o Reported higher rates of self-reporting
Computer-assisted personal interview (CAPI) – Interviewers read questions from screens and then type in answers from respondents’
Computer-assisted self-interviewing (CASI) – Respondent keys in answers, which are scrambled so that interviewer cannot access them
TELEPHONE SURVEYS
o 95.5% of all households have telephones (2005, US Census Bureau)
o Random-Digit Dialing
Obviates unlisted number problem
Often results in business, pay phones, fax lines
o Saves money and time, provides safety to interviewers, more convenient
o May be interpreted as bogus sales calls; ease of hang-ups
TELEPHONE SURVEYSo Advantages:
Money and time Control over data collection
o Disadvantages: Surveys that are really ad campaigns Representativeness
COMPUTER-ASSISTED TELEPHONE INTERVIEWING
(CATI)o A set of computerized tools that aid telephone
interviewers and supervisors by automating various data collection tasks
o Easier, faster, more accurate but more expensive
o Formats responses into a data file as they are keyed in
o Can automate contingency questions and skip sequences
COMPARISON OF THE THREE METHODS
o Self-administered questionnaires are generally cheaper, better for sensitive issues than interview surveys
o Using mail: Local and national surveys are same cost
o Interviews: More appropriate when respondent literacy may be a problem, produce fewer incompletes, achieve higher completion rates
o Validity low, reliability high in survey research
o Surveys are also inflexible, superficial in coverage
SPECIALIZED INTERVIEWING
o Two variations: General interview guide: Less structured, lists
issues, topics, questions you wish to cover; no standardized order
Standardized open-ended interview: More structured, specific questions in specific order; useful in case studies, retrieves rich detail in responses
FOCUS GROUPS
o 12-15 people brought together to engage in guided group discussion of some topic
o Members are selected to represent a target population, but cannot make statistical estimates about population
o Most useful when precise generalization to larger group is not necessary
o May be used to guide interpretation of questionnaires following survey administration
STRENGTHS OF SURVEY RESEARCH
o Useful in describing the characteristics of a large population
o Make large samples feasibleo Flexible - many questions can be asked on a
given topic
WEAKNESSES OF SURVEY RESEARCH
o Can seldom deal with the context of social life
o Inflexible in some wayso Subject to artificialityo Weak on validity
SHOULD YOU DO IT YOURSELF?
o Consider start-up costs
o Finding, training, paying interviewers is time consuming and not cheap, and requires some expertise
o Mail surveys are less expensive, and can be conducted by 1-2 persons well
o The method you use depends on your research question