MONDAY 13 TH MAY 2013

Research Methods

Finals Exam Review

The Exam

Is worth 15% of your grade

Will include:

- 40 multiple choice questions

- 3 short answer questions

How to ask Questions

Open-ended

Respondent is asked to provide his or her own answer

Produces narrative data

Closed-ended

Respondent selects an answer from a list

Choices should be exhaustive and mutually exclusive

Produces data that can be readily processed

Guidelines for Asking Questions

Make items clear – avoid ambiguous questions; do not ask “double-barreled” questions

Short items are best – respondents like to read and answer a question quickly

Avoid negative items – leads to misinterpretation

Avoid biased items and terms – do not ask questions that encourage a certain answer

Avoid terms that have bad associations

Questionnaire Construction

General questionnaire format – critical, must be laid out properly

Matrix questions – same set of answer categories used by multiple questions

Contingency questions – relevant only to some respondents – answered only based on their previous response

Ordering Questions

Ordering may affect the answers given

Estimate the effect of question order

Perhaps devise more than one version

Depending on whether it is in-person, or via mail, internet etc.

Begin with most interesting questions

End with less interesting, demographic data

Self-Administered Questionnaires

Can be home-delivered

Researcher delivers questionnaire to home of respondent, explains the study, and then comes back later

Mailed (sent and returned) survey is most common

Researchers must reduce the trouble it takes to return a questionnaire

Warning Mailings and Cover Letters

Used to increase response rates

Warning mailings – “address correction requested” card sent out to determine incorrect addresses and to “warn” residents to expect questionnaire in mail

Cover letters – detail why survey is being conducted, why respondent was selected, why is it important to complete questionnaire

Follow-ups to mail surveys increase response rates

Survey Research

Surveys are best suited for studies that have individuals as their units of analysis

Strengths: Surveys tend to be high on reliability and generalizability, but

validity can often be a weak point

Surveys useful in describing characteristics of large population

Survey Research

Weaknesses

Standardized questionnaire items often represent the least common denominator in assessing people‟s attitudes, orientations, circumstances, and experiences

Surveys often appear superficial in their coverage of complex topics

Difficult populations are hard to contact through customary sampling methods

Generally weaker on validity and stronger on reliability

Subject to recall error

Social desirability may be a problem

What is Field Research?

Field research is usually associated with qualitative data

It encompasses two different methods of obtaining data:

Direct observation

Asking questions

Often no precisely defined hypotheses to be tested

Used to make sense out of an ongoing process

Questions related to field research tend to be „How?‟ and „Why?‟

Characteristics of Qualitative Research

Context is critical

In-depth and detailed

Researcher immerses self in data

Bias is recognized

Inductive rather than deductive (bottom up)

Data are interpreted rather than analyzed

Roles of the Observer in Field Research

Full participant (e.g., a participant in a demonstration against stop and frisk)

Participant as observer (e.g., Perrone‟s research on drug use in New York dance clubs)

Observer as participant (e.g., observational studies of the police)

Complete observer (e.g., research in a courtroom setting, or setting that is open to the public)

Recording Observations

Note-taking, tape recording when interviewing and when making observations (dictation device)

Videotaping or photographs can make records of “before” and “after” some physical design change

Field notes – observations are recorded as written notes, often in a field journal; first take sketchy notes and then rewrite your notes in detail

Structured observations – observers mark closed-ended forms, which produce numeric measures

Strengths and Weaknesses of Field Research

Strengths:

Provides great depth of understanding

Flexibility (no need to prepare much in advance)

More appropriate to measure behavior than surveys

High on validity

Weaknesses:

Low on reliability – often very personal

Generalizability – personal nature may produce findings that may not be replicated by another

Precise probability samples can‟t normally be drawn

Agency Records

Published Statistics – government organizations routinely collect and publish compilations of data (e.g., NCVS, Census Bureau, BJS; often available in libraries and online

Nonpublic Agency Records – agencies produce data not routinely released (e.g., police departments, courthouses, correctional facilities)

New Data Collected by Agency Staff – collected for specific research purposes; less costly than collecting the data yourself and more control

Agency records are often used for descriptive studies

Problems with Reliability and Validity

Virtually all criminal justice record keeping is a social process – “social production of data”

Records reflect decisions made by CJ personnel as well as actual behavior by juveniles and adults

Discretion factors in keeping records

Criminal justice organizations are often more interested in keeping track of individual cases than in examining patterns

Potential for clerical errors due to volume of data

Users of data series collected over time must be especially careful

Imperative that you understand how the data were collected

Secondary Data Analysis

Sources – websites (BJS, NCVS, ICPSR, NACJD), libraries

Advantages

Cheaper

Faster

Benefit from work of skilled researchers

Disadvantages

Data may not be appropriate to your research question

Not useful for evaluation studies (which are designed to answer specific questions about specific programs)

Threats to validity

Content Analysis

Systematic study of messages – can be applied to virtually any form of communication

Decide on operational definitions of key variables

Decide what to watch, read, listen to & time frame of documents

Analyze collected data

Well-suited to answer “Who says what, to whom, why, how, and with what effect?”

Content Analysis

Systematic study of messages – can be applied to virtually any form of communication

Decide on operational definitions of key variables

Decide what to watch, read, listen to & time frame of documents

Analyze collected data

Well-suited to answer “Who says what, to whom, why, how, and with what effect?”

Examines content and meaning

Content Analysis

Essentially a coding operation

Communications need to be coded according to some conceptual framework

Choice between depth & specificity of understanding:

Manifest content – visible, surface content – similar to using closed-ended survey questions

Latent content – underlying meaning

Evaluation Research

It is gaining in popularity among researchers that really want to make a difference

Federal requirements for program evaluations often accompany the implementation of new programs

Funding is available for program evaluations

Can utilize a number of different research designs

Links the intended actions and goals of criminal justice policy to empirical evidence that supports them having the desired effects

Conditions Requisite for Randomized Experiments

Staff must accept random assignment and agree to minimize exceptions to randomization

Case flow must produce enough subjects in experimental and control groups for statistical tests

Experimental interventions must be consistently applied to experimental and withheld from control group

Need equivalence prior to intervention, and ability to detect differences in outcome measures after intervention

The Policy Process

Begins with a demand for a new course of action or opposition to an existing policy

Policy makers consider their ultimate goals and means of achieving those goals

Resources are allocated (considerations include; personnel, equipment, supplies etc.)

What are the policy outputs (i.e. what is actually produced?)

What is the impact of the policy output?

Interpreting Data

Empirical research is a logical rather than a mathematical operation

Statistics – branch of math appropriate to research

Descriptive statistics – used to summarize and describe data in manageable forms

Inferential statistics – assist in forming conclusions from our observations; usually about a population based on studying a sample

Types of Analysis

Univariate analysis – describing single variables (e.g. # of males; average age; place of birth etc.)

Bivariate analysis – describes the associations that connect one variable with another

Multivariate analysis – examines relationships among three or more variables

Data Analysis

Central tendency - mean, median, mode

Means are susceptible to extreme values. A few very large, or a few very small numbers can change the mean dramatically

Because of this, it is important to examine measures of dispersion

Simplest measure of dispersion is the range – the distance from the highest to the lowest value (e.g. 13 to 19 years)

Standard deviation – the average amount of variability in a set of scores (i.e. the average amount each individual observation varies from the mean)

The larger the standard deviation, the larger the average distance each data point is from the mean of the distribution

Rates

Fundamental descriptive statistics in criminal justice research

Used to standardize some measure for comparative purposes

Total Murders in Four States, 2004

Total Murders Total Population

California 2,407 35,894,000

Florida 946 17,397,000

Louisiana 574 4,516,000

Pennsylvania 650 12,406,000

Short Answer Questions

1. Compare and contrast the strengths and weaknesses of field research and survey research. Give specific attention to the topics of validity, reliability and generalizability

2. Discuss the advantages and disadvantages of secondary data and give an example of each

3. Identify the difference between the various roles of the observer in field research. Give an example of a research scenario that would be appropriate for each role

Final Class

Final Exam

Article critiques due