Measurement in Survey Research

Developing Questionnaire Items with Respect to Content and Analysis

Measurement• "Measurement is the process by which scores or

numbers are assigned to the attributes of people or objects.

• "Measurement involves rules of assigning numbers to represent quantities of attributes."

• The most important aspect of measurement is specifying the rules for assigning the numbers to the characteristics that will be measured.

• Critically important with large samples or if data is to be analyzed with computer software.

Backward Research

• What type of analysis technique do clients understand?

• What type of numerical coding of items conforms to the analysis?

• What type of questionnaire items lend themselves to types of numeric coding?

• What type of variables can be measured with certain types of questionnaires?

What type of analysis technique do clients understand?• Counts, proportions

• Cross-tabulations

• Means and comparisons of means

• Correlation, regression techniques

What type of numerical coding of items conforms to the analysis?• Uniquely classify responses

• Show magnitude of a characteristic

• Report an average response

• Identify individuals with “0” on a characteristic from a group having it.

What type of questionnaire items lend themselves to types of numeric coding?

• Open ended—do not lend themselves to coding.

• Forced choice, choosing among a set of prewritten options components.

What type of variables can be measured with certain types of questionnaires?

• Choice, current cell phone carrier.

• Frequency, usage, minutes

• Attitudes, liking, disagreements, agreements

• Amounts, prices for specific features

Concepts and Constructs:The characteristic we wish to measure.

• Concepts and constructs are interchangeable terms for what is measured.

• Constitutive definition: Delineates the major characteristics of a given construct.

Operational Definition

• Taking a natural object and "converting" it into a variable (or set of variables), something that takes on different values.

• The measurement / operational definition translates the constitutive definition into steps that must be followed in order to assign numbers to the construct.

Levels of Measurement

• Nominal coded data: Ability to distinctly categorize, or conversely, allows the determination of equality.

• Ordinal coded data: Allows determination of magnitude, rank, greater or less than.

• Interval coded data: Captures the distance apart two or more respondents are with respect to an attribute.

• Ratio data possess a natural or absolute zero, indicating a true absence of a characteristic. Permits statements concerning the equality of ratios.

Each level bears all the properties of the one listed above it.

Nominal or nominally-coded data• System designed to code an object in one

and only one of a set of mutually exclusive classes.

• Variables such as gender, religious denomination, and political affiliation are typically nominally coded.

• Sometimes variables are called discrete, categorical, qualitative or non-metric.

• Sample proportions are the appropriate inferential statistics.

Checklist Items

• Series of individual nominally coded (0/1) measures.

• Each “box” becomes a column in a data file.

• The researcher may choose to summate scores on these variables into another item.

Ordinal or ordinal-coded data

• Indicate the "rank," or magnitude of an attribute.• Identify which objects have "more" or "less" of an

attribute.• Typically used with rank-ordering tasks in

consumer research, i.e., "list the following brands in the order of your preferences."

• Remember, the purpose is simply to distinguish order, nothing more -- limits the statistical tests that can be performed to sample proportions.

Nominal and Ordinal Items

• Inferential statistics are limited to sample proportions, percentages

• Important when attempt to identify the largest segment, developing

• Used for classification purposes, splitting the sample, forming groups

Checklist Items

• Series of individual nominally coded (0/1) measures.

• Each “box” becomes a column in a data file.

• The researcher may choose to summate scores on these variables into another item.

Interval or interval-coded data• Differences between numbered coding are

proportional to the differences in the concept/trait.

• Allow interpretation of how much more a respondent has of an attribute, or how far apart two or more respondents will be regarding an attribute.

• Permit calculation of means and standard deviations, such that a broad range of statistical tests can be performed.

Likert item:People should shop at local merchants even though the prices may be significantly higher.

1 . . . . . . 2 . . . . . . 3 . . . . . . 4 . . . . . . 5Stronglydisagree

NeutralDisagreeStrongly

agreeAgree

Numeric differential item:Paying higher prices at local merchants...

1 . . . . 2 . . . . 3 . . . . 4 . . . . 5 Showsintelligence

Showsignorance

Ratio data

• Ratio data possess a natural or absolute origin, or legitimate zero point.

• Coding allows statements regarding ratios--a 10 versus a 5 means one respondent has twice as much of a variable.

Customer Satisfaction

• Degree of satisfaction

• Meeting or exceeding expectations

Satisfaction Approach• Overall, how satisfied are you with Farm Credit Services as

a source of financing for your agricultural operation? Would you say you are: (PLEASE “X” ONE ANSWER)

(1) Very satisfied(2) Satisfied(3) Somewhat satisfied(4) Neither satisfied nor dissatisfied(5) Somewhat dissatisfied(6) Dissatisfied(7) Very dissatisfied

Expectations Approach

• Now thinking about your entire relationship with Farm Credit Services, which of the following best describes how well FCS is doing overall in relation to your expectations for what an ag lender can and should do? : (PLEASE “X” ONE ANSWER)

(1) Exceeding your expectations(2) Meeting your expectations (3) Almost meeting your expectations(4) Failing to meet your expectations

Ordinal or ordinal-coded data

• Indicate the "rank," or magnitude of an attribute.• Identify which objects have "more" or "less" of an

attribute.• Typically used with rank-ordering tasks in

consumer research, i.e., "list the following brands in the order of your preferences."

• Remember, the purpose is simply to distinguish order, nothing more -- limits the statistical tests that can be performed to sample proportions.

Interval or interval-coded data• Differences between numbered coding are

proportional to the differences in the concept/trait.

• Allow interpretation of how much more a respondent has of an attribute, or how far apart two or more respondents will be regarding an attribute.

• Permit calculation of means and standard deviations, such that a broad range of statistical tests can be performed.

Likert item:People should shop at local merchants even though the prices may be significantly higher.

1 . . . . . . 2 . . . . . . 3 . . . . . . 4 . . . . . . 5Stronglydisagree

NeutralDisagreeStrongly

agreeAgree

Numeric differential item:Paying higher prices at local merchants...

1 . . . . 2 . . . . 3 . . . . 4 . . . . 5 Showsintelligence

Showsignorance

Ratio data

• Ratio data possess a natural or absolute origin, or legitimate zero point.

• Coding allows statements regarding ratios--a 10 versus a 5 means one respondent has twice as much of a variable.

Items with ratio properties:

• In the last month, how many checks have written at establishments serving alcoholic beverages.

• How many charges to your credit card were at locations serving alcoholic beverages.

• How many purchases to liquor stores have you made in the last month.