MEASUREMENT,

SCALING, AND

QUESTIONNAIRE

Rahul Pratap Singh Kaurav, Ph.D. Asst. Professor, Marketing

Unit II

MBA 203

MEASUREMENT

Not everything that can be counted counts, and not everything that counts can be counted.

- Albert Einstein

What Do I Measure?

Measurement The process of describing some property of a phenomenon, usually by assigning numbers in a reliable and valid way.

Concept A generalized idea about a class of objects, attributes, occurrences, or processes

Operational Definitions - I

Operationalization The process of identifying scales that correspond to variance in a concept involved in a research process.

Scales A device providing a range of values that correspond to different characteristics or amounts of a characteristic exhibited in observing a concept.

Correspondence rules Indicate the way that a certain value on a scale corresponds to some true value of a concept.

Operational Definitions - II

Variable Anything that varies or changes from one instance to another; can exhibit differences in value, usually in magnitude or strength, or in direction.

Capture different values of a concept.

Constructs Concepts measured with multiple variables.

Levels of Scale Measurement

Nominal Assigns a value to an object for identification or classification purposes.

Most elementary level of measurement.

Ordinal Ranking scales allowing things to be arranged based on how much of some concept they possible.

Have nominal properties.

Interval Capture information about differences in quantities of a concept.

Have both nominal and ordinal properties.

Ratio Highest form of measurement.

Have all the properties of interval scales with the additional attribute of representing absolute quantities.

Examples - I

Examples - II

Mathematical and Statistical Analysis of Scales

Discrete Measures Measures that can take on only one of a finite number of values.

Continuous Measures Measures that reflect the intensity of a concept by assigning values that can take on any value along some scale range.

Three Criteria for Good Measurement

Sensitivity

Reliability Validity

Good Measurement

Reliability

Reliability The degree to which measures are free from random error and therefore yield consistent results.

An indicator of a measure’s internal consistency.

Internal Consistency Represents a measure’s homogeneity or the extent to which each indicator of a concept converges on some common meaning.

Measured by correlating scores on subsets of items making up a scale.

Split-half Method

Assessing internal consistency by checking the results of one-half of a set of scaled items against the results from the other half.

Coefficient alpha (α) [Cronbach’s alpha]

The most commonly applied estimate of a multiple item scale’s reliability.

Represents the average of all possible split-half reliabilities for a construct.

Validity

Validity The accuracy of a measure or the extent to which a score truthfully represents a concept.

Does a scale measure what was intended to be measured?

Establishing Validity: Is there a consensus that the scale measures what it is supposed to measure?

Does the measure correlate with other measures of the same concept?

Does the behavior expected from the measure predict actual observed behavior?

Validity (cont’d)

Face Validity A scale’s content logically appears to reflect what was intended to be measured.

Content Validity The degree that a measure covers the breadth of the domain of interest.

Criterion Validity The ability of a measure to correlate with other standard measures of similar constructs or established criteria.

Construct Validity Exists when a measure reliably measures and truthfully represents a unique concept.

Reliability V/s Validity

Sensitivity

Sensitivity A measurement instrument’s ability to accurately measure variability in stimuli or responses.

Generally increased by adding more response points or adding scale items.

SCALING

Refusing to have an opinion is way of having one, isn’t it.- Luigi Pirandello

What is An Attitude?

Attitude An enduring disposition to consistently respond in a given manner to various aspects of the world.

Components of attitudes: Affective Component

The feelings or emotions toward an object

Cognitive Component

Knowledge and beliefs about an object

Behavioral Component

Predisposition to action

Intentions

Behavioral expectations

Techniques for Measuring Attitudes

Ranking Requiring the respondent to rank order objects in overall performance on the basis of a characteristic or stimulus.

Rating Asking the respondent to estimate the magnitude of a characteristic, or quality, that an object possesses by indicating on a scale where he or she would rate an object.

Sorting Presenting the respondent with several concepts typed on cards and requiring the respondent to arrange the cards into a number of piles or otherwise classify the concepts.

Choice Asking a respondent to choose one alternative from among several alternatives; it is assumed that the chosen alternative is preferred over the others.

Attitude Rating Scales - I

Simple Attitude Scale Requires that an individual agree/disagree with a statement or respond to a single question.

This type of self-rating scale classifies respondents into one of two categories (e.g., yes or no).

Example:

Do you want to see Narendra Modi as PM of India?

_______ AGREE ______ DISAGREE

Attitude Rating Scales - II

Category Scale A more sensitive measure than a simple scale in that it can have more than two response categories.

Question construction is an extremely important factor in increasing the usefulness of these scales.

Example:How important were the following in your decision to visit

Shimla? (check one for each item)

VERY SOMEWHAT NOT TOO

IMPORTANT IMPORTANT IMPORTANT

CLIMATE ___________ ___________ ___________

COST OF TRAVEL ___________ ___________ ___________

FAMILY ORIENTED ___________ ___________ ___________

EDUCATIONAL/HISTORICAL ASPECTS_________________________________

FAMILIARITY WITH AREA ___________ ___________ ___________

Attitude Rating Scales - III

Likert Scale A popular means for measuring attitudes.

Respondents indicate their own attitudes by checking how strongly they agree or disagree with statements.

Typical response alternatives: “strongly agree,” “agree,” “uncertain,” “disagree,” and “strongly disagree.”

Example:It is more fun to play a cricket match than tennis one.

___Strongly Agree ___Agree ___Not Sure ___Disagree ___Strongly Disagree

Attitude Rating Scales - IV

Semantic Differential A series of seven-point rating scales with bipolar adjectives, such as “good” and “bad,” anchoring the ends (or poles) of the scale.

A weight is assigned to each position on the scale. Traditionally, scores are 7, 6, 5, 4, 3, 2, 1, or +3, +2, +1, 0, -1, -2, -3.

Example:

Exciting ___ : ___ : ___ : ___ : ___ : ___ : ___ Calm

Interesting ___ : ___ : ___ : ___ : ___ : ___ : ___ Dull

Simple ___ : ___ : ___ : ___ : ___ : ___ : ___ Complex

Passive ___ : ___ : ___ : ___ : ___ : ___ : ___ Active

Attitude Rating Scales - V

Image Profile A graphic representation of semantic differential data for competing brands, products, or stores to highlight comparisons.

Because the data are assumed to be interval, either the arithmetic mean or the median will be used to compare the profile of one product, brand, or store with that of a competing product, brand, or store.

Attitude Rating Scales - VI

Numerical Scales Scales that have numbers as response options, rather than “semantic space” or verbal descriptions, to identify categories (response positions).

In practice, researchers have found that a scale with numerical labels for intermediate points on the scale is as effective a measure as the true semantic differential.

Example: Now that you’ve had your automobile for about one year, please tell us how satisfied

you are with your Ford Taurus.

Extremely Dissatisfied 1 2 3 4 5 6 7 Extremely Satisfied

Attitude Rating Scales - VII

Constant-Sum Scale Respondents are asked to divide a constant sum to indicate the relative importance of attributes.

Respondents often sort cards, but the task may also be a rating task (e.g., indicating brand preference).

Example: Divide 100 points among each of the following brands according to your preference for the brand:

Pepsi _________

Limca _________

Slice _________

Attitude Rating Scales - VIII

Graphic Rating Scale A measure of attitude that allows respondents to rate an object by choosing any point along a graphic continuum.

Advantage:

Allows the researcher to choose any interval desired for scoring purposes.

Disadvantage:

There are no standard answers.

Some other type of graphical rating scales

Ladder Scale

Selecting a Measurement Scale

Some Practical Questions:

Is a ranking, sorting, rating, or choice technique best?

What type of category labels, if any, will be used for the rating scale?

How many scale categories or response positions are needed to accurately measure an attitude?

Should a balanced or unbalanced rating scale be chosen?

Should a scale that forces a choice among predetermined options be used?

QUESTIONNAIRE

How often misused words generates misleading thoughts.- Herbert Spencer

Questionnaire Quality and Design: Basic Considerations

Questionnaire design is one of the most critical stages in the survey research process. A questionnaire (survey) is only as good as the questions it asks—ask a bad question, get bad results.

Composing a good questionnaire appears easy, but it is usually the result of long, painstaking work.

The questions must meet the basic criteria of relevance and accuracy.

Decisions in Questionnaire Design

1. What should be asked?

2. How should questions be phrased?

3. In what sequence should the questions be arranged?

4. What questionnaire layout will best serve the research objectives?

5. How should the questionnaire be pretested?

6. Does the questionnaire need to be revised?

What Should Be Asked?

Questionnaire Relevancy All information collected should address a research question in helping the decision maker in solving the current business problem.

Questionnaire Accuracy Increasing the reliability and validity of respondent information requires that:

Questionnaires should use simple, understandable, unbiased, unambiguous, and nonirritating words.

Questionnaire design should facilitate recall and motivate respondents to cooperate.

Proper question wording and sequencing to avoid confusion and biased answers.

15–34

Wording Questions - I

Open-ended Response Questions Pose some problem and ask respondents to answer in their own words.

Advantages:

Are most beneficial in exploratory research, especially when the range of responses is not known.

May reveal unanticipated reactions toward the product.

Are good first questions because they allow respondents to warm up to the questioning process.

Disadvantages:

High cost of administering open-ended response questions.

The possibility that interviewer bias will influence the answer.

Bias introduced by articulate individuals’ longer answers.

Few Examples

What names of local banks can you think of?

What things do you like most about working for Federal Express? What do you like least?

Why do you buy more of your clothing in Give-&- Take than in other stores?

How would you describe your supervisor’s management style?

Please tell us how our stores can better serve your needs.

Wording Questions - II

Fixed-alternative (Close-ended) Questions Questions in which respondents are given specific, limited-alternative responses and asked to choose the one closest to their own viewpoint.

Advantages:

Require less interviewer skill

Take less time to answer

Are easier for the respondent to answer

Provides comparability of answers

Disadvantages:

Lack of range in the response alternatives

Tendency of respondents to choose convenient alternative

Types of Fixed-Alternative Questions - I

Simple-dichotomy (dichotomous) Question

Requires the respondent to choose one of two alternatives (e.g., yes or no).

Types of Fixed-Alternative Questions - II

Determinant-choice Question

Requires the respondent to choose one response from among multiple alternatives (e.g., A, B, or C).

Types of Fixed-Alternative Questions - III

Frequency-determination Question

Asks for an answer about general frequency of occurrence (e.g., often, occasionally, or never).

Types of Fixed-Alternative Questions - IV

Checklist Question

Allows the respondent to provide multiple answers to a single question by checking off items.

Phrasing Questions for Self-Administered,Telephone, and Personal Interview Surveys

Influences on Question Phrasing: The means of data collection—telephone interview, personal interview, self-administered questionnaire—will influence the question format and question phrasing.

Questions for mail, Internet, and telephone surveys must be less complex than those used in personal interviews.

Questionnaires for telephone and personal interviews should be written in a conversational style.

Example

Guidelines for Constructing Questions - I

Avoid complexity: Simpler language is better.

Guidelines for Constructing Questions - II

Avoid leading and loaded questions.

Avoid ambiguity: Be as specific as possible.

Guidelines for Constructing Questions - III

Avoid double-barreled items. Please agree or disagree with the following statement: Cars should be faster and safer.

How often and how much time do you spend on each visit to a hospital?

Does your department have a special recruitment policy for men and women?

Should the government spend less money on the military and more on education?

A: "So instead of murdering your neighbor, did you go

home and bake a cake which you donated to the Girl?"

W: "No."

A: "So you admit you murdered your neighbor!"

Guidelines for Constructing Questions - IV

Avoid making assumptions.

Avoid burdensome questions that may tax the respondent’s memory.

Did you saw any other celebrity endorsing this brand?

Make certain questions generate variance.

What Is the Best Question Sequence?

Order bias

Bias caused by the influence of earlier questions in a questionnaire or by an answer’s position in a set of answers.

Funnel technique

Asking general questions before specific questions in order to obtain unbiased responses.

Filter question

A question that screens out respondents who are not qualified to answer a second question.

Pivot question

A filter question used to determine which version of a second question will be asked.

What Is the Best Question Sequence?Example - I

What Is the Best Question Sequence?Example - II

What Is the Best Question Sequence?Example - III

What Is the Best Layout? - I

Traditional Questionnaires Multiple-grid question

Several similar questions arranged in a grid format.

What Is the Best Layout? - II

Traditional Questionnaires The title of a questionnaire should be phrased carefully:

To capture the respondent’s interest, underline the importance of the research

Emphasize the interesting nature of the study

Appeal to the respondent’s ego

Emphasize the confidential nature of the study

To not bias the respondent in the same way that a leading question might

Internet Questionnaires

Graphical User Interface (GUI) Software The researcher can control the background, colors, fonts, and other features displayed on the screen so as to create an attractive and easy-to-use interface between the user and the Internet survey.

Layout Issues Paging layout - going from screen to screen.

Scrolling layout – entire questionnaire appears on one page and respondent has the ability to scroll down.

Internet Questionnaire Layout - I

Push Button A small outlined area, such as a rectangle or an arrow, that the respondent clicks on to select an option or perform a function, such as submit.

Status Bar A visual indicator that tells the respondent what portion of the survey he or she has completed.

Radio Button A circular icon, resembling a button, that activates one response choice and deactivates others when a respondent clicks on it.

Internet Questionnaire Layout - II

Drop-down Box A space saving device that reveals responses when they are needed but otherwise hides them from view.

Check Boxes Small graphic boxes, next to an answers, that a respondent clicks on to choose an answer; typically, a check mark or an √ appears in the box when the respondent clicks on it.

Internet Questionnaire Layout - III

Open-ended Boxes Boxes where respondents can type in their own answers to open-ended questions.

Pop-up Boxes Boxes that appear at selected points and contain information or instructions for respondents.

Pretesting and Revising Questionnaires

Pretesting Process Seeks to determine whether respondents have any difficulty understanding the questionnaire and whether there are any ambiguous or biased questions.

Preliminary Tabulation A tabulation of the results of a pretest to help determine whether the questionnaire will meet the objectives of the research.