Measurement

MeasurementObjectives:design different question typeswhen to use these typesadvantages/disadvantagescoding for subsequent data analyses

IssuesResponse FormatMeasurement Level of question

Response FormatsOpen endedsimpleprobed (looking for more details)Closed endeddichotomous (two choices only) Do you like this classYesNomultiple category (need to decide how may are to be picked)Do you like this class? 1=not at all2= somewhat3=like4=like very much5=like a lotScaled response (a type of close ended)LabeledDo you like this class? 1=not at all2= somewhat3=like4=like very much5=like a lotUnlabeledDo you like this class? 12345

Levels of MeasurementCATEGORICAL SCALESOrdinal and Nominal Scalesto label objectse.g., gender, occupation, yes/no, school yearTo indicate rank ordere.g., preference, higher than/lower than, best liked/least liked

Types of Analysiscounts, percentages

Levels of MeasurementMETRIC SCALESInterval and Ratio Scalesfor ratinge.g., attitude scales, agree/disagree, actual numbere.g., how many times

Types of Analysis counts, percentages, averages, correlations,

Comparing Metric and Categorical ScalesWhat is your age? _____ yearsIndicate the age range you belong to< 1818-2525-3536-4546-5556-65> 65

Other ConsiderationsHow many categories for rank orderHow many scale items? (sensitivity)Do you like this class? 1= not at all like2345=like very muchDo you like this class? 1= not at all like 2 3 45 6 7 8 9 10 11 12 13 14 15=like very muchEven or odd number of scale itemsBalanced versus unbalanced scalesPlease rate the service at this restaurant 1= very poor2=poor3=neutral4=good5=very goodPlease rate the service at this restaurant 1= poor2=fair3=good4=very good5= excellent

Commonly Used ScalesLikert Scale5 point agree/disagree scale with a set of statementsmeasures intensity of feeling

Semantic Differentialbipolar adjectives as anchorsscale levels are usually not given descriptorstypically 7 point scales are usedused for evaluation of objects, brands, etc.

Reverse Scaling IssuesWhy do you need reverse scalingAvoid halo effectCheck to see if respondent is paying attentionConsistency of responses

I consider Brand A to be a quality brand1= strongly disagree2=disagree3=neutral4=agree5=strongly agreeBrand A is poorly made1= strongly agree2=agree3=neutral4=disagree5=strongly disagree

Some other scalesgraphic scale

Stapel scalenegative and positive numbers

Continuous scale

Constant Sum scale

How to choose a questionquestionnaire administration method (data collection mode)ability of the respondentwhat kind of analysis do you need (scale level desired)e.g., do you want to report a count, a percentage, an average Previous research studies

What makes a good scale?1. Reliabilityconsistency of scale2. Validityhow accurate is the scale

Testing for Reliabilityadministering same questions to same respondents at two different timesby using call backsrespondents may remember earlier responsesif time gap is too long, conditions may changeUse reverse scaled itemsRepeat some items

Testing for ValidityFace validitydoes the question measure what it is supposed to measurethis is a subjective evaluation