Constructing questions for interviews & questionnairesTheory and practice in social researchWilliam Foddy

PrefaceThese slides include one theoretical framework for constructing questions and a small set of good and bad practices for constructing questionsThis is not only or complete set, but gives hints about what should you think when constructing questions.

Errors in gathering data through survey procedures areResponders failure to understand questions an intendedA lack of effort or interest on the part of respondersRespondents unwillingness to admit certain attitudes or behavioursThe failure of respondents memory or comprehension process in the stressed condition of the interviewInterviewers failures of various kinds (e.g. the tendency to change wording, failures in presentation procedures and the adoption of faulty recording process

Examples that illustrate the inadequate of many of the questionsFactual questions sometimes elicit invalid answer: As simple as ageRelationship between what respondents say and what they actually do in not always very strong: E.g. racism: Questions: Do you let a black person inside a bar? Do they really?Responders attitudes, beliefs, opinions, habits interests often seem to be extraordinary unstableSmall changes in wording sometimes produce major changes in the distribution of responsesRespondents commonly misinterpret questionsAnswer to earlier question can affect to later questionOrder of options changes the respondents: e.g. middle optionFormat of question: Open vs. closeRespondents answer even if they don't know about the topicCultural context affects

The principal assumptions that have defined the general orientation adopted by survey researcher in the pastThe researcher has clearly defined the topic about which information is requiredRespondents have the information that researcher requiresRespondents are able to access the required information under the conditions of the research situationRespondents can understand every question as intendedRespondents are willing (or at least, can be motivated) to give the informationResponses are more valid if they know why that is askedAnswers are more valid if they are not suggestedThe research situation does not affect to the resultsThe process does not affect the respondentAll responses are meaningfully comparable

The traditional survey model

The key issue: the comparability of aswerThe reseacher must be clear about the nature of the information required and encode a request for this formatThe respondent must decode this request in the way the researcher intends it to be decodedThe respondent must encode an answer that contains the information researcher has requested The reseacher must decode an answer as the respondent intendedd it to be encoded

Symbolic interactioinist theoryHuman beings interpret and define each other's actions. Not only simple stimulus-responseHuman beings can be objects of their own attention. Argue themselves, take pride themselvesConscious social behavior is intentional behavior. Situation affects.Interpreting, planning and acting are ongoing process which begin anew at every stage of a social processIn situation there is a role and person wants to hold on it.Always existsIn brief: social situation is constantly negotiating a shared definition of the situation; taking one another's viewpoint into account; and interpreting another's behavior

A model of symbolic interactionist view of question answer behaviour

The implication of the symbolic interaction theory for social researchRolesRespondents try to find a mutually shared definition of the situationInterpretation of researcher actsClues of what kind of information researcher wants

OtherMultidimensionality. Nothing is one-dimensionalLevel of generality in the statementsLevel of generality in the topicUtterance frame: descriptive, explanatory or evaluative

Defining topic properly

Researcher has clearly defined the required informationDefined topicGlobal vs. local (29)Dimensions (economic, technical...)Is PL good thing: Marketing vs. Technical E.g. better social secure in cost of taxesResearcher has clear idea what kind of information is required

Responder have required informationHypothetical question => hypothetical AnswerWillingness to answer vs. information to answerOpinions vs. factsGrounded opinionsSerious opinionsResponds are capable of verbalising information.Not hurry, give time to answer

Formulating intelligible requests for information

The meaning of individual wordsContext specific meanings Ambiguities wordsDifferent meaningsCultural decenciesRelative difficulty of words foreign wordsThe operation of unintended nuances associated with apparently similar wordsForbid vs. allow

The meaning of conceptsLack of empirical referencesHow should beDifficul, unclear, wide conceptsContext decencies (often, usually etc.)Concrete: Not: often, quite often, seldom. Use 1,2,3,4,5

Structural complexitiesNumber of wordsLess is better, but not complex onesGrammaticalAsking too much in one questiondivide & conquernot what you like this and that!Not too much negatives: Double is bad

Good practicesDescriptive introduction: I'd like to describeExplain before question The addition of clauses, phrases and instructionsAnecdotesEducation of the respondent affects

Contextual influences on respondents' interpretation of questions

Clues afforded by either the Question itself or its componentsClues associated with the questionSome existing idea alwaysLeading questionsClues in componentsAdditional phrasesNote: Vegetables e.g. spinach not associated vegetarian in general as intended

Clues in response options.Approve or disapprove - not only approve in questionScalePre-set response options as memory cluesMay not remember other optionsInformation range covered by the options.Who counts own averages?Abstract: no knowledge of normative levels Response biased with number of optionsRespondent favor first when reads, last when hears - vary the orderLet responder tell, if no-one match

Impact of preceding questionsInfluence associated with prior Q. Issues:Does one have any knowledge?How deep is knowledge or how thoroughHow one interpretsPsychological need for consistentNeed to stay in one opinionEven-handednessFair for all sides

Impact of preceding questions (contd)The impact of the previous answersThe impact of the overall situationSemantics of human mind.Willingness to give information

Contextual influences on respondents' interpretation of questions

Descriptive accountsHow to describe the situationPerspectives

Level of social generality of responsesYou? Singular or plural?

ExplanationsRespondent can frame an explanation in many different waysWhy did you do XCausal antecedent - what caused to do XGoal antecedent - purpose for XEnablement factors - how possibleCausal consequences - what happened after XThe researcher's expectations

EvaluationsAlways relativeStandards necessary does not existsEvaluative standard that are external to the questionPleasant/easy flight - pilot vs. passengerEvaluative standard that are built in to the questionAgree -disagree

Limitations of human memory

Limitations of human memoryLong term memory problemsNobody remembers allAsk from present to past, not past to present.Short term memory problemsSimple questions

Filters

FiltersEstablishing the relevance of questions to respondentRespondents tend to answer all the questions I don't know enoughDon't know vs. don't have opinion Position of filtersMiddle and dont know are problematicMiddle category at allsubjective viewpoint to strongly (dis)agreeHow strongly?

Reducing question threat

Reducing question threatAlways some threatBiasedRefuse

How?Casual, Do you happen to have?Imputation of deviance, You know everyone does.AnonymousLessen psychological immediacy of the Q (e.g. other instead self. Numeric coding of alternatives)Decrease specificity of the information called (e.g. broad response categories)Adopt knowing so that respondents have to confirm rather than volunteer'Kinsey' straight at eyes and askAdopt indirect so that respond give answer without knowingPlace threatening Q at the end of series

How? (contd)Door in the face: Ask direct, if does not answer ask indirect/about e.g. salary.Ask long, dont hesitate top repeat, give time and encourage to use timeThe definition of Q threat

Threat causes biasTopics that are desirable and over-reportedBe good citizenBe well informed and cultural personHave fulfilled of moral and social responsibilitiesTopics that are socially undesirable and under-reportedIllness and disabilitiesIllegal and contra-normative behaviorFinancial statusFeeling of guilt or personal fearsThreat associated with the nature of the relationship between interviewer and respondSocial equityFear of political or economical sanctions

Open vs. closed question debate

Open vs. closed question debateCoding responses to open questionsFormulating response option for closed question

Evaluation of the use of open questionsOpen Q doesnt suggest answer -- or does it?Respondents use of "probes" Indicate level of knowledgeAssumptionswill answer open Q if knowsdon't try to answer open Q if don't knowwil answer closed question if don't knowAnswers indicate the salience of the topic in the responder's mindIs there evidence?Avoid format effectNot in the middle! Is it in the closed?

Evaluation of the use of open questionsAllows complex motivationIndicates more how respondent has interpreted itIndicate motivation that have influenced respondent's orientation to the topicIndicates the frame of referenceProblems associated with probing inadequate answer Turns to closeProblems associated with coding response!

Evaluation assumptions associated with closed questionsAnswers the question in the same way -meaningfully comparable?Easier to answer More easily analyzedInterpretationsProblems associated with recording responses to closed questionsStart with general instructionsExplain why to answer

Measuring attitudes

Measuring attitudesList of respond alternatives p. 153Define topic clearlyApplicability of the topic to respondents has to be establishedDont knowRespondents has to know what sort of answers they should knowLevel of generalitySpecs of standards"Strongly agree"

Measuring attitudesStimulus centered effects Number of categories7 +/- 2Anchoring effects of he category labelsThe word in the positive/negative end?Problems in the batteries of rating scalesMeaning can alterSome categorylong list. Don't know last when answering the firstAmbiguity

Checks to ensure that questions work as intended

Checks to ensure that questions work as intendedEditing rules (list p. 184)Piloting QuestionObservation hard, but must be done in the beginning to ensure that questions work e.g. had to repeat?Question testingRephrase the question in responder's own wordsDouble interviewCome in the beginningAllow aloud thinking

Esim KKKKuinka monta parametroitua komponenttia on tyypillisess toimitettavassa tuotteessa (0=ei kytet)? Miten paljon parametreja on tyypillisess parametroidussa komponentissa?Parametrien mrn vaihteluvli?Millaisia parametreja komponenteissa on?Millaisia arvoalueita parametreilla on (kokonaisluku, arvoalue, yksi kokonaisluku joukosta , yksi annetuista vaihtoehdoista, reaalilukuja, joukkoja)? Mit muita?