tqm in interview data collection process matti simpanen
TRANSCRIPT
TQM in Interview Data Collection ProcessMatti Simpanen
22.9.2010 2M.Simpanen
Data collection=Interviews!Data collection=Interviews!
Weighting, corrections
Analyse, reporting
Quality outcome Good reputation and glory!
Survey plan, sampling
Research problem
22.9.2010 3M.Simpanen
Interview is not a constant variable!
My mission is try researchers to bear in mind the data collection should not be forgotten in quality reports
Researchers should demand quality in data collection process, and not to keep it as given, constant quality
Also financiers should consider the quality of data, not only how get data with lowest expensies
22.9.2010 4M.Simpanen
Topics
The employment system The recruitment process Interviewer training The standardised interviewing method Setting the objectives Quality monitoring
22.9.2010 5M.Simpanen
Survey process is a coherent process
A survey is a process in which every individual stage is influenced by the other stage
Every part of the survey process has to have adequate understanding of the survey process, its backgrounds, goals and operating principles
Even interviewers or other field workers Needs continuous quality monitoring (various perspectives)
Interviewers employment system What kind of employment system helps to keep quality high?
Remuneration From what interviewers are paid? Efficienty or quality? System has to be encouraging and fair
22.9.2010 6M.Simpanen
Quality starts from the recruitment process Systematic recruitment process
to measure applicant’s ability to understand and follow the standardised interview method (see later)
to measure interaction and style of interview (positiveness) to measure ability to adopt backgrounds and goals of surveys
quickly (general understanding of surveys and interview work) to measure general commitment abilities and a tolerance of routines Interviewers represent survey organisation, are impartials
In Statistics Finland recruitment process divides into three phases: Phone contact (systematic evaluation of applicant's voice,
interaction, etc). Structured job application form (internet-based) Job interviews (f-to-f)
22.9.2010 7M.Simpanen
The quality demands training and orientation To orietate interviewers / other field workers to survey process
basic training, project training, further training Basic training
The goal is to orientate new interviewers to do interviewer’s work to familiarise interviewers to standardised interviewing method to orientate new interviewers to the survey process as a whole to orientate new interviewers to the organisation (commitment)
In SF one week basic training + repetition and follow-up after 1-2 months
22.9.2010 8M.Simpanen
Interviewer skills need continuing training
Further trainingto develop and to maintain interviewers skillsto motivate interviewersto commit interviewers to the survey process and to the quality
work
to repeat items of standardised interviewing method Goal of training is to get interviewers understand their role in the
survey process and the value of their work
22.9.2010 9M.Simpanen
Questionnaire standardsInterviewing techniquesInterviewing styleInterview interaction
Standarised interviewing method
22.9.2010 10M.Simpanen
Questionnaire standards
Structured way to make questions, answer alternatives, introductions, descriptions to the questionnaires
factual and other (opinion) questionsstandards are guiding the interviewer helps researcher to formulate questions
for example does the question really measure the researcher wants it to measure
Helps the interviewer realise how to ask different kind of questionsgives time to interviewer to use his/her skills of interaction to get
the interviewsinterviewer´s work is to do interviews, not to think how different
questions should be asked (to reduce interviewer effect)
22.9.2010 11M.Simpanen
Interviewing techniques
the manner in which interviewer presents questions and answer alternatives, introductions, descriptions
the manner interviewer presents the question modules and introductions and
clarifies questions if the respondent does not understand the question (neutral probing)
Interview technique is NOT monotonous repeating the questionsInterview techniques allow more time to use to the interactionUsing interviewing techniques the interviewer ”teaches” the respondent
to understand our way to do interviewing work (the right answer atmosphere)
To keep quality high (to ask questions similarily leads to less interviewer effect and more comparable and reliable data)
22.9.2010 12M.Simpanen
Examples: Question order
• How satisfied are you to your life as general?
• How satisfied are you to your spouse?
• Correlation: +0,37
• How satisfied are you to your spouse?
• How satisfied are you to your life as general?
• Correlation: +0,62
22.9.2010 13M.Simpanen
Interviewer: Always read the following text as well:
Did you return in August or July from a domestic trip with overnight stay, with a destination of at least 30 km from your home.
Included are all trips with overnight stay in Finland; also trips to own free-time residence, visits to friends or relatives, and business and professional trips?
Amount of "No"-answers (no trips) in year 2001 and 2002
Travel Survey
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec All year
Month
0
10
20
30
40
50
60
70
80
90
100%
'02 '01
"Always read the following text as well"
22.9.2010 14M.Simpanen
Interviewing style
ArticulationPaceUse of pauses and emphasisSpeech mannersNeutrality of languageVoice in general
22.9.2010 15M.Simpanen
Interview interaction
We put effort to interaction:The basic precondition is to get the respondent to agree to the
interview Interviewers behave to convince the respondent (persuasion)
To create right kind of atmosphere by applying the interviewing technique and to use the correct interviewing style
In SF we measure by recordings the interviewers´ interviewing techniques, interviewing style and interaction
22.9.2010 16M.Simpanen
Objectives, follow-up
To maintain quality high needs the objectives, the goal settingsefficiency and qualityto the projects and to the interviewersshort-term and long-term goals, even mid-term goals on projectschedule, response rate, quality
Quality control by monitoring (recordings, supervisor monitoring)Objectives to commit interviewer
to the projectto the surveyto the organisation
Cost control system
22.9.2010 17M.Simpanen
Feedback system
Appraisals or personal discussionsInterviewer feedback surveysRespondent feedback surveys
22.9.2010 18M.Simpanen
Appraisals, discussions
Basic/project training is not enough to keep quality of interviews highNeeds also discussions with interviewers time to time
how the employee has succeededhow to improve his/her performanceco-operation in the work society
motivationprofessional development
and again to commit him-/herself to the survey and to interview work
In SF each employee has annual development and performance discussions (appraisals) with their immediate or own supervisor
personal performance -> flexible personal pay component
22.9.2010 19M.Simpanen
Interviewer feedback surveys
Systematic, comparable feedback surveys from each projectsto develope questionnaires and survey processessystematic feedback
Feedback surveys show also that researchers/organisation respects interviewers’ attitudes and their work
Also systematic feedback on the interviewers’ working conditions, work methods and job orientation
to motivateto commit the interviewers part of the survey processes
22.9.2010 20M.Simpanen
Respondent feedback surveys
Systematic, comparable feedback surveys from each projectsto develope questionnaires and survey processessystematic feedbackresponse burdenvalidation
22.9.2010 21M.Simpanen
At the end...
The main point is NOT to say every organisation should have its own interviewing organisation but ...
survey is the coherent process - including several subprojects every party of the whole process has to have similar values of the
process every party has to have similar understanding of the process every party needs to commit themselves to the survey process - not
only an individual project Processes should be systematic and systematically evaluated To maintain quality of data collection demands continuing follow-up All these mentioned before have a great influence to the quality Research, analyse and report also paradata of interviews Poor data collection cannot be corrected with mathematical tricks!
