modeling and simulation of survey collection using paradata presented by: kristen couture...
TRANSCRIPT
Modeling and Simulation of Survey Collection Using
ParadataPresented by: Kristen CoutureCo-authored by: Yves Bélanger
Elisabeth Neusy
2
Outline
Motivation for Simulating Survey Collection Details of Simulation Modeling using Paradata Preliminary Results Conclusions Future Work
3
Motivation Ultimate goal: make CATI survey collection more efficient
Recent initiatives in the field• Experimentation with call attempts and calling priorities
• Takes time, lack of control, costly, results not always easy to interpret
Need for a controlled environment, where impact of each experiment can be tested prior to collection
4
Microsimulation What is microsimulation?
• A modeling technique that operates at the level of individual units, such as persons, households, vehicles, etc.
For us: microsimulation = a "virtual collection" system
Recreates CATI collection environment with Simulation Software (SAS Simulation Studio)
Allows manipulation of parameters in simulated environment
5
Microsimulation What are the elements of our microsimulation?
• Cases
• Queues
• Interviewers
• Rules of the Call Scheduler (flows and priorities)
• Output Call Transaction File
6
Overview of MicrosimulationParadata
Model Call Outcomes Model Call Duration
Model Parameters
Collection Parameters
Simulation Model
SAS Simulation Studio
7
Modeling using Paradata Use existing survey data (Blaise Transaction History)
Call Outcome• Multinomial logistic regression
Call Duration• Create histograms and fit distributions for each of the outcomes
Output Model parameters• Estimated parameters from logistic regression model• Fitted distribution and parameters• Input into simulation model
8
Modeling using Paradata: Call Outcome Multinomial Logistic Regression Model
Model probability of outcomes (sum of probabilities = 1) k+1 outcomes xi = explanatory variables from paradata pj = probability of outcome j = parameters from logistic regression model
kjxp
p n
iiij
k
j ,...,1for,log11
ij
9
Modeling Call Outcome: An Example Paradata: Existing RDD survey
5 outcomes:
Unresolved, Out of Scope, Refusal, Other Contact, Respondent
7 explanatory variables entered into the modelTime of Call: Afternoon, Evening, WeekendResidential Status: Residential phone numberCall history: Unresolved, Refusal, Contact
Estimated parameters from model are entered into simulation
10
Time of Call
Call History
Modeling Call Outcome: An Example Calculate probability of each outcome
pj values
11
Microsimulation
Collection Parameters
Simulation Model
Paradata
Model Call Outcomes Model Call Duration
Model Parameters
12
Preliminary Results: Two examples Investigate how collection parameters impact response rates
Two Examples:• Example 1 : Different distributions of interviewers throughout the day• Example 2: Different distributions of interviewers throughout the day
combined with different time slices
Purpose:• Demonstrate how users can manipulate collection parameters to test
specific collection scenarios• Verify that simulation results reflect collection
13
Example 1
Change allocation of interviewers throughout the time periods
3 Time Periods each 4 hours in length
30 interviewers per day for 30 days
What happens to response rates?
Time period# of
Interviewers
Morning
(9h-13h)4
Afternoon
(13h-17h)4
Evening
(17h-21h)22
Fixed Total 30
One Possible Scenario
14
Example 1
Impact on Response Rate when Changing Concentration of Interviewers in Evening
40%
45%
50%
55%
60%
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30
# of Interviewers in Evening (Total = 30)
Res
po
nse
Rat
e
15
Example 2 Same setup as Example 1
Add time slices: control maximum number of attempts made at different time periods throughout the day
What happens to response rates?
Time period# of Interviewers
Max # of Attempts
Morning 4 2
Afternoon 4 2
Evening 22 16
Fixed Total 30 20
One Possible Scenario
16
Example 2
Morning/Afternoon Evening
Morning/Afternoon 47% 37%
Evening 42% 52%
Time period with majority of attempts permittedTime period with the
majority of interviewers
Response Rates
17
Conclusions Create simple simulation model using paradata
that produces results that reflect collection
Able to test different collection parameters to see impact on response rates without spending a lot of money or time
Approach adaptable to all types of CATI surveys
18
Future Work Improve logistic model by adding more
parameters Add more complicated collection procedures to
the model such as interviewer characteristics Simulate collection with multiple surveys at a
time to see impact Run simulation for a survey to predict outcome
and compare with actual results from field
19
For more information, Pour plus d’information,please contact: veuillez contacter :
Kristen [email protected]
Yves Bé[email protected]
Elisabeth [email protected]