RecentFindingsonUsingRichMediainOnlineSurveys
BernieMalinoffJeffreyHenning,PRC
PanningforRespondents
EngagedRespondentsareGolden
LosingRespondentsAlongtheWay
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Engaged
Completed
ParRcipated
Invited
Source: Vovici
ButisaGhostTownourFuture?
PaperSurveysInspiredtheWeb
MaybeTechnologyistheAnswer?
WebSurveysasEntertainment
WhatCanWeTaketotheBank?
ELEMENT54R&D
DriversofSurveyCooperaRon
RespondentCooperaRonSurveyInvite
QuesRonnaireLength/IncenRve
TopicInterest
Panel/ClientReputaRon
VisualAppeal
Typicalrespondentexperience
ResponsePa^erns/Flatlining
"Engagement&
Enjoyment"
OnscreenLayout
RespondentExpectaRons
Divergence/VarietyofNewQuesRonTypes
Sources: Spyglass Solutions/2x4 Research & Consulting
‘BlackBox’PhaseofOnlineDesign
FinalQuesRonnaire
ResearcherGivesto
Programming
ProgrammingDeliversa“Link”
Researcher/ClientReview
ProjectKick‐offBriefingPlanningSampleDraj(s)...
InterviewingCoding
TabulaRonAnalysisReport
‘RespondentEngagement’ConvergingIssues
DataConsistency
RespondentUsability
MajorR&DIniRaRves DataConsistency RespondentUsability
Balanced5CellDesign:2000TotalInterviews
RadioBu^on
Anchor@‘1’
Anchor@‘5’
Anchor@’10’
Experimental
QuesRonnaireCoverage
StudyHighlights
+8%overstatedpast7dayconsumpRon‐slidervs.radiobu^on
‐8%understatedissueimportance‐slidervs.radiobu^on
36%variancehowojenbrushteeth‐slidervs.radiobu^on
eg‐NetPromoterScore(NPS)
GivenDifferences,WhatistheBenchmark?
RadioBu^on
Default,EasierStandards
Sliders
Scales/AnchorsMa^er
DragnDrop
CauRon:ContentsHot
SurveyCompleRonTime&RespondentEnjoyment
RichMediavs.RadioBu^onQuesRons
EyesDon’tLie‐ResearchProtocol
EyeTracking
Recording
Interviewing
EyeTracking
"1.0vs.2.0"UsingEyeTrackingforSurveyUsability
... to better understand the destination, by examining the journey.
EyeTracking‐'AreasofInterest'
Inferencevs.InstrucRon
Inferencevs.InstrucRon42%didnotreadthequesRon
33%didnotreadthe"tradiRonal"
quesRon
Don’tInterruptMeWhenI’mThinking
ErrorPrevenRon&Recovery
Thisrespondentneeded5minutestocompletethisquesRonandencountered3errormessages.
ThequesRonis‐howdowetreatthedatawhichemergesfromsomeonewhotriedhisng"next",andthenaddedorchangeddatatoescape?
ErrorPrevenRon&Recovery
RespondentinterpretedtheerrormessagelocaRonasaguidetowheretoputtheanswer.RecommendaRon‐havesomewaytohighlighttheenRrerow,avoidinganyvisualcuesorbiases.
MinimumStandards‐eg:HorizontalLayout
Horizontallayoutensuresconsistentviewing,regardlessofscreensize
EqualRaRngsfor‘Engagement’Metrics
1.08.20
8.86
7.02
2.08.25
8.85
6.56
SurveyEnjoyment
EaseofUse
DesiretoRepeat
SexyQuesRons,DangerousResults?ConsideraRons
1. ThereisnoBe^er,Faster,Cheaper‐youcanonlypick2.2. Wemustreturntotheusabilityprincipleswhichguided
papersurveydesign,notCATIdesign,tomaximizeconfidenceinourdata,andbusinessdecisions.
3. TherearewaystomakeallquesRons/surveysmoreappealing,intuiRveandfuture‐relevant.
4. 'ResponsibleInnovaRon'iskey.5. Asanindustry,wemustdeviseandreinforcebest
pracRcesforonlineresearchinterfaces,likewehaveforsomanyelementsoftheresearchprocess.
TipsforMigraRngtoRichMedia• DiscreRonisthebe^erpartofvalor• SplitballottestrichmediaquesRonsagainsttradiRonalquesRonstoidenRfydifferences
• Visualchoicesintroducebias,especiallyformoreabstractchoices
TechnologyisNottheOnlyAnswer
TipsforEngagement• Keepthesurveyshort• MakethequesRonsinteresRng
• Minimizetheuseofmatrixes
• Userichmediainaconsideredmanner
CreaRngEngagingSurveysisHard
WeWelcomeYourQuesRonsBernieMalinoffPresident
element54
h^p://twi^er.com/BernieMalinoff
Bernie.malinoff@element‐54.com
(514)833‐2268
VoviciBlog
Vovici ’s Jeffrey Henning blogs about survey research, EFM, voice of the customer and online feedback.
blog.vovici.com
JeffreyHenningFounder&VP,Strategy
Vovici
h^p://twi^er.com/jhenning
(800)787‐8755
NPR Listens: How Panel Management helps NPR ‘Dial In’ on Unmet Needs
Jeanne Bliss:
EarnyourCustomersRave
Lessons from a loyalty Leader: A conversa>on with Navy Federal Credit Union
ContactInformaRonBernieMalinoff
• bernie.malinoff@element‐54.com
• 514‐833‐2268• @berniemalinoff
• h^p://www.element‐54.com
JeffreyHenning,PRC
• 781‐848‐8100x550• @jhenning
• h^p://www.vovici.com
