traditional versus adaptive techniques
DESCRIPTION
comparison study between traditional online survey vs our Adaptive SurveyTRANSCRIPT
CloudMR Adaptive Technology versus
Traditional Survey Monkey Survey 17 September 2014
Background & Objectives
• CloudMR says that Adaptive technology is better than traditional surveys
• Objective: Contrast Adaptive & traditional – Replacement for rating scales – Quantifies open-ended comments – Reduces effort – Tolerates low sample sizes
2
Executive Summary • Adaptive survey got 40% more responses • Adaptive results usually match the
traditional rating scales results • Adaptive collects & ranks new ideas not
even conceived in the traditional design • Adaptive quantifies open-ended
comments effectively even at very low sample sizes
• The demographics and overall ratings are the same for both survey types
3
Responses
4
eMail: We are demonstra,ng the difference between Survey Monkey style surveys and CloudMR style surveys. Please par,cipate in this short survey about your last customer support experience. [Link] Thanks. Gene
CloudMR Intro: Please tell us about your last customer support experience that you remember. There are 4 ques,ons -‐-‐ all on this page.
Survey Monkey Intro: Please tell us about your last customer support experience that you remember. There are about 20 clicks on 4 pages.
123
88
123
86
CloudMR
Survey Monkey
Starts
Starts
Completes
Completes
Samples
30%
26%
21%
12%
1%
2%
7%
29%
36%
15%
8%
5%
1%
6%
Totally sa,sfied
Very sa,sfied
Somewhat sa,sfied
Neither sa,sfied nor dissa,sfied
Somewhat dissa,sfied
Very dissa,sfied
Totally dissa,sfied
CloudMR n=123
Survey Monkey n=86
5 Overall, how sa,sfied are you with the last customer support experience that you remember? Please indicate your gender. Which one of the following includes your age?
• Responses came from the same panel • As you would expect, the overall sa,sfac,on and
demographics are sta,s,cally iden,cal
CloudMR Survey MonkeyMale 70% 58%Female 30% 42%Age 18-‐24 2% 2%25 to 34 26% 22%35 to 44 28% 26%45 to 54 23% 24%55 to 64 16% 22%65 or older 4% 2%Refused 0% 1%n= 123 86
Results Comparison
6
CloudMR Survey MonkeyTake responsibility 1 (New) Not askedStatus updates 2 11TTR 3 10Opening case 4 3Train representatives 5 (New) Not askedEscalation process 6 1TTFR 7 6Website 8 2Add CHAT support 9 (New) Not askedTCO 10 4Implementing solution Not mentioned 9Rep solution Not mentioned 8Time to diagnose Not mentioned 7Time to reach rep Not mentioned 5
RankItem
Verba,m Ideas
CloudMR Results
7 What new features should we add to CA Clarity PPM to increase the chance that you would recommend it to colleagues who have a need for project and por[olio management
Timely status reports
Take responsibility
Faster TTR Make easier to open case
Train Reps
Faster TTFR Improve website
Improve escala,on process
Add CHAT support
Reduce TCO 5%
7%
9%
11%
13%
15%
17%
19%
22% 27% 32% 37%
Priority
Agreement n =123
• Quan,fied categories to improve
• Drill down to actual comments
Keep me informed about the status
Customer Service / Support with responsibility.
Faster @me to resolve my issue -‐ 24 hours
Survey Monkey Results
8
31% 31%
25% 30% 30% 29% 28% 28% 26% 24% 27% 25% 25% 23%
18%
43% 38%
41% 36% 35% 33% 34% 33% 35% 36% 29% 31% 29%
24% 27%
Rep communica,on Rep diagnosing Rep Courtesy
Rep knowledge Rep status updates
TTR Implemen,ng solu,on
Rep solu,on Time to diagnose
TTFR Time to reach rep
TCO Opening case
Website Escala,on process
Totally Sa,sfied
Very Sa,sfied
Thinking about the customer support process, please rate each of the following.
Recap
• Expect 40% increase in total par,cipa,on for Adap,ve technology Response
• Same top improvement areas 6 of 10 ,mes • Adap,ve produced addi,onal insights not an,cipated by the research design
• Adap,ve is 2 dimensional, tradi,onal is 1
Adap,ve vs. tradi,onal Results
• Can replace a series of ra,ng scales • Quan,fies open-‐ended answers • Works well with a rela,vely low sample size
Adap,ve Technology
9
Adap:ve: Provides faster & easier design, superior response, similar results from 1/10th the number of ques,ons, extra
insights, works with small sample
APPENDIX
Methodology
• A panel of respondents was asked to participate in two surveys
• One traditional rating-scale survey was conducted on Survey Monkey – 20 questions were asked: 17 rating scales, 2
demographic & 1 open-ended • One Adaptive survey was conducted on
CloudMR using Adaptive technology – 4 questions were asked: 1 rating scale, 2
demographic & 1 adaptive 12
Adaptive Technology
Some top ideas and some
new ideas
Ini,al ideas
Idea Pool
Select Ten
• Ques,on about any topic? Select the ideas you agree with and drag them into priority order • Idea • Idea • Idea • Idea • Idea, etc. • Add your own ideas
Adap,ve Ques,on
Get answers and new ideas New Ideas
Invite Any Group to Par,cipate
Details
plus
respondent ideas
An advanced market research methodology that replaces tradi,onal open-‐ended ques,ons and ra,ng scales
Adaptive Surveys®
• Deliver actionable improvement ideas • Uncover drivers you didn’t even ask about • Prioritize touch points, themes, sub-themes
and comments • Float innovative ideas to the top – even if
only one customer mentions it • Set up in minutes and deliver results in hours
or days • Are short and conversational, resulting in
high response rates & low drop-off rates
14