chatbots and digital assistant analysis
TRANSCRIPT
What are Digital Assistants?
Personal Assistants Customer Service Assistants
Chatbots
Messenger “Chatbot” Global Reach Messengers are where that conversation is at.
WeChat, China - 700 million users
Facebook Messenger – 1 billion users
KIK – 300 million users
Line, Japan – 215 million users
“A brand’s best friend?” - The experiments
Usability Test
How do people interact with a DA?
Behavioural Analysis
SIRI + Facebook Messenger interaction
patterns
Facebook Brand
Page Analysis What do direct brand to consumer
conversations look like now?
Messenger ‘Chatbot’
Prototype and Analysis
1 2
3 4
1
Location based questions
“If I ask for a baked chicken recipe, or how to remove a stain, why not link
straight to a 'how to' video on YouTube from a reputable brand?”
“I need to adjust the way I talk, so Google understands me".
Weather and Time Questions Information Searches
“I like it when Siri replies to me (via voice), not when it gives me a list of tiny
links"
“I like it when there is a photo in the result”
Usability Test
Usability test summary
People set the conversational bar
very high
The knowledge base is controlled by Apple or Google.
The brand is not included in the conversation.
These assistants are not a brand’s best friend.
For brands though…
“This was the first time I used SIRI in two
years. Wow it’s got really good. When I first
used it didn’t understand me. I didn’t use it
again."
1
Behavioural data analysis Word frequency for mobile searches that contain “How”
Are device makers now
ceding the customer
relationship to search
engines?
TV
FIX
MO
NE
YW
OR
KR
EM
OV
EC
AR
AN
DR
OID
PLA
YP
HO
NE
FR
EE
HA
IRIP
HO
NE
CO
OK
RE
SE
TA
PP
XB
OX
CA
RD
WA
TE
RC
LE
AN
UN
LO
CK
GA
LA
XY
CA
LO
RIE
SB
AB
YD
OG
HO
ME
INS
TA
LL
ON
LIN
EB
EC
OM
EG
OO
GL
EF
AC
EB
OO
KO
PE
NW
OR
TH
BO
XG
RO
WM
US
ICO
ILC
ON
NE
CT
MO
VIE
SP
AY
WIN
DO
WS
“How do I connect YouTube from
Phone to TV?”
“How do I edit my contacts on my
Samsung Galaxy?”
2
Behavioural data analysis What to people do on SIRI?
70K SIRI SEARCHES 4K IMAGES TAGGED
2
Movies 18%
Music 1%
Nature 6%
Other 9%
Sports Faces 48%
Sports Logos 18% Wolfram 8%
Maps 8%
Bing Search 63%
Behavioural data analysis “Do Messenger Patterns indicate how people might want to interact?”
52%
18% 30-60 seconds
5% 60-90 seconds
30-60 seconds
0-30 seconds
Individual
messenger
sessions are brief
and succinct.
2
Behavioural test summary
Brands need to be part of the digital
conversation in their domain space
Are succinct as more than half of individual messenger
interactions are less than 30 seconds.
Are often a Q & A hunt for information.
Images can bring this information to life.
Digital conversations:
2
Facebook brand page analysis “What do brand consumer digital interactions look like now?”
u = user (the person who first posted to the page and started a conversation chain)
b = the brand
0 = other users have decided to comment within the conversation chain.
3
Facebook Brand Page Analysis
26% of customer posts
were not answered
22% were responded to
and the conversation
was complete
3
A large and complex
long tail of
interactions.
For the 13K page
posts there were 942
different chain types.
For 10% of multi-chain
conversations the
brand was not present.
Facebook brand page analysis 3
Facebook brand page analysis
Tesco had the
smallest % of
unanswered posts
and the highest
chain variation.
3
0
10
20
30
40
50
60
0
100
200
300
400
500
600
700
800
900
1000
Boots Lidle Aldi UK Tesco Marks andSpencer
% Unanswered
Chain Types
Facebook page test summary
Public Facebook posts are
likely the tip of the iceberg.
Open conversation ‘chains’ can be complex.
Facebook brand page conversations are public.
In KLM’s case 7 times more messages across the private channel.
Brands are likely to prefer direct/private digital communication.
For brands though…
3
Meet “Kit” the chatbot
Structured Survey
Open “AI” chat
4
Tested with 150
Kantar Employees
Messenger “Chatbot” prototype and analysis Finally we can participate! “
4
The good
A simple “structured” survey with elements including
“buttons” and “quick replies” worked!
Say “Hi” to Kit!
4
The bad - aka attempts at open conversations
0
10000
20000
30000
40000
50000
60000
70000
80000
1 3 5 7 9 1113151719212325272931333537394143454749515355575961636567
Total Duration of Chat (seconds)
Majority of interactions 1-15 Minutes.
Conversation ID
Seco
nd
s
Conversations that last 16 hours
Conversations that
span 1-2 hours
Although individual interactions are
short and sweet a user pause and pick
up the conversation at any time.
It was easy to get lost in open chat and
forget the purpose of the interaction.
52% of posts to the bot in “open
conversation mode” were one word
only.
4
The bad - aka too smarty pants too quickly
”What do you do to treat it?”
“I have a broken ankle”.
“I am wearing a boot.”
“Has it been a tough few days?”
“Yes.”
“Ok thanks. Please remember to say Hi again!”
“Do you not feel sorry for me?””
4
4 AI tools – initially subtle and in the background
A brand’s best friend? Key conclusions
Chatbots combined with vast Messenger networks offer an unprecedented
opportunity for brands to connect with customers at scale.
However chatbot creation is not straight forward.
Customers expect brief and accurate interactions.
However direct brand to consumer conversations can be complicated and
nuanced.
Early adopters and influencers will try to stump an AI enabled chatbot which may
create a PR risk.
DAs/chatbots have the potential to become a brand’s best friend.
In the short term, due to the complexities of an AI managing open conversation, it is likely the brand
bot experiences will start simply and rely heavily on buttons and other structured UI elements.