chatbots and digital assistant analysis

David Wright

David.Wright@kantar.com

Digital Assistants –

a brand’s best friend?

What are Digital Assistants?

Personal Assistants Customer Service Assistants

Chatbots

Messenger “Chatbot” Global Reach Messengers are where that conversation is at.

WeChat, China - 700 million users

Facebook Messenger – 1 billion users

KIK – 300 million users

Line, Japan – 215 million users

“A brand’s best friend?” - The experiments

Usability Test

How do people interact with a DA?

Behavioural Analysis

SIRI + Facebook Messenger interaction

patterns

Facebook Brand

Page Analysis What do direct brand to consumer

conversations look like now?

Messenger ‘Chatbot’

Prototype and Analysis

1 2

3 4

1

Location based questions

“If I ask for a baked chicken recipe, or how to remove a stain, why not link

straight to a 'how to' video on YouTube from a reputable brand?”

“I need to adjust the way I talk, so Google understands me".

Weather and Time Questions Information Searches

“I like it when Siri replies to me (via voice), not when it gives me a list of tiny

links"

“I like it when there is a photo in the result”

Usability Test

Usability test summary

People set the conversational bar

very high

The knowledge base is controlled by Apple or Google.

The brand is not included in the conversation.

These assistants are not a brand’s best friend.

For brands though…

“This was the first time I used SIRI in two

years. Wow it’s got really good. When I first

used it didn’t understand me. I didn’t use it

again."

1

Behavioural data analysis Word frequency for mobile searches that contain “How”

Are device makers now

ceding the customer

relationship to search

engines?

TV

FIX

MO

NE

YW

OR

KR

EM

OV

EC

AR

AN

DR

OID

PLA

YP

HO

NE

FR

EE

HA

IRIP

HO

NE

CO

OK

RE

SE

TA

PP

XB

OX

CA

RD

WA

TE

RC

LE

AN

UN

LO

CK

GA

LA

XY

CA

LO

RIE

SB

AB

YD

OG

HO

ME

INS

TA

LL

ON

LIN

EB

EC

OM

EG

OO

GL

EF

AC

EB

OO

KO

PE

NW

OR

TH

BO

XG

RO

WM

US

ICO

ILC

ON

NE

CT

MO

VIE

SP

AY

WIN

DO

WS

“How do I connect YouTube from

Phone to TV?”

“How do I edit my contacts on my

Samsung Galaxy?”

2

Behavioural data analysis What to people do on SIRI?

70K SIRI SEARCHES 4K IMAGES TAGGED

2

Movies 18%

Music 1%

Nature 6%

Other 9%

Sports Faces 48%

Sports Logos 18% Wolfram 8%

Maps 8%

Bing Search 63%

Behavioural data analysis “Do Messenger Patterns indicate how people might want to interact?”

52%

18% 30-60 seconds

5% 60-90 seconds

30-60 seconds

0-30 seconds

Individual

messenger

sessions are brief

and succinct.

2

Behavioural test summary

Brands need to be part of the digital

conversation in their domain space

Are succinct as more than half of individual messenger

interactions are less than 30 seconds.

Are often a Q & A hunt for information.

Images can bring this information to life.

Digital conversations:

2

Facebook brand page analysis “What do brand consumer digital interactions look like now?”

u = user (the person who first posted to the page and started a conversation chain)

b = the brand

0 = other users have decided to comment within the conversation chain.

3

Facebook Brand Page Analysis

26% of customer posts

were not answered

22% were responded to

and the conversation

was complete

3

A large and complex

long tail of

interactions.

For the 13K page

posts there were 942

different chain types.

For 10% of multi-chain

conversations the

brand was not present.

Facebook brand page analysis 3

Facebook brand page analysis

Tesco had the

smallest % of

unanswered posts

and the highest

chain variation.

3

0

10

20

30

40

50

60

0

100

200

300

400

500

600

700

800

900

1000

Boots Lidle Aldi UK Tesco Marks andSpencer

% Unanswered

Chain Types

Facebook page test summary

Public Facebook posts are

likely the tip of the iceberg.

Open conversation ‘chains’ can be complex.

Facebook brand page conversations are public.

In KLM’s case 7 times more messages across the private channel.

Brands are likely to prefer direct/private digital communication.

For brands though…

3

Meet “Kit” the chatbot

Structured Survey

Open “AI” chat

4

Tested with 150

Kantar Employees

Messenger “Chatbot” prototype and analysis Finally we can participate! “

4

The good

A simple “structured” survey with elements including

“buttons” and “quick replies” worked!

Say “Hi” to Kit!

4

The bad - aka attempts at open conversations

0

10000

20000

30000

40000

50000

60000

70000

80000

1 3 5 7 9 1113151719212325272931333537394143454749515355575961636567

Total Duration of Chat (seconds)

Majority of interactions 1-15 Minutes.

Conversation ID

Seco

nd

s

Conversations that last 16 hours

Conversations that

span 1-2 hours

Although individual interactions are

short and sweet a user pause and pick

up the conversation at any time.

It was easy to get lost in open chat and

forget the purpose of the interaction.

52% of posts to the bot in “open

conversation mode” were one word

only.

4

The bad - aka too smarty pants too quickly

”What do you do to treat it?”

“I have a broken ankle”.

“I am wearing a boot.”

“Has it been a tough few days?”

“Yes.”

“Ok thanks. Please remember to say Hi again!”

“Do you not feel sorry for me?””

4

4 AI tools – initially subtle and in the background

A brand’s best friend? Key conclusions

Chatbots combined with vast Messenger networks offer an unprecedented

opportunity for brands to connect with customers at scale.

However chatbot creation is not straight forward.

Customers expect brief and accurate interactions.

However direct brand to consumer conversations can be complicated and

nuanced.

Early adopters and influencers will try to stump an AI enabled chatbot which may

create a PR risk.

DAs/chatbots have the potential to become a brand’s best friend.

In the short term, due to the complexities of an AI managing open conversation, it is likely the brand

bot experiences will start simply and rely heavily on buttons and other structured UI elements.