OUT OF THE BLUE

iAM1/4 .ME

The Digital Revolution is all around us. The impact of digitization on established businesses, institutions, and start-ups is profound, and is opening up new dimensions to deliver products and services and to interact with customers.

However, perhaps even deeper than the industry changes are the underlying changes that digital is having on our personal and professional lives. In a positive feedback cycle, users foster the development of innovations and, at the same time, change behaviors and preferences upon using them. The expectation of technology to make our lives easier and faster is growing. As digital continues changing what we do, the question arises: is digital changing who we are?

Out of the Blue digs into these changes: new behaviors, new preferences, and new social mores that are emerging with such a strength and transformative power that we could be witnessing the dawn of a new concept of the self: the i.AM.

Knowing how individuals evolve in the digital revolution will be critical for business and institution. Out of the Blue’s purpose is to contribute to this knowledge.

Under the overarching theme iAM, we have developed four different concepts.

.me: How individualism and personalization take on new dimensions with digital.

.now: How digital is changing our perception of time and our use of resources

.here: How digital is broadening the concepts of place and presence

.mind: How digital is affecting our minds and our attention

The material is crowdsourced and linked for further reference and reading. We are only scratching the surface and invite you to join us on this journey. Both .here and .mind are scheduled for a January release.

– Oliver Wyman’s Communications, Media and Technlogy practice

2

EDITORIAL

3

“ Are our devices turning us into a new kind of human?”

– Amber Case, cyborg anthropologist, TED Radio

4

iAM

Digital is an integral part of our lives. Technology is blurring the lines between the digital, physical and biological spheres while reshaping our perception of space and time and even the concept of our identity or persona.

The expectation of technology to make our lives easier and faster is growing. As digital continues changing what we do, the question arises: is digital changing who we are?

5

iAM INDEX

.MEHow individualism and personalization take on new dimensions with digital.

.NOWHow digital is changing our perception of time and our use of resources

.HEREHow digital is broadening the concepts of place and presence

.MINDHow digital is affecting our minds and our attention

Ogilvy & Mather China Center For Psychological Research, Shenyang Phone Wall Campaign. Used under permission

6

.ME How individualism and personalization take on new dimensions with digital

Digital is reinforcing an overall trend towards individualism. Over the past decade, the smartphone has become our digital umbilical cord, the physical support and connection with our digital selves. It has made computing truly personal.

Now, technological advancements in artificial intelligence, machine learning, voice-based interactions, and emotional tracking technologies promise to capture the “digital me”: to deliver customized content and personalized interaction in ways that can scale to reach millions.

7

“ Our goal is to build a personal Google for each and every user. Not a single Google that we all can use, but an individualized Google for everyone.”

– Sundar Pichai, CEO Google, introducing Google Assistant

8

In the UK, adult smartphone users spent nearly 2 hours online each day using a smartphone compared to just over an hour on laptops and PCs

In the US, smartphones make up the largest share (37%) of connected devices in the home. M-commerce contributes about 1 in every 6 dollars spent via digital commerce

In China, 92.5% of Internet users use their smartphones to pay rent, buy things, and even monitor school home work around the clock. On average web users spent 3.8 hours a day on Internet

Within the EU, there are 75 mobile broadband SIM cards per 100 people; up from 34 in 2012

In July 2016, Apple announced that it has sold its one billionth iPhone

9

More people get online using mobile than any other technology:

Between 2010 and 2014, the number of unique mobile subscribers increased by 120% worldwide.

At the end of 2015, 3.24 billion people (44% of the global population) were connected to the Internet.

Our smartphone is the first device that is 100% personal, and it is becoming our primary computer:

We are spending more time on our smartphones – even at home and in the office – and we increasingly use our mobiles manage our personal and professional lives, including purchasing products and services, consulting information and making payments.

ME AND MY MOBILEHOW THE MOBILE MADE COMPUTING PERSONAL

In 2015, mobile messaging apps outgrew usage of social media.

In Q216, more than 90% of Facebook’s 1.71 billion users were on mobile devices. 84% of the company’s $6.2 billion advertising revenue came from mobile.

Sources: CNNIC, Ofcom, ComScore, European Commission, Business Insider

10

IT IS ALL ABOUT MEHOW INTERNET IS EVOLVING INTO THE “INTERNET OF ME”

Ubiquitous information, connectivity, social networks, and an overall trend towards individualism have given people of all ages the freedom to construct their identities more freely than ever, and to share what they do and who they are with others online.

Our willingness and appetite to share who we are and what we like is changing the nature and traffic of the Internet. The “Internet of we,” in which we went online primarily to find information on certain topics, has evolved into the “Internet of me,” where individuals are primarily the topic of interest and the hub of data flows. We are sharing our personal experiences and information like never before, and we spend several hours a day uploading, tweeting, and curating digital information about ourselves.

Our online behaviour is amassing a wealth personal information that offers companies and organizations endless possibilities to track, analyse, and predict individual behaviour. By using data to unveil our inner identities companies are offering highly customized and personalized services.

“ What we've learned over time is: it's not who they are in a superficial sense – like gender, age, even geography. It's not even what they tell you. It's what they do.”

– Todd Yellin, VP, Product Innovation, Netflix

11

DISCOVERING THE NEW .ME (1/2)THE REVIVAL OF ARTIFICIAL INTELLIGENCE TO CUT THROUGH THE CLUTTER

Among digital abundancy we are drowning in too many apps and too much data. We are always connected, toggling between different devices. Our photo collections keep growing, music and media is available at a click, and messaging apps buzz for our attention.

In this setting, contextual awareness and predictive analysis – underpinned by artificial intelligence and machine learning fed by the wealth of personal information stored in the cloud – point toward more intuitive, personal, and relevant “information discoveries” as they are needed.

Although it might be difficult to envision today, the central role of the smartphone in our lives might fade away. According to Bob Donell of Techanalsyis, we are entering a new era of computing where the role of hardware and devices becomes secondary, and where voice-based interactions, driven by deep-learning algorithms in the cloud will enter the main stage.

Looking to the future, the next big step will be for the very concept of the “device” to fade away. Over time, the computer itself – whatever its form factor – will be an intelligent assistant helping you through your day. We will move from mobile first to an AI first world.”

– Sundar Pichai, CEO Google

THE DAWN OF A POST DEVICE ERA?

Global PC shipments reached their historical high of 95.4 million units in 2011. Since then, global demand has gradually declined.

Smartphone sales recorded their highest growth in 2010. Double digit growth followed in the period between 2011-2015. Gartner estimates the growth for 2016 to slow down to 7%.

Sources: Fastcompany, Gartner, Statista.com

12

DISCOVERING THE NEW .ME (2/2)THE REVIVAL OF ARTIFICIAL INTELLIGENCE FOR PERSONALIZATION AT MASS SCALE

Advancements in artificial intelligence and machine learning are also opening up the doors for personalization at an unprecedented scale, potentially reaching millions of customers.

Use cases for artificial intelligence range from personalized recommendations for discovery, customer segmentation, programmatic advertising, marketing forecasting, and prediction to helping customers navigate vast amounts of information, customer service, acquisition, and retention.

Over the past year, tech companies have raced to open up their artificial intelligence software as open source and integrated machine learning capabilities within their cloud service offerings. Companies of all sizes now have an increasing set of tools to use artificial intelligence to create personal and smart experiences at mass scale.

TECH COMPANIES AI TOOLS and OFFERINGS

Google: In November 2015, Google released open source machine learning software package TensorFlow

IBM: In November 2015, IBM transferred its SystemML machine-learning software to Apache Software Foundation, which supports several major open-source projects

Facebook: In January 2015, Facebook made several of its AI tools available on the Torch open source library

AirBnB: In June 2015, AirBnB released Aerosolve, its open-source machine-learning tool on which Airbnb’s pricing algorithm relies, on the Github code-sharing platform

IBM: offers a wide range of cognitive computing services, including Watson

Microsoft: In April 2016, Microsoft added “Cognitive Services”, a series of 22 APIs to extend their machine learning service within its cloud service offering Azure

Amazon: In April 2015, Amazon launched Amazon Machine Learning within Amazon Web Services.

13

BENEFICIAL INTELLIGENCE

Personal finance Digit startup enables personalized micro savings

Digit promises its users to “save money, without thinking about it.” The company connects to a user’s checking account, analyzes the user’s spending habits and income history, and uses an algorithm to automatically set aside small amounts of money – on average between $2 and $18 – at times when the user is least likely to miss it.

The service is free, but Digit keeps the interests of the savings generated by its users. In March 2016, Digit said it was saving its users $14 million a month.

Language learning app Duolingo personalizes learning experiences

Language learning app Duolingo uses a task-based, gamified, multiple-choice format to get users up to speed on learning a language.

The founders used machine learning to figure out from users themselves how they learned a second language best.

The learning app now has over 120 million users, and is using machine learning to adjust language learning for different users and cultures, including automatic adjustments of the difficulty level, depending on personal performance and user interface tweaks for positive enforcement.

M-commerce app Wish personalizes product listings

The Wish shopping app connects shoppers in the United States and Europe directly with primarily Chinese wholesalers and retailers that sell at steep discounts. The company uses machine learning algorithms and natural language processing to fine-tune and curate what products are shown to which customer based on browsing behavior, showing only a handful of highly targeted products.

Wish expects to reach $2 billion in sales in 2016, and has 100 million registered users.

AI powered lunchbot Forkable offers tailored menu plans for team lunches

Workers enter their dietary requirements and preferences and Forkable’s “lunchbot AI” creates what the company calls “the equivalent of a Spotify playlist” for each person’s lunch.

Users can replace, customize and cancel orders using the app’s web interface that are requested automatically and scheduled to arrive at a specified time.

The system is designed to manage differing employee allowances and budgets.

US retailer Macy’s uses AI shopping assistant backed by IBM’s Watson

Macy’s On Call is a mobile tool that will help shoppers find their way round 10 US stores piloting the system. The system uses Watson’s Natural Language Classifier API and Satisfi, an intelligent engagement platform, to understand and respond to input.

The service will be customized by location to tell customers where the shoe department is in a particular store, or whether that branch has a certain size in stock.

Major music streaming services personalizes music discovery

Spotify, Apple Music and Google Music Play use a combination of taste prediction algorithms and human curation combined with data analysis to help users navigate the vast ocean of music streaming catalogues and differentiate in a commoditized music space. Read more

HOW COMPANIES USE MACHINE LEARNING TO PERSONALIZE OFFERINGS

14

HUMAN CURATION AND MACHINE LEARNING FOR MUSIC DISCOVERYHow curated playlists help users navigate the vast ocean of streamed music and keep them coming back for more

On-demand music streaming is becoming increasingly popular, and although paying subscriber growth has been slow, users are listening to more music than ever, largely thanks to personalization features such as playlists that use a combination of machine learning and human curation to cater for a wide array of music interests.

Around 100 music curators are working at Apple, Google, and Spotify, to assemble, name, and update tens of thousands ready-made playlists for every conceivable genre, activity, or mood. Spotify has around 4,500 playlists, Apple Music has 14,000 and Google Play Music has tens of thousands.

The music curators have access to a suite of proprietary tools that streamline the maintenance process of playlists, such as performance trackers of a playlist (age range, gender, region, time of day, subscriber tier, etc.), and breakdown analysis of individual songs in the playlist (number of plays, average play length, number of skips, number of saves/likes, etc.).

On-demand audio music streaming overtook video music streaming for the first time in the US in the first six months of 2016 according to Nielsen.

Spotify says 50% of its more than 100 million users globally are listening to its human-curated playlists (not counting those in “Discover Weekly”).

According to an industry estimate, 1 out of every 5 plays across all streaming services today happens inside of a playlist.

We curate songs purely based on what we think the listeners are going to like and what we see in the data.”

– Doug Ford, Spotify

CASE STUDY

Source: Buzzfeed, Techinsider

15

By using big data analysis and deep learning, Spotify has been able to draw a detailed map of every user’s taste that is feeding automated personalized weekly playlists:

• “Discover Weekly”, is a weekly personalized “mixtaple of fresh music” of 30 songs based on collaborative filtering algorithms. The service was launched in July 2015 and now has40 million unique users with 5 billion tracks streamed.

• “Fresh Finds” is a service aimed at “understanding the brand new,” launched in the form of six generic genre playlists, updated every Wednesday. To feed the updates, Spotify crawls new trending artists on music blogs and review sites on the web. The identified artists are then filtered through a real-time analysis of Spotify listening behavior of around 50,000 early adopters who tend to discover new artists before they break. The identified songs are then sorted and ordered by Spotify music curators into six different genre buckets

• In August 2016, Spotify launched “Release Radar”, a weekly personalized playlist of the artists that people follow mixed with new discoveries based on the listening habits

The enabling technology partly comes from The Echo Nest, a music intelligence platform and MIT Media Lab startup from 2005 acquired by Spotify in 2014, that has synthesized billion of data points for more than 37 million songs to power smart music applications.

HUMAN CURATION AND MACHINE LEARNING FOR MUSIC DISCOVERYHow Spotify leverages user generated content, taste predictive algorithms, and human curation to “connect the right music with the right ears,” for millions of users on a weekly basis

CASE STUDY

Sources: The Verge, Fivethirtyeight, Quartz, Brian Whitman blog (Principal Scientist, Spotify), Fast Company, Spotify

16

The bet on recommendations seems to pay off in an increasingly commoditized music world where players have access to the same music catalogues: According to a recent report by Cowen & Co, the monthly churn of Spotify’s 35 million paid subscribers is 2.2%, compared to Apple Music that has monthly churn rate of 6.4% for its 15 million paid subscribers*.

Both “Discover Weekly” and “Fresh Finds” are helping up-and-coming artists: Some bands have landed on the Fresh Finds playlist after being played only 10 times on the platform, and more than 8,000 artists on Spotify had half of their total listeners find them on Discover Weekly.

Quartz envisions a future in which Spotify uses its giant database and clever algorithms to find up-and-coming bands for a music label of its own, similar to how Netflix produces its own content based on Big Data insights into its users’ tastes and preferences.

HUMAN CURATION AND MACHINE LEARNING FOR MUSIC DISCOVERYHow personalization and customer engagement has resulted in lower churn, and how Spotify might use customer insights and big data to launch its own music label

The end result is a playlist of songs designed to feel like it was handcrafted for you by a good friend.

I see personalization as a huge part of not just attracting users, but keeping them.”

– Matthew Ogle, product owner, Discover Weekly

Sources: Cowen & Co, Quartz

Note: Updated subscriber figures as of September 2016: Spotify 40 million paying customers, Apple Music 17 million paying customers

CASE STUDY

17

LET’S JUST TALK (1/2)THE REVIVAL OF ARTIFICIAL INTELLIGENCE FOR “CONVERSATION AS A PLATFORM”

Artificial intelligence is also spurring new user interfaces based on conversation.

The growing popularity of messaging apps combined with advancements in AI has seen the revival of the chatbot for “contextual commerce” or “conversational commerce” that seek to sell services and products based on the conversations users have within an messaging app. Based on the success of Chinese WeChat, that was among the first to invite business to use chat based conversations with its users.

Facebook, Microsoft, and Google have been introducing chatbots recently. At Microsoft’s worldwide partner conference in July 2016, Microsoft’s CEO Sataya Nedella said that chatbots will “fundamentally revolutionize how computing is experienced by everybody.” According to Nadella, the change will be gradual and bots will start off by augmenting apps, but in time, human language will be taught to all computers and become the new interface.

The complexity is too much. We need to tame it. We need to be able to make it much more natural for people to get things done, vs. this thing about let me remember the 20 apps I need to get anything done.”

– Sataya Nedella, CEO Microsoft

Virtual agents and chatbots will be the top consumer applications of artificial intelligence over the next five years, according to a consensus poll by TechEmergence, a marketing research firm for AI and machine learning.

Source: TechEmergence

18

LET’S JUST TALK (2/2) THE REVIVAL OF ARTIFICIAL INTELLIGENCE FOR VOICE INTERACTION

If recent advancements in speech recognition using deep learning keep their current pace, chances are high that we will once again rely mainly on our voice when interacting with our smartphones (and other devices).

Voice-operated virtual assistants such as Apple’s Siri, Microsoft’s Cortana, and Google Now are readily available on most smartphones. Newer devices, like Amazon’s Alexa, offer a simple way to look up information, switch on the lights, turn on the music, and build shopping lists with your voice.

Voice-based services can be can be accessed by the simplest of devices, with little more than an audio input, an audio output, and a wireless connection since all the computing power takes place in the cloud.

MACHINES ARE ALMOST AS GOOD AS PEOPLE AT SPEECH RECOGNITION

Over the past year, tech companies have reported major advancements in word error rates for speech recognition using deep learning and neural networks:

In February 2016, Chinese Baidu reported an error rate of 3.7% for short phrases using its Deep Speech 2.

In April 2016, the IBM Watson team reported a word error rate of 6.9%.

In June 2015, Google reported an 8% error rate, down from 15% in 2014, using neural networks that are now more than 30 layers deep.

In comparison, the transcription of human error rate is around 4%.

20 percent of queries on Google’s mobile app and on Android devices are voice searches.”

– Sundar Pichai, CEO, Google Google I/O keynote, May 2016

Sources: MIT Technology Review, Silicon Valley AI Lab, IBM Watson Developer Community, Venturebeat

19

BOTS & VIRTUAL ASSISTANTS

Microsoft

In 2014, Microsoft launched its intelligent personal assistant Cortana as a key ingredient for “makeover” of the future operating systems for Windows Phone and Windows. In 2015, Cortana was also launched for Xbox, iOS and Android.

Microsoft has also developed Xiaoice, a chatbot created by a team of software developers and psychological experts to create a balance of IQ (intelligence) and EQ (emotional intelligence). Xiaoice, that has the personality of a 17-year-old girl, has more than 40 million registered users, and approximately 25% of them (10 million people) have said “I love you” to her.

In March 2016, Microsoft was forced to withdraw its infamous chatbot Tay after Twitter users taught it to be racist.

Facebook

In April 2016, Facebook launched chatbots for businesses within its Messenger app, to allow businesses to deliver automated customer support, e-commerce guidance, content, and interactive experiences through chatbots. The platform is primarily powered by Facebook’s Applied Machine Learning team and Wit.ai, a startup Facebook acquired in 2015.

In August 2015, Facebook launched a beta of a personal chatbot assistant named M in San Francisco, powered by a combination of AI and humans, so called “M trainers”, that complement AI to perform trickier judgment calls and tasks that AI can’t cope with.

M can suggest and book reservations, find birthday gifts, and retrieve information. Read more

Amazon

In June 2015, Amazon officially launched its Alexa Voice Service, that powers its wireless home speaker Echo, the Alexa skills set, and the Alexa Fund.

The Alexa platform is open to developers to allows others to build Alexa skills and integrate Alexa into their own products. In June 2016, Amazon said that its internal Alexa team had more than 1,000 employees and that tens of thousands of developers are building skills for Alexa.

The Alexa Fund has invested in 16 startups, with a focus on smart home and wearable products to date. Over the next year, the Alexa Fund will be expanding investments into startups that focus on robotics, developer tools, healthcare, accessibility, and more. Read more

Google

Google started investing in voice recognition, language understanding and machine translation long before most of its rivals.

At its May 2016 developer conference, Google launched Google Assistant, a service that is integrated in its new conversational user interface products: Allo, its new independent chat bot app, as well as Google Home, its smart portable speaker that is powered by voice assistance technology (launched in September 2016).

The Google Assistant is seen as an important step in Google’s ambition to extend its search functions across all kinds of devices and interfaces.

HOW COMPANIES ARE USING TWO-WAY TEXT-BASED OR VERBAL CONVERSATIONS TO RESPOND TO CONSUMER NEEDS IN A MATTER OF SECONDS

20

FACEBOOK CHATBOTSLeveraging technology to enable conversation a with 1 billion audience

In September 2016, there were more than 30.000 chatbots on the Facebook chatbots platform, with more than 300 million people using its audio and video-calling features every month. Examples of companies that have used chatbots include:

Disney: Disney launched a bot for release of Zootopia, allowing fans to chat with the movie’s lead character, Officer Judy Hopps, extending the story to the real world by enlisting fans to help solve multiple ZPD (Zootopia Police Department) cases. Millions of messages were sent, with an average chat duration time of several minutes.

NBA: The NBA’s bot on Messenger was focused on providing fans instant access to highlights for the NBA finals and draft 2016. The bot had over 350,000 interactions.

Shopify: Shopify offers the platform components needed for its merchants to send post-sale receipts and shipping updates in Messenger. More than 16,000 Shopify merchants offer this service to their customers, with nearly 200,000 order status updates sent every single day.

MONTHLY ACTIVE USERS MESSAGING APPS

Facebook Messenger: 1 billion ( July 2016)

Whatsapp (owned by Facebook): 1 billion (Feb 2016)

WeChat: 762 million (Q116)

LINE: 218 million (April 2016)

Kik: 175 million

Snapchat: 150 million ( June 2016)

What we are seeing is a clear uptake of activity. The majority of interactions with the bot are placing an order. The vast majority are new customers for our brand. We are reaching the early-adopter audience that the general advertising doesn’t hit. We are reaching the younger demographics.”

– Chris McCann, President, 1-800-Flowers.com

CASE STUDY

Source: Facebook, messengerblog.com, The Verge

21

MYKAI PERSONAL BANKING BOT Using chatbots to enable life-style banking

Personal finance chatbot maker Kasisto, a spinoff of Siri-maker SRI International, aims to make financial services more ubiquitous and enable lifestyle banking by embedding financial decision-making in consumers’ everyday life.

In June 2016, the company launched two chatbots for intelligent banking conversations on Facebook Messenger, Slack, and via SMS in the US: MyKAI for personal finances and KAI Banking to help banks interact with their customers on messaging platforms.

By using MyKAI, people are able to check their account balances, categorize and track their spending, pay with Venmo and in the future monitor their investment portfolio. MyKAI has been trained to answer more than 1,000 questions about financial terminology and the user’s personal finance.

The KAI chatbots for banks has been used by Digibank in India and will be implemented by Royal Bank of Canada.

DIGIBANK IN INDIA, AN ALL DIGITAL EXPERIENCE POWERED BY THE KAI VIRTUAL ASSISTANT CHATBOT

In April 2016, Singaporean retail bank DBS launched a mobile-only bank, offering a savings account with a 7% interest and an unmetered debit card.

The bank relies 100% on smartphones, it has no branches, and there is no paper-work involved for opening up an account. Instead, an account can be activated from any of the 500-plus Cafe Coffee Day outlets across India, and customer authentication is done by a biometrics-enabled ID.

The customer service is provided by a 24x7 AI-driven Virtual Assistant from Kaisto, trained to answer over 10,000 questions, from customer on-boarding to user education to customer service. It also includes an intelligent, intuitive budget optimizer that helps customers be smarter about their money.

CASE STUDY

Image: Mykai press kit

22

DONOTPAY: “THE WORLDS FIRST ROBOTIC LAWYER”How a Stanford student used artificial intelligence to provide personalized simple legal advice and help people understand their rights

In autumn 2015, 19-year-old Stanford student Joshua Browder launched DoNotPay, a chatbot that asks the user a series of questions designed to work out whether a parking ticket can be appealed within 30 seconds. Examples of the questions asked are whether there were clear parking restriction signs or if the driver was travelling to the hospital urgently. After determining that an appeal is viable, it then walks the user through the steps of appeal. 

The service is available in London and New York, and has been used by over 250,000 people. Of the $4 million worth of tickets overturned, about 150,000 were in London. It is currently only available through the DoNotPay website, but Browder is in talks with Facebook to incorporate it in the Messenger app. 

Browder plans to expand the artificial intelligence lawyer, which can also work out compensation for delayed flights, to help vulnerable groups navigate complicated legal systems, including people who are HIV-positive and refugees in foreign countries. The latter will use IBM Watson to translate Arabic and English.

CASE STUDY

Source: The Guardian, Venture Beat

23

AMAZON ECHOEcho, the first device with voice as main interface

Amazon’s Echo is a 24-cm tall cylindrical wireless device that doubles as smart home appliance and portable Bluetooth speaker. It is the first computer for which the main interface is voice. The only physical controls are an on-off switch, a button to mute the microphone, and a volume knob.

Echo connects to Wi-Fi and the Alexa Voice Service to play music, provide information, news and weather updates, order Amazon items, control compatible smart home devices, and activate “Alexa skills” developed by third parties by using voice commands.

Amazon is now working on endowing Alexa with conversational skills. In September 2016, the company announced a $2.5 million prize to spur the development of conversational artificial intelligence. The company will sponsor up to 10 university student teams with $100,000 each, and will award $500,000 to the best team. Additionally, a prize of $1 million will be awarded to the winning team’s university if their socialbot achieves the grand challenge of conversing coherently and engagingly with humans on popular topics for 20 minutes.

The Echo is currently available in the US and the UK. Customer response has been overwhelmingly positive with over 40,500 reviews and an average customer rating of 4.4 stars.

The most popular items being ordered through Alexa are coffee, batteries, pet food, and paper towels.

Alexa may be Amazon’s most loved invention yet – literally – with over 250,000 marriage proposals from customers and counting.”

– Jeff Bezos, CEO, Amazon

CASE STUDY

Source: MIT Technology Review, Press release: The Alexa Price, Press release: Amazon Q316 results

Image: Amazon press kit

Conversational artificial intelligence – The Alexa price

24

As our expectations continue to rise, and we increasingly expecting technology (and companies) to be empathetic, to understand how we feel, to simplify our lives, and to help us to become the people we want to be. Solving emotional intelligence is a natural path to solve true machine learning and make machines understand us better.

Better cameras, computer vision algorithms, emotion-sensing facial analysis software, and semantic analysis are helping machines get better at reading the important clues our body language and speech reveal about our mood and intent, including the movement of our eyes, our gestures, the way we talk, and even how we move our heads.

Companies have a lot to gain by connecting with customers’ emotions: An eight year research project at Harvard identified 300 universal motivating emotions, and validated their impact on consumer behavior through extensive surveys and data collection. Twenty-five important emotional motivators, including desires to “stand out from the crowd,” “have confidence in the future,” and “enjoy a sense of well-being,” significantly affected customer value across all the categories studies. Fully connected customers were found to spend twice as much annually as highly satisfied customers.

As technology is increasingly applied to situations where it must interact with everyone, it is all the more vital that it does so in a way that is courteous and respectful of people’s feelings.”

– Dr. Rosalind Picard, Affective Computing Group, MIT Media Lab

According to study by Forrester, emotion and how an experience makes a customer feel, is the number one factor in customer loyalty - ahead of effectiveness and ease - across 94% of the industries studied.

THE VALUE OF CAPTURING THEEMOTIONAL.ME (1/2)THE DAWN OF ARTIFICIAL EMOTIONAL INTELLIGENCE

Sources: Harvard Business Review, Forrester

Image: Frank Behrens, Flickr Creative Commons. Original photo has been cropped

25

THE VALUE OF CAPTURING THEEMOTIONAL.ME (2/2)AI FOR SMARTER PROFESSIONAL AND PERSONAL DECISIONS

Behavioral economists and philosophers argue that artificial intelligence can help us overcome our human flaws, such as overconfidence, limited attention, and cognitive biases, which inevitably cause errors in our judgment and decision making.

According to Daniel Kahneman, Nobel-prize winning behavioral economist and psychologist, much of human error is not even attributable to a systematic cause or biases, but to “noise.” Noise is random, unpredictable, and impossible to explain. By using algorithms, organizations can temper human judgment with “disciplined thinking.”

According to philosopher Alain De Botton, artificial intelligence can give us a helping hand in managing our personal lives too, since “our emotional frailties far outweigh our incapacities in raw mathematics or data management.” The six areas that artificial emotional intelligence are set to revolutionize include self-knowledge, education, news media, art, shopping, and relationships.

Algorithms are noise-free. People are not.

When you tell team leaders that there is 50% variability [in expert judgment] when they expected 5% or 10%, then they’re willing to take an algorithm.“

– Daniel Kahneman, Nobel-prize winning behavioral economist and phycologist

Source: Knowledge Wharton, Wired

Image: Riccardo Cuppini, Flickr Creative Commons

We make extremely poor decisions about how we should manage relationships. We have little idea what job to focus on and when to quit.

We don’t know what to spend our money on. We get holidays wrong, have no clue how to repair friendships or handle tricky employees, and fumble as to how to reconcile with our parents.”

– Alain De Botton, author and philosopher

26

ARTIFICIAL EMOTIONAL INTELLIGENCE

iDAvatars offer conversational avatars for customer service

Sophie is a highly advanced, emotionally intelligent Avatar who provides patient care and data feedback for healthcare clients by talking, listening, comforting, soothing, and relating to patients.

Kate is a Virtual Assistant who provides 5-star customer service for financial, insurance, education, healthcare, retail, government, and other clients. Kate guides a customer through even the applications, instructions and forms and answers questions and charting progress along the way. Read more

Chatbot Karim delivers physiological support to Syrian refugees

Silicon Valley startup X2AI has created an artificially intelligent chatbot called Karim that can have personalised text message conversations in Arabic to help people with their emotional problems.

As the user interacts with Karim, the system uses natural language processing to analyse the person’s emotional state and returns appropriate comments, questions and recommendations. Read more

SRI International is working on a new generation of virtual assistants that respond to users’ emotions

SRI International, the research lab that made Apple’s Siri, has developed a technology called SenSay Analytics, that is designed to identify emotional state based on a variety of cues, including typing patterns, speech tone, facial expressions, and body movements.

For example, the virtual assistant might be able to tell from a patient’s pattern of speech if he or she were becoming confused, then slow down. Read more

Emotional robot Pepper greets guests and keep people company

Softbank Japan has developed an emotional robot called Pepper together with French company Aldebaran that uses facial recognition to pick up on sadness or hostility, and voice recognition to hear concern. In Japan, over 7,000 Pepper robots have been used to greet guests, answer questions and play with kids.

2016 will see the launch of Pepper in Europe and US for business applications. Read more

MIT Personal Robotics Group helps children learn Spanish

MIT’s Personal Robotics Group led has created Tega, a cuddly social robot that uses an Android device to process movement, perception, and thinking and can respond appropriately to individual children’s behaviors as it teaches Spanish to preschoolers. Over time, it learned how the cues influenced a student’s engagement, happiness, and learning successes. As the sessions continued, it ceased to simply mirror the child’s mood and began to personalize its responses in a way that would optimize each student’s experience and achievement. Read more

Biometric bracelet help users “to become the best version of themselves”

Zenta is a wearable biometric bracelet that uses advanced sensing technology, and machine learning algorithms to build a holistic view of the wearer’s emotional and physical wellbeing.

Zenta constructs a personalized digital profile and provides the user with actionable insights on how to maximize their personal well-being. Read more

HOW COMPANIES ARE USING SPEECH RECOGNITION, COMPUTER VISION, EMOTION-SENSING FACIAL ANALYSIS SOFTWARE, AND SENSORS TO UNDERSTAND AND RESPOND TO USER’S EMOTIONS

27

EMOTIONAL ROBOT PEPPER SET TO CONQUER THE WORLD Pepper, the emotional robot

Softbank Japan has developed an emotional robot called Pepper together with French company Aldebaran that uses facial recognition to pick up on sadness or hostility, and voice recognition to hear concern.

The physical shape of Pepper was deliberately design to incentivize engagement. Its height, shape, and gesticulating arms are all designed to show empathy.

In Japan, over 7,000 have been sold to families and businesses. Pepper robots have been used Softbank and Nescafe stores to welcome guests, answer questions and collect data. SoftBank has also closed a partnership with Honda to use the artificial-intelligence system from Pepper to build a car that can read a driver’s emotions and “talk” to them to keep them company.

In May 2016, Pepper was launched in Europe at a price of €19,900, including professional services. A partnership has already been closed with several European companies. Usage cases include Pepper as a patient companion, retail assistant, and receptionist.

Pepper will be available in the US at the end of 2016. Softbank and Google released an Android SDK to help Pepper understand Americans better.

Sources: Pepper Partnerships in Europe, Wired

Image: Softbank Robotics

CASE STUDY

28

EMOTION TRACKING SMART BRACELETS Wearables that focus on emotional well-being over physical health to help you “become the best version of yourself”

A number of companies, with varying success, are launching wearable smart bracelets that make recommendations to alleviate stress and unhappiness, sending their wearers off for a walk or a meditation when emotions run high.

Emotion-tracking smart bracelet SenceBand by biotech company Planexta Sence tracks 64 different emotional states from the wearer’s wrist. It uses a clinical-grade ECG (EKG)-tracking technology to track advanced heart rate analysis, including heart rate variance (HRV). The SenceHub uses algorithms to turn the raw data into actionable analytics, and sends of notifications about the emotional states of the user of the people in his/her network. ScenceBand is available for preorder on Indiegogo and Kickstarter for $129.

Announced in late 2015, the Feel Wristband by Sentio Solutions uses four integrated sensors that measure the body’s skin temperature, blood volume pressure, and galvanic skin response. Based on the data it gathers, the app graphs a visualization of emotion levels throughout the day and offers personalized suggestions for stress reduction and self-improvement. Feel can be preorder at a price of $149.

Meanwhile Vinaya, the London-based manufacturer of smart jewelry and other wearables, that built the AI and machine learning for its devices in-house, went into administration in December 2016. The company’s crowdfunded biometric bracelet Zenta, that raised over $270,000 on Indiegogo and was scheduled to ship in April 2017 will not ship, and the company is reportedly planning a restructuring to focus on B2B instead of B2C.

EMOTIONAL TRACKING TO IMPROVE THE PERFORMANCE OF FINANCIAL TRADERS

Andrew Lo, director of the Laboratory for Financial Engineering at the Massachusetts Institute of Technology, has been conducting simulations of stressful situations facing financial traders. The simulations used wrist watch sensors to measure the pulse and perspiration of traders to warn them to step away from their desks to avoid high risk behavior when emotions were running high. The information could also be monitored by risk managers to spot problems brewing on a specific desk, such as unauthorized trading, before too much damage is done.

Likewise, Humanyze, a startup founded by MIT graduates has created a sensor-laden badge that transmits data on speech, activity, and stress patterns.

CASE STUDY

29

PRIVACY PROTECTION FOR .ME A personal privacy assistant to manage your data

Technology may also provide us a helping hand in coming to terms with a problem it caused in the first place – how to manage our privacy and personal data in the digital age.

A team at Carnegie Mellon University is creating a personalized privacy assistant app, which uses natural language processing, machine learning, crowdsourcing, and a private interface design to learn the privacy preferences of their users over time, with the ultimate aim of making it simpler for users to manage their smartphone app permissions.

By using machine learning, the assistant analyzes a user’s response to a small number of questions, focusing on the particular apps they have on their phones. Based on the results, the agents are capable of semi-automatically configuring many settings and making privacy decisions on their user’s behalf.

The privacy assistants alert users about practices the users may not feel comfortable with and even occasionally nudge users to reconsider the implications of some of their privacy decisions. Research at Carnegie Mellon has shown that when people learn exactly how many times apps share information, they rapidly act to limit further sharing.

Another potential use case of the assistant would be to recognize when an app is leveraging user data without asking permission to do so.

Sources: CIO.com, Carnegie Mellon University

FOOD FOR THOUGHT

30

DIGITAL MOURNING, MEMORIES AND MINDCLONESHow do we cope with digital death and digital legacies?

Technology is reshaping our relation to death. First, we die tangibly and cease to exist in the physical world. But what happens to our digital selves and to our digital legacies?

Viktor Mayer-Schönberger of the Oxford Internet Institute says that we might be producing more memories than we can cope with: “Forgetting is built into the human brain. So for thousands of years we’ve developed ways to preserve special memories.” Today, though, it is quicker and easier to save every bit of our vast digital trails than it is to sort and discard what we don’t want.

In her book, Virtually Human: The Promise and the Peril of Digital Immortality, futurist Martine Rothblatt argues that our “mindfiles” (i.e. our uploaded thoughts, memories, preferences, beliefs) create a virtual existence that software engineers can “mindware” and use to create humanlike consciousness in computer software, or a cyberconsciousness. In the next decade or two, these efforts will result in the first digital copies of our identities, which will be our “mindclones”.

Sources: The Atlantic, New Scientist, LACMA Unframed, The Verge

By design, our feeds are largely ephemeral, and we treat them that way, parcelling out seemingly insignificant moments – until, all of a sudden, we stop.

That collection of moments, once thought to be of infinite supply, will now outlive your physical form, freezing in time the flow of your whole existence.”

– Ghost in the Machine, Jenna Wortham, New York Times

VIRTUAL GRIEF AND VIRTUAL DEATH

As our digital legacies outlive our physical selves, technology is offering new ways to mourn and share memories of someone that has passed away.

In May 2015, Eugenia Kuyda, co-founder and CEO of a Russian artificial intelligence startup Luka, developed a chatbot that lets anyone talk to her dearly departed friend Roman Mazurenko, a tech entrepreneur who died in a car accident in November 2015. The Roman chatbot is a dialogue model that uses smaller datasets of past conversations with Roman (photos, texts, articles) on top of a neural net. Users can text or chat with @Roman like they would have done when he was alive–and the Roman AI replies like Roman would have done in real life.

The Hereafter Institute evaluates a person’s digital afterlife using new technologies. It offers multiple digital preservation options, including 3D body scanning, wearable memorialization, and the embedding of personal data into everyday objects.

FOOD FOR THOUGHT

31

NARCISSISM, WEALTH DISTRIBUTION, AND DIGITAL DIVIDEHow do we cope with the flipsides of individualism?

Warning voices point towards the rise of a modern narcissism, with the selfie as the ultimate expression of self-promotion, and the opportunities lost in using social media to drive social connection.

Despite advancements in advancements in technology and productivity, the gap in income and wealth distribution around the world is increasing:

According to Oxfam, the gap between the global richest and the global poorest has widened. In 2015, 62 people had the same wealth as the poorest half of humanity, compared 388 people in 2011.

According Allianz, the distribution of wealth is becoming increasingly unequal in developed countries with Gini coefficients well above 70, particularly in the US (80.52) Sweden (79.9), UK (75.72), Austria (73.59) and Germany (73.34).

According to a report published by the World Economic Forum, the digital divide between developed countries and many emerging economies is increasing because of the high cost of digital access.

Far from giving us a stronger sense of our own identities, the 1 million odd selfies taken every day across the world (the average millennial is expected to take 25,700 selfies in his or her lifetime) will only propagate insecurities and provoke precisely the kind of neurotic and self-questioning behavior that characterizes adolescence.”

– Elsa Goodhardt, author of the book I Selfie, Therefore I Am

Sources: Telegraph, Oxfam, Allianz, World Economic Forum

Image: 'Selfi #6: Flawless' by Dean Christensen, used under permission

FOOD FOR THOUGHT

Narcissism has overtaken altruism as the driving force behind social connection. Popular culture fuels this behavior, reinforcing the idea that being social is actually about self-promotion.”

– Peter Kim, Chief Digital Officer, Cheil Worldwide

32

DIVE DEEPER

Affective Computing, Rosalind Picard, Affective Computing Group, MIT Media Lab

AI’s Language Problem, MIT Technology Review, August 2016

The New Science of Customer Emotions, Harvard Business Review, November 2015

20+ Emotion Recognition APIs That Will Leave You Impressed, and Concerned, Nordic APIS, December 2015

Baidu’s Deep-Learning System Rivals People at Speech Recognition, MIT Technology Review, December 2015

Why Do So Many Digital Assistants Have Feminine Names? The Atlantic, March 2016

The Real Story of How Amazon Built the Echo, Bloomberg, April 2016

Mind vs. Machine, Brian Christian, The Atlantic, November 2011

Digital legacy: The Fate of Your Online Soul, New Scientist, April 2011

For Sympathetic Ear, More Chinese Turn to Smartphone Program, The New York Times, July 2015

Art, Death, and Memorial: A Participatory Event with Gabriel Barcia-Colombo, LACMA, the LA County Museum of Art, August 2016

Speak Memory – When Her Best Friend Died, She Rebuilt him Using Artificial Intelligence, The Verge, October 2016

TEXT

The Post Device Era, Bob O’Donnell

Emotional Intelligence, The School of Life

Deep Learning in Practice: Speech Recognition and Beyond, Andrew Ng Baidu

Recommender Systems, Harvard University

Me, My Selfie & I: Narcissism or Empowerment? Institute of Ideas

How Computers Are Learning to Be Creative, TED talk, Blaise Agüera y Arcas, principal scientist at Google

HereafterLife, Gabriel Barcia-Colombo & Sarah Rothberg, Hereafter Institute

Are Our Devices Turning Us Into A New Kind Of Human? TED Radio, Amber Case

VIDEO & AUDIO

Out of the Blue is a periodic publication from the Oliver Wyman CMT (Communications, Media and Technology) Research and Knowledge Management Team aimed at highlighting trends within the communication, media and technology space.

The Out of the Blue Team

Communications, Media and Technology Knowledge Management

Mia Burner, Senior Knowledge Manager Borja Xercavins, Senior Consultant

Editorial

Lorenzo Miláns del Bosch, Partner, Madrid Iria Aparicio, Manager, Knowledge Management

Marketing

Kate Wildman

Design

Paloma Sanchez Michael Tveskov Melissa Ureksoy

For more information, please visitwww.oliverwyman.com

Copyright© Oliver Wyman