© 2014 IBM Corporation

A Big Data Telco Solution

Laura Wynter Director, IBM Research Singapore Collaboratory IBM Master Inventor Research Scientist, Watson Research Center, New York

WKWSCI

SYMPOSIUM

2014 Big Data, Big Ideas for Smarter Communities

The Global Reach of IBM Research

IBM Research Labs

IBM Research – Openings in 2011

IBM Research – Openings in 2012

China

Watson Almaden

Austin

Tokyo

Zurich

India

Dublin

Australia

Brazil

Africa Next Gen Public Sector Water & transportation Human Capacity Development

Natural Resources Disaster management Healthcare/Life Sciences

Natural Resources Smarter Devices Human Systems/Events

Analytics & Intelligence Systems &Software Industry research

Internet of Things Big Data / Analytics Enterprise Cloud Services Energy, Commerce, Traffic

Big Data Analytics HW & SW Quality Cloud Mobile

Haifa

Smarter Cities Analytics

Services Big Data Analytics Front Office Digitization

Semiconductors Systems Software Services Analytics

Semiconductors Processors

Analytics Storage Nanotech Healthcare

Nanotech Security Business

Analytics Systems

Industry Solution Lab

Singapore •Analytics

IBM Research Singapore Collaboratory: who we are

3

Statistics Transportation science

Data mining & management

Computer science

Optimization

Computational science

A team of research scientists and research software engineers with expertise in mathematical & computer sciences, a branch of our global IBM Research presence

IBM Confidential

Telco Data Monetization • Telcos have lot of data about their customers from daily operations –

especially location and movement data.

• Our objective is to build an asset for Telcos to leverage these data about their customers to enable emerging new market opportunities.

• Key to such data monetization is the ability to connect different data pieces to better understand customers, their preferences, life style, intent etc.

Two Scenarios: Aggregate and Individual

• Aggregate Anonymized Analytics: Sensing City-scale people movement from Telco data and leveraging for Transit Optimization

• Enriched Consumer Profile: Customer Analytics with Mobility Profiles from Telco Data

Let’s review the potential areas of Business Benefit of Big Data for Vodafone

GPS

External Data

Customer Service

Representatives ... could offer

personalized price promotions to different customer segments in real-time

Business Development ... could find new mechanisms to monetize network traffic and partner with upstream content providers

Network Operations ... could identify network bottlenecks in real-time for faster resolution

Executive Leaders ... could get real-time reports and analysis based on data inside as well as outside the enterprise (web, social media etc.)

Business Analysts ... Could analyze social

media buzz for the new

services/offerings to gauge

initial success and any

course correction needed

Finance ... could analyze all Call Detail Records (CDRs) to identify and reduce revenue leakage due to unbilled / underbilled CDRs Marketing

... could analyze subscriber usage pattern in real-time and combine that with the profile for delivering promotional or retention offers

What if …

A data sharing platform should capture and structure location, time and content about the consumer from multiple industries to drive profitable consumer

actions

Structured Repeatable

Linear

Monthly sales reports Profitability analysis

Customer surveys

Other Industries

Other Data

Industry Reports

Retail

Social Media Data

Customer • Segment

• Social Network • Demographics • Sex, Age Group, etc

• Tenure • Rate plan

• Credit Rating, ARPU Group

Device •Class

•Manufacturer •Model

•OS •Media Capability •Keyboard Type

Transactions • Voice, SMS, MMS

• Data & Web Sessions • Click Streams

• Purchases • Downloads • Signaling,

Authentication • Probe/DPI

Network • Availability

• Throughput/Speed • Latency • Location • Facilities

Interface • Discovery • Navigation

• Recommendations

Product/Service • Subscriptions

• Rate Plans • Media Type

• Category/Classification • Price

Starts, Stops Success Rates

Errors

Throughput Setup Time

Connection Time Usage

Recency Frequency Monetary Latency

Telco Data Cross Industry Data

Enriched Consumer Profiles for Enabling Telco Data Monetization

• We develop enriched consumer profiles by deriving insights about consumer preferences, life style, and intent from location, mobility and call data joined with use case appropriate data sources.

• Enriched consumer profiles are utilized to enable new services and effective campaign through targeted segmentation.

Two Scenarios: Aggregate and Individual

• Aggregate Anonymized Analytics: Sensing City-scale people movement from Telco data and leveraging for Transit Optimization

• Enriched Consumer Profile: Customer Analytics with Mobility Profiles from Telco Data

Sensing City Scale People Movement from Telco Data

Cities Demonstrated: Istanbul (Turkey), Dubuque (USA) for Transit Optimization and a series of subsequent client pipeline

Challenge Cities have very little real understanding of where citizens, goods and

transportation move during the day. Without this information it is difficult to accurately plan and manage the usage of roads and infrastructure.

Solution Using a variety of real time data from “smart phones”, GPS devices, terminals, traffic cameras, public transportation schedules and transit data, develop models of zonal density, flow of goods and origin / destination pairs. From these models, drive processes to manage this flow against a specific objective.

Benefits Evaluates the efficacy of existing transit system and transportation infrastructure; provides the structure for design incentive strategies to win new riders – information, incentives, services; optimize fleet operations in situations where demand outpaces supply; manage revenue through better zoning and permits. comprehensive solution that will address the management of congestion, fleet management, people attending events, and multimodal transit

10

Sensing People Movement from Telco Data

Example Challenges Objective: Derive people movement model from tower level information

(communication between cell phone and tower)

Key Challenges • CDR data is typically sparse

– Uncertainty both in space and time domain

– Location/movement from sparse and often incorrect (tower information) information

• Tower oscillation is very common in cellular network

• Typically only short term (e.g. one week) data is available due to various privacy regulations

Figure: Example for CDR and GPS. Left: CDR with tower oscillation; Right: GPS points

13

Meaningful Location Detection and O/D Estimation

• Meaningful locations are the locations where people spend a significant amount of time, e.g. home, work, mall.

• Duration of stay (dos) is used to measure how meaningful each cluster is.

– i.e. Given a threshold (e.g. 30 min), if the duration of stay (dos) in a cluster is more than the threshold, then the location of the cluster locates is a meaningful location.

• Home and work can be identified by selecting the locations with the largest accumulated dos in the night time and day time of week days.

• After meaningful locations detection, users’ traces are described in a set of meaningful locations.

• Trips and O/D pair can be segmented from users’ trace on these meaningful locations.

• For example:

Identifying Meaningful Locations

Where People Live Where People Work

Istanbul Movement Analysis

- 4.7 million phones w. 3B+ events/week

- Accurate detection of home, work & meaningful locations

Traffic Monitoring Uses basic analytics building blocks already seen to display time based traffic flow levels mapped to city road system. A snapshot at 8:30am:

15

Time of Day Density Maps

Prominent Trip Attractors and Producers

Major attractor of trips in Dubuque

Commuter Pain Index

Feeder Bus Route Optimization for M4 Metro Line on Anatolian side of Istanbul

Feeder bus routes based on demand to 4 metro stations on Kadikoy-Kartal metro line

Optimal Bus Stop Location Design

• Stops are added by considering the greatest potential demand for transit and accessibility at origin and destination

• Some stops are added to far places in which demand to the area already served by existing stops is potentially large

20

21

Clean sheet Optimization of Bus Routes based on Demand Models

Clean sheet optimization to

minimize opex, unmet demand

and travel time

Constraints include fleet size,

max transfers, duration, etc.

Optimal routes can • reduce OPEX cost up to

40%

• reduce unmet demand by

37%

• reduce avg. travel time from

37 minute average to 10-22

minute average

Two Scenarios: Aggregate and Individual

• Aggregate Anonymized Analytics: Sensing City-scale people movement from Telco data and leveraging for Transit Optimization

• Enriched Consumer Profile: Customer Analytics with Mobility Profiles from Telco Data

Consumer Analytics with Enhanced Consumer Profiles

• Derive advanced location/mobility attributes and patterns from Telco data to enrich consumer profiles with mobility context

• Derive predictive model about consumers location and mobility patterns

• Leverage enriched consumer profiles for data monetization opportunities by correlating and joining other data sources

• Build an operational asset on IBM Big Data platform to enable Telco to extract mobility attributes and patterns efficiently

Set of example mobility attributes • Base set of example mobility attributes

–Home and work location

–Weekday top locations

–Weekend top locations

–Meaningful location detection

–Classification of where and when time spent

–Detecting tourism pattern

–Detecting specified habits related to mobility

– Trip purpose

–Anomaly in mobility from baseline patterns

–Detecting who’s who in the household based on mobility pattern

• Advanced predictive models (Next Best Location) –Likely place a person would be at a future time

–Likelihood of a person going to a Mall during this weekend

–When this person is likely to be a tourist

Determining Buddies, Hangouts, Life Style Example Lifestyle Attributes for marketing demonstration

Subscriber Lifestyles

Popular Locations

Subscriber Pairings

Who Are You?

Homebody

Daily Grinder

Delivering the Goods

Globetrotter

Nomad

10 Top Hangouts

Best Buddies

Next Steps • Given the lifestyles, popular locations, and best buddy data => predict where individuals or

groups of similar individuals will be and when. • Use time series modeling and clustering we can create time/location based marketing

campaigns targeted at homogenous groups in specific locales.

26

Enhanced Micro-segmentation with Mobility Model

Mobility Patterns

Mobility Model •Location and movement pattern (space, time) •Meaningful location detection •Meaningful location classification •Trip purpose •Estimated Duration of stay •Estimated Duration of travel •Mode of travel •Calling patterns •Detecting tourist patterns •Detecting student patterns •Estimated demographic profile of user of phone •Anomalies in regular patterns

Example Data Monetization Use Cases

Telcos cannot assume that person who buys phone

is the user. Discovering profile of actual user is helpful in

retail & marketing

Smarter LBS would take movement patterns (i.e, likely to be in a shopping complex on

Saturday afternoon etc.) into account instead of merely using momentary location

Telcos can find out inter-city travel patterns which are helpful to T&T

Banks can correlate ATM usage with Movement patterns for better mgmt

Life style and brand preference determination from mobility data for targeted segmentation

27

Buying Patterns

Social Patterns

Demographics

•Gender •Age group •Address •Income

•Historical buying patterns •Buying preferences •…..

•Social network influencers • friends choices • friends activities

Attributes for Customer Segmentation

28

Enhanced Micro-segmentation with Mobility Model

Mobility Patterns

Buying Patterns

Social Patterns

Demographics

•Gender •Age group •Address •Income

Historical buying patterns

Social network influencers

Mobility Model •Location and movement pattern (space, time) •Meaningful location detection •Meaningful location classification •Trip purpose •Estimated Duration of stay •Estimated Duration of travel •Mode of travel •Calling patterns •Detecting tourist patterns •Detecting student patterns •Estimated demographic profile of user of phone •Anomalies in regular patterns

Enhanced Attributes for Customer Segmentation

Building Context and Intent from Location data • Deriving location: location information may be derived using multi-modal

information

– CDR data, tower data, device data, Wi-fi etc.

– Accuracy of location information depends on data fidelity etc.

• Building context: making sense of the location information

– Correlate location information with business data

– Various other correlation rules may be used to build a rich context

• Inferring intent: infer consumer level intents by leveraging location and mobility patterns

Deriving Location Inferring Intent Building Context

Enriched Consumer Profile Hub

Customer Profile Hub

IPTV - Subscription Billing -VOD Billing & viewed - channel viewing history -- contents purchased -Logs & Tuning Events - package subscription

Mobile - Location - URL+App Transactions - xDRs and inb. roaming - RAN (incl. HLR/VLR) - Top Up - Pkgs - Billing - SMS, browing URLs

Other: - Devices - Dealer Network - Contact Center - Call Recordings - Trouble Tickeing - Campaign Results (Imagine) - Loyalty - Competition Website - Retail Transactions

Fixed - CDR - URL (IP) -Radius (IP-Cust) - Pkgs - Billing

Historical Transactions/

Events

Partners/Retailers

Advertisers

Other/Internal

GIS - Business map and numbers - Point of Interest maps

Co

nsu

me

rs o

f n

ew In

sigh

ts

Feedback

Social Media Data

1

2

3

Advanced Analytics Platform

End-use

Applications

Analytics

Visualization

Big Data Analytics

Warehouse

Predictive Analytics

Sens

e

Analyze Act

Search / Explore

KPIs

Dashboards

Drill-Downs

Reports

Marketing

Campaigns

Rules Engine

Behavioral

Analysis

Outcome

Optimization

Propensity

Scoring

Model

Creation

Structured /

Unstructured

Data

Data Governance

Data Integration

ETL/ELT

Cha

ng

e C

ap

ture

Da

ta Q

ua

lity

/ V

alid

ity / S

ecu

rity

- P

riva

cy

Fo

rma

t / U

nit C

on

ve

rsio

n

Con

so

lid

atio

n / D

e-d

up

lica

tio

n

Data

Repositorie

s

Network

Data

Customer

Behavior Data

Custo

me

r

Data

Pro

du

ct

D

ata

N

etw

ork

To

po

log

y

Data

Contin

uous F

eed

Sourc

es

Usage Data

Reference

Data

Historical

Analysis Data

Demographics

Segmentation

Location

Past Actions

Propensity

Scores

Behaviors

Predictive Model

Deployment

Actionable

Insight

Stream Processing

Streaming Data

Operational

Systems

4

5

AAP Capabilities

High Performance Historical analysis (Big Data Platform)

Model Based Analytics - behavioral scoring, micro segmentation, correlation detection analysis

Real-time scoring, classification, detection and action

Visualize, explore, investigate, search and report

Take action on analytics

IBM’s Advanced Analytics Platform (AAP) Supports Use Cases across the business with New Era Capabilities

Create new Services and Business Models Transform Operations

Build Smarter Networks

Personalize Customer Engagements

1

1

2

3

4

5

5

Retailer Customer Profile

Real Time Targeted Advertisement for IPTV

AAP (Advanced Analytics Platform)

3 - AAP catches the

new football interest

flag, his frequent

sports shopping, and

realtime matches

Tom’s profile with an

offer for 20% off

coupon to an Nike

store.

4 - Tom is also an

existing SMS Opt-

In mobile cust.

5 – Tom receives

targeted IPTV

advertisements based

on his IPTV, mobility

and social profiles

2 - Tom is channel surfing,

mostly sports channels,

primarily football games where

Nike advertises a lot (AAP enhances

his customer profile, after 10 football

games viewed in 1st month,

with an interest flag as a “football fan”) Enhanced Cust. Profile

Interest / Mobile # / Email

1- Tom activates IPTV service

with the America 50 package and

adds the ESPN sports ala carte

option (we have an initial

customer profile with his fixed #

and a mobile#)

A la carte option Sports Packages

tom@gmail.com

212-201-1234

Language Package

Location Based Real Time Offering on Mobile Phone

Lisa

4 - AAP catches that

Lisa is entering a mall,

and matches her

“Fashion” interest flag

and “Perfume”

preference, realtime

with an offer for 20%

off coupon for Byonce

fragrance at Sephora

in that mall.

5 - Lisa receives

an SMS/email/App

notification that

her mobile app

account contains a

new offer for

Beyonce perfume.

Beyonce Fan Page

2 - She follows a

friend’s post on FB and

clicks the Like button on

the Beyonce Fan Page.

3 - Lisa’s IPTV viewing

& mobile clickstream

behaviors set her Interest

flag to “Fashion” and one

preference to “Perfume”.

6 - Lisa uses

the mWallet

app on her

smartphone to

purchase some

perfume at POS

via NFC.

1- Lisa is a mobile subscriber

with Telco and downloads the

mobile app and agrees to receive

offers related to her interests.

AAP (Advanced Analytics Platform)

Retailer Customer Profile

Enhanced Cust. Profile

Interest & Preference

IPTV a la carte option & Mobile Features/Apps

IPTV Lang Pkg & Mobile Pkg

Thank You Your feedback is important!