Visualizations for Event Sequences Exploration

Krist Wongsuphasawat

Data Visualization Scientist Twitter, Inc.

@kristw

Data Visualization Summit San Francisco, CA

Apr 11, 2013

Life event%

event%event%

event%event%

event%event%event%

event%

event%

event%

event%

event%

event% event%event%

( 7:00 am, Wake up )

Life event%

event%event%

event%event%

event%event%event%

event%

event%

event%

event%

event%

event% event%event%

Time Event type%

Life event%

event%event%

event%event%

event%event%event%

event%

event%

event%

event%

event%

event% event%event%

“Event Sequence”

Daily Activity

7:30 a.m. Wake Up

7:45 a.m. Exercise

8:30 a.m. Go to work

Traffic Incidents

9:30 a.m. Notification

9:55 a.m. Units arrived

10:30 a.m. Road cleared

http://timeline.national911memorial.org/

Event Sequences

and more…

Medical Transportation

Education

Web logs

Sports

Logistics

Outline

What are event sequences?

How to visualize them?

Apply to big data

Visualization Techniques

Event sequence

glyphs timeline

http://timeline.verite.co/

simple event sequence timeline.js

Horizontal axis = time

Glyphs = events

Event sequence

glyphs timeline

Interval +

interval

•  Car crash (point) 10 a.m.

time

•  Meeting (interval) 10 – 11 a.m.

CATT Lab, University of Maryland -- http://teachamerica.com/VIZ11/VIZ1102Pack/index.htm

interval >> width tra!c incident

http://stoicloofah.github.io/chronoline.js/

interval >> width chronoline.js

Event sequence

glyphs timeline

Interval +

Event types

+

width

types

time

Nurses’ actions Doctors’ actions

They all look similar.

types

time

Nurses’ actions Doctors’ actions

Better?

http

://w

ww

.gua

rdia

n.co

.uk/

wo

rld

/int

erac

tive

/20

11/m

ar/2

2/m

idd

le-e

ast-

pro

test

-int

erac

tive

-tim

elin

e

types >> color The path of protest

http

://ti

meg

lider

.co

m/w

idg

et/

types >> colors + shapes timeglider.js

Event sequence

glyphs timeline

Interval +

Event types

+

High density

+

colors shapes

width

high density

time

Too many overlaps and occlusions

Google Chrome > Developer Tools > Timeline

high density >> facet Google Chrome

scripting rendering & painting

Facet

loading

http://www.cs.umd.edu/lifelines

high density >> facet Lifelines

high density >> binning British History Timeline

bin by year

high density >> aggregation CloudLines

Raw event data

Kernel Density Estimation + Importance Func. + Truncation

Encode cloud size

Krstajic, M., Bertini, E., & Keim, D. A. (2011). CloudLines: Compact Display of Event Episodes in Multiple Time-Series. IEEE Transactions on Visualization and Computer Graphics, 17(12), 2432.

high density >> aggregation CloudLines (2)

Event sequence

glyphs timeline

Interval +

Event types

+

High density

+

colors shapes

width

facet aggregation

linear

non-linear

binning

circular timeline

2008 2009 2010 2011 2012

linear

repeating patterns circular

Jan Feb

Mar

Apr

May

Jun Jul

Aug

Sep

Oct

Nov

Dec

VanDaniker, M. (2010). Leverage of Spiral Graph for Transportation System Data Visualization. Transportation Research Record: Journal of the Transportation Research Board, 2165, 79–88.

circular timeline (2) Tra!c Incidents

stacked timeline

2008

2009

2010

2011

2012

2008 2009 2010 2011 2012

linear

200

8

200

9

2010

2011

2012

Rios, M., & Lin, J. (2012). Distilling Massive Amounts of Data into Simple Visualizations : Twitter Case Studies. Proceedings of the Workshop on Social Media Visualization (SocMedVis) at ICWSM 2012 (pp. 22–25).

stacked timeline (2) Tweet Volume

Event sequence

glyphs timeline

Interval +

Event types

+

High density

+

colors shapes

width

facet aggregation

linear

non-linear

binning

Event sequence

Event sequence

Event sequence

...

1 2 n collection

collection multiple timelines

Event sequence #1

Event sequence #2

Event sequence #3

Event sequence #4

Event sequence

Event sequence

Event sequence

...

1 2 n collection

Millions!

Event sequence

Event sequence

Event sequence

...

1 2 n

Interactions

collection

Interaction #1

align

Interaction #1

align

Interaction #1

align

Interaction #2

rank

Interaction #2

rank

Rank by number of events or any criteria

Interaction #3

filter

Interaction #3

filter

Select only event sequences with events Set your own filters

Interaction #4

group

Interaction #4

group

Group by sequence length

1

2

3

or any clustering algorithm / properties

Interaction #5

search •  Simple search –  Sequence matching –  Subsequence matching

•  Regular Expression

ABC

A B* (C|D)

AABCDEFGH AXAYBZCED

AB C 75% D 25%

X 50% Y 50%

Interaction #5

search (2) •  Dynamic

ABC D 50% E 50%

X 70% Y 30%

Interaction #5

search (2) •  Dynamic

•  Similarity search ABCD Similar to

ABCD ABD ACE …

Event sequence

Interactions

search

Event sequence

Event sequence

...

1 2 n

Aggregation

rank

filter group

align by

time

collection

aggregation by time

temporal summary

bin & count

Day 1 Day 2 Day 3 Day 4 Day 5

Wan

g, T

. D.,

Pla

isan

t, C

., S

hnei

der

man

, B.,

Sp

ring

, N.,

Ro

sem

an, D

., M

arch

and

, G.,

Muk

herj

ee, V

., et

al.

(20

09

).

Tem

po

ral S

umm

arie

s: S

upp

ort

ing

Tem

po

ral C

ateg

ori

cal S

earc

hing

, Ag

gre

gat

ion

and

Co

mp

aris

on.

IE

EE

Tra

nsac

tio

ns o

n V

isua

lizat

ion

and

Co

mp

uter

Gra

phi

cs, 1

5(6

), 10

49

–10

56.

aggregation by time

temporal summary

Event sequence

Interactions Aggregation

search

Event sequence

Event sequence

...

1 2 n

rank

filter group

align by

time

by sequence

collection

aggregation by sequence

LifeFlow

e.g. 1) What happened to the patients after they arrived?

Arrival!

ICU!

?

? ? ? ?

?

2) What happened to the patients before & after ICU?

aggregation by sequence

LifeFlow

Millions of records!

overview / summary

Demo LifeFlow

Wongsuphasawat, K., Guerra Gómez, J. A., Plaisant, C., Wang, T. D., Taieb-Maimon, M., & Shneiderman, B. (2011). LifeFlow: Visualizing an Overview of Event Sequences. Proceedings of CHI'2011 (pp. 1747–1756).

Demo LifeFlow

Wongsuphasawat, K., Guerra Gómez, J. A., Plaisant, C., Wang, T. D., Taieb-Maimon, M., & Shneiderman, B. (2011). LifeFlow: Visualizing an Overview of Event Sequences. Proceedings of CHI'2011 (pp. 1747–1756).

Demo LifeFlow

Wongsuphasawat, K., Guerra Gómez, J. A., Plaisant, C., Wang, T. D., Taieb-Maimon, M., & Shneiderman, B. (2011). LifeFlow: Visualizing an Overview of Event Sequences. Proceedings of CHI'2011 (pp. 1747–1756).

aggregation by sequence

LifeFlow

contact!

home!

profile!

home!

home!

start! photos! home!

http://www.google.com/analytics

aggregation by sequence

Google Analytics

contact!

home!

profile!

start! photos! home!

http://www.google.com/analytics

aggregation by sequence

Google Analytics

contact!

home!

profile!

start! photos!

home!

videos!

http://www.google.com/analytics

aggregation by sequence

Google Analytics

top pages only

height = number of visits

Event sequence Outcome +

Game #1

Time%

10th minute Goal

90th minute Goal

25th minute Concede

Win (1)

or any sports

Game #1

Game #2

Time%

Game #3

Game #n

Lose (0)

Win (1)

Win (1)

Win (1)

Goal% Concede% Goal%

Goal% Goal% Concede%

Goal% Concede%Concede%

Concede% Goal%Goal%Goal%

aggregation by sequence with outcome

Outflow (Careflow) overview / summary

Event Sequences!with Outcome!

Assumption

e1%

Record #1

e2% e3%

Record #1

Events are persistent.

Assumption

e1%

Record #1

e2% e3%

e1%

Record #1

e1% e1%

Events are persistent.

Assumption

e1%

Record #1

e2% e3%

e1%

Record #1

e1%e2%

e1%e2%

Events are persistent.

Assumption

e1%

Record #1

e2% e3%

e1%

Record #1

e1%e2%

e1%e2%e3%

Events are persistent.

Assumption

e1%

Record #1

e2% e3%

e1%

Record #1

e1%e2%

e1%e2%e3%

[e1]

[e1, e2]

[e1, e2, e3] States

Events are persistent.

Select alignment point

Pick a state

What are the paths that led to ?

What are the paths after ?

Soccer: Goal, Concede, Goal

Example

Outflow Graph

[e1, e2, e3]!

Alignment Point

Outflow Graph

[e1, e2, e3]!

[e1, e2]!

[e1, e2, e3, e5]!

[e1]!

[ ]!

Alignment Point

1%record%

Outflow Graph

[e1, e3]!

Alignment Point

2%records%

[e1, e2, e3]!

[e1, e2]!

[e1, e2, e3, e5]!

[e1]!

[ ]!

Outflow Graph

[e1, e2, e3, e4]!

Alignment Point

[e3]!

3%records%

[e1, e3]!

[e1, e2, e3]!

[e1, e2]!

[e1, e2, e3, e5]!

[e1]!

[ ]!

Outflow Graph

[e2, e3]!

[e2]!

Alignment Point

n%records%

[e1, e2, e3, e4]!

[e3]!

[e1, e3]!

[e1, e2, e3]!

[e1, e2]!

[e1, e2, e3, e5]!

[e1]!

[ ]!

Outflow Graph

[e2, e3]!

[e2]!

Alignment Point

n%records%

Average outcome Average time No. of records

= 0.4 = 10 days = 10

[e1, e2, e3, e4]!

[e3]!

[e1, e3]!

[e1, e2, e3]!

[e1, e2]!

[e1, e2, e3, e5]!

[e1]!

[ ]!

Soccer Results

2-1!

2-0!

1-1!

0-2!

2-2!

3-1!

1-0!

0-1!

0-0!

Alignment Point

Alignment%Future&Past&

e1!e2!

e1!

e2!

e1!e2!e3!

e1!e2!e4!

Color is outcome measure.%

Node’s height is number of records.%

Time edge’s width is duration of transition.%

Node’s horizontal position shows sequence of states.%

7me%edge%

link%edge%

End of path%

Wongsuphasawat, K., & Gotz, D. (2012). Exploring Flow, Factors, and Outcomes of Temporal Event Sequences with the Outflow Visualization.

IEEE Transactions on Visualization and Computer Graphics, 18(12), 2659–2668.

Event sequence

Interactions Aggregation

search

Event sequence

Event sequence

...

1 2 n

rank

filter group

align by

time

by sequence

collection

Outcome +

Application to Big Data Analysis

Something sounds simple X

magnitude of big data =

Big mess & Big reward

Event Sequence Analysis at

eBay CheckoutProcStep1

PaymentReview

CheckoutProcStep2

CheckoutProcStep3

PaymentConfirm

CheckoutProcStep4

CheckoutProcStep5

CheckoutProcStep6

CheckoutSuccess

She

n, Z

., W

ei, J

., S

und

ares

an, N

., &

Ma,

K.-

L. (

2012

).

Vis

ual a

naly

sis

of

mas

sive

web

ses

sio

n d

ata.

IE

EE

Sym

po

sium

on

Larg

e D

ata

Ana

lysi

s an

d V

isua

lizat

ion

(LD

AV

), 6

5–72

.

Event Sequence Analysis at

eBay alignment

Event Sequence Analysis at

Twitter •  Data

–  TBs of session logs everyday •  Complexity

–  millions of sessions per day –  1000+ types of events –  long sessions

•  Goal –  Overview of how users are using Twitter

•  Technique –  LifeFlow

Simplify!

Event Sequence Analysis at

Twitter (2) •  So far

–  millions of sessions per day –  millions of sessions on the same screen –  1000+ types of events –  simplified sets of events

•  e.g., pages only, selected pages only

–  long sessions –  limited session length to 10-20 events

Event Sequence Analysis at

Twitter (3) Session%Start%

Page%A% Page%B% Page%C%

Page%B% Page%A%

Page%C%

Page%D%

Page%B%

Page%D%

*fake data

Page%C%Page%D%

Page%C%

Event Sequence Analysis at

Twitter (4) •  Implementation

–  Hadoop  –  Web-based (js)

•  More –  Stored preprocessed data in smaller db

(MySQL/Vertica)

HDFS MySQL / Vertica

Batch pig scripts

Visualization

Interactive

Krist Wongsuphasawat krist.wongz@gmail.com

@kristw

•  Life is full of event sequences.

•  How to visualize an event sequence

Takeaway Messages

Event sequence

glyphs timeline

Interval +

Event types

+

High density

+

colors shapes

width

facet aggregation

linear

non-linear

binning

Krist Wongsuphasawat krist.wongz@gmail.com

@kristw

•  Life is full of event sequences.

•  How to visualize an event sequence

•  How to visualize collection of event seq.

Takeaway Messages

Event sequence

Interactions Aggregation

search

Event sequence

Event sequence

...

1 2 n

rank

filter group

align by

time

by sequence

collection

Outcome +

Krist Wongsuphasawat krist.wongz@gmail.com

@kristw

•  Life is full of event sequences.

•  How to visualize an event sequence

•  How to visualize collection of event seq.

•  Applicable to big data

•  New techniques happen everyday.

Takeaway Messages

delete keep

http://notabilia.net/

…

Smurf Communism - Wikipedia

http://www.evolutionoftheweb.com

Krist Wongsuphasawat krist.wongz@gmail.com

@kristw

•  Life is full of event sequences.

•  How to visualize an event sequence

•  How to visualize collection of event seq.

•  Applicable to big data

•  New techniques happen everyday.

Takeaway Messages