Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Real-Time Visual Analytics for Event Data Streams Fabian Fischer, Florian Mansmann, Daniel A. Keim

27th March 2012, ACM SAC 2012 Riva del Garda (Trento), Italy

2 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Visual Analytics

Interactive Visualization is a way to tightly combine human factors and data analysis.

Human Analyst

Understanding

Expert Knowledge

Experience

…

Cognition

Intuition

Data Mining

Clustering

Statistics

Massive Processing Power

Machine Learning

…

Classification Burst Detection

Use Case for Event Streams

Analyzing System Log Events (event stream of server log messages)

The National Archives (UK), 2011

4 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Framework Architecture Real-Time Visual Analytics for Event Data Streams

Event Service Data Streams

Message Broker

raw messages

Data Storage analyzed events

Event Analyzer(s) Event Analyzer(s) Event Analyzer(s) Event Analyzer(s)

Event Analyzer(s) Event Analyzer(s) Event Analyzer(s) Event Visualizer analyzed

events

connect to data storage

raw messages

Fingerprint

Normalization

Rules

Scoring

Aggregation

6 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Relaxed Event Timeline Visualization Focus on Temporal Aspect of Data Streams (Monitoring & Exploration)

s1

s2

s3

A

B E

C D F G H I

J

K

color mapped to priority selected scale: one hour (h)

hnow - 1 hnow

Demo/Video

12 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Main Contributions

• Generic processing and analysis architecture for event data streams to support real-time visual analytics applications.

• A system for pluggable visualizations for real-time and historical event data.

• Dynamic timeline visualization to directly interact with multiple streams to visualize highly co-occurring events.

13 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Future Work

• Controlled system evaluation.

• Integration of advanced algorithms for burst and anomaly detection.

• Integration of more visualizations based on the learned design principles.

• Use the Event Visualizer for other datasets.

– Feb 2012 – Successful participation in the Honeynet Forensic Challenge 2011/10 [1].

[1] http://ff.cx/fc10/

14 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

Thank you very much for your

attention!

Questions?

For more information about this work or about visual analytics please contact

Fabian Fischer

Tel. +49 7531 88-2780 Fabian.Fischer@uni-konstanz.de

http://ff.cx/

@f2cx

15 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

References I

J. Thomas and K. Cook (2005). Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Computer Society, 2005.

W. Aigner, S. Miksch, H. Schumann, and C. Tominski (2011).

Visualization of Time-Oriented Data. Human-Computer Interaction. Springer Verlag, 1st edition, 2011.

16 Fabian Fischer | Data Analysis and Visualization Group | University of Konstanz

References II

G. Chin, M. Singhal, G. Nakamura, V. Gurumoorthi, and N. Freeman-Cadoret (2009).

Visual Analysis of Dynamic Data Streams. Information Visualization, 8(3):212-229, 2009.

M. Schaefer, F. Wanner, F. Mansmann, C. Scheible, V. Stennett, A. T. Hasselrot, and D. A. Keim (2011).

Visual Pattern Discovery in Timed Event Data.

In Proceedings of Conference on Visualization and Data Analysis, 2011.