De prakti jk van Big Data

En waarom de huidige technologie niet (alt i jd) voldoet

Friso van Vollenhovenfvanvollenhoven@xebia.com

Big Data

Big Data

Big Data

Big Data

Requirement:Full table scan, 200GB table

Big Data

Big Data

Egypte, 27 januari 2011

Big Data

Requirement:40.000 updates per seconde, 24/7.

Databases

++=

Databases

++=

SANstorage

network

HDFS en MapReduce

storagenetwork

CLIENT

bottleneck

SELECT SESSION, COUNT(*) FROM WEB_CLICKS GROUP BY SESSION;

HDFS en MapReduce

storagenetwork

bottleneck

CLIENT

SELECT SESSION, COUNT(*) FROM WEB_CLICKS GROUP BY SESSION;

HDFS en MapReduce

HDFS en MapReduce

SELECT * FROMWEB_CLICKS;

SELECT * FROMWEB_CLICKS;

SELECT * FROMWEB_CLICKS;

HDFS en MapReduce

GROUP BY SESSION

HDFS en MapReduce

COUNT(*)

COUNT(*)

COUNT(*)

HDFS en MapReduce

SELECT * FROMWEB_CLICKS;

SELECT * FROMWEB_CLICKS;

SELECT * FROMWEB_CLICKS; COUNT(*)

COUNT(*)

COUNT(*)

GROUP BY SESSION

MAP REDUCE

MAP REDUCE

MAP REDUCE

SORT/SHUFFLE

NoSQL

A B C D E F G H I J K L M N O P Q R S T U V W X Y Zindex