RTBDAReal Time Big Data Analytics

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Agenda• Why?• How?• What?• Questions?• Let’s connect and take this further

Why?

• Data touch points have increased• Machine generated or auto-

generated data• Data generated based on behavior

(not mere accounting entries)• Online data generation• Data generation from social touch

points• Analytics on unstructured data from

multiple sources is picking up• Large Memory and Faster disks• Virtualization is becoming popular

World is changing a lot and pretty fast

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

What is expected from IT in changing world?• Gone are the times when IT was just supporting business• Today IT is driving business and for IT is business!

• Boom of Technology companies• Companies investing in IT innovation

• Decisions are based on various systems• Services are offered and facilitated to end customers

based on data from multiple system• New offerings and customer acquisition happens based on

correlation between data in different system• Many key decision are to be taken in real time based on

patterns, learning, experiences and corelation

How?

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Real Time Analytics - Explained• It is about answering and validating questions

which were unknown a few years ago• It is the capability of using all the data that you

have at your disposal to do dynamic analytics• It is not just about storing data but also querying

that data in real time to make better decisions• But what’s is the use if time for making a decision has already

come and passed?

• Right action, at right time and at right place, when it matters!

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Example Applications• Analysizing ticks, tweets, images, weather

trends• Trading alogorithm in real time• Identifying individual buyers and use a holistic

behavioral model to advertise to them.• Identifying scoial fraud based on citizen profile,

and geospatial data.

What?

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Real Time Big Data Analytics - What it takes?• A data store• A Real Time analytics engine• A quick way of accessing the data you need

with intuitive way of expressing your query• A dash boarding technology

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

What is needed from the Data Store?

• Which can scale to store large amount of data and process it faster• Which should be capable of storing and aggregating data coming from

different sources and structure• Capable of dynamically adapting to changing data structure over a

period of time.• Capability to query your data store in most intuitive way

• So that queries can be expressed, answered and validated quickly• Capability to express complex aggregation tasks

• e.g. count of users buying camera Vs count of users buying camera and lens

• Capability to express correlation• e.g. which of the users reading technology article would buy a

particular handphone

MongoDB makes IT easy!

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Flexibility - Combine data from multiple Sources• MongoDB can incorporate different data

formats from multiple source and structures• Agility - No matter how often the structure

changes, MongoDB can accept changes with minimal interventions

• Helps your analytical engine to be comprehensive• MongoDB can analyze data of any structure directly within the

database, giving results in real time, without expensive data warehouse

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Scalability - Scale out horizontally as Data Grows!• MongoDB offers scalability - is built to perform and

scale out on commodity hardware, in your data center or in cloud.

• MongoDB’s offers sharding capability which distributes data across commodity nodes with complete application transparency

• Various scaling options – range based, hash based and location aware sharding

• Tag Aware sharding can help you keep the ‘hot data’ (most relevant) on fastest hardware• Colder Data (older data) will automatically moved to slower shards

Confi

denti

al in

form

ation

, for

inte

rnal

use

on

ly

Complexity Vs Performance: Classic Dilemma of Real Time Analytics• MongoDB helps run complex ad-hoc analytics and

reporting• Rich Query Support• Robust aggregation framework - with Pipeline approach• Native Map-reduce support• Has rich index support - various kind of indexes

• Secondary• Geospatial• Text• Time to Live

Questions?