fujitsu world tour 2015 · refine value e s you need an ... 3 asia rats 2000 lionhead 4 europe rats...

0 Copyright 2015 FUJITSU

Human Centric Innovation

Fujitsu World Tour 2015

From Big Data to Big Value


From Big Data to Big Value

Gernot Fels Global Marketing, Fujitsu


Some dreams organizations dream

What if you could …

Predict what will happen?

Market trends

Customer behavior

Business opportunities

Problems

Take the right decisions?

Accelerate your decisions?

Have respective actions initiated automatically?

Fully understand the root cause of cost?

Skip useless activities?

Quantify and minimize risks?

Know, what you don’t know?

Can these dreams come true?


Entering new dimensions …

Versatile data sources – internal and external

DB

Text

Mail

Multi-media

Machine logs

Web logs

Social media

Internet

Sensors (GPS, RFID, …, IoT)

…

Versatility

http://www.iconarchive.com/show/my-seven-icons-by-itzikgur/Pictures-Canon-icon.html

http://www.iconarchive.com/show/pretty-office-8-icons-by-custom-icon-design/Sound-on-icon.html

http://www.iconarchive.com/show/button-icons-by-deleket/Button-Play-icon.html

http://www.iconarchive.com/show/rumax-icons-by-toma4025/mails-icon.html

http://www.iconarchive.com/show/soft-scraps-icons-by-deleket/Text-Edit-icon.html

http://www.iconarchive.com/show/ivista-2-icons-by-gakuseisean/Misc-Database-3-icon.html

http://www.iconarchive.com/show/glaze-icons-by-mart/log-icon.html

http://www.iconarchive.com/show/oxygen-icons-by-oxygen-icons.org/Categories-applications-internet-icon.html

http://universalfuze.com/uploads/wireless-icon.gif




DB

Text

Mail

Multi-media

Machine logs

Web logs

Social media

Internet


…

Various data types

Structured

Unstructured

Semi-structured

Poly-structured

Variety

Versatility




DB

Text

Mail

Multi-media

Machine logs

Web logs

Social media

Internet


…

Various data types

Structured

Unstructured

Semi-structured

Poly-structured

Large data volumes

TB to PB

Volume

Variety

Versatility




DB

Text

Mail

Multi-media

Machine logs

Web logs

Social media

Internet


…

Various data types

Structured

Unstructured

Semi-structured

Poly-structured

Large data volumes

TB to PB

High velocity

Generation and processing

Real-time demands

Volume

Variety

Versatility

Velocity




DB

Text

Mail

Multi-media

Machine logs

Web logs

Social media

Internet


…

Various data types

Structured

Unstructured

Semi-structured

Poly-structured

Large data volumes

TB to PB

High velocity

Generation and processing

Real-time demands

Generate new value by correlation of data and new ways of analytics.

Volume

Variety

Versatility

Velocity

Value


The promise of Big Data for business and society

Discover hidden secrets

Predict opportunities

Identify and minimize unknown risks

Take better and faster decisions

Accelerate business processes

Increase performance and productivity

Improve efficiency and effectiveness

Profitability and competitive advantage

Better utilize our planet‘s resources

Big Data makes dreams of organizations come true.

http://mediaportal.ts.fujitsu.com/pages/view.php?ref=28617&search=!collection3&offset=96&order_by=field12&sort=DESC&archive=0&k=


Big Data is the new oil

Volume

Variety

Versatility

Velocity

Value

Discover

Extract

Refine

Value



Volume

Variety

Versatility

Velocity

Value

Discover

Extract, collect

Transform

Analyze, visualize

Decide, act

Discover

Extract

Refine

Value



Volume

Variety

Versatility

Velocity

Value

Discover

Extract

Refine

Value

Infr

astr

uct

ure

Serv

ices

You need an infrastructure and services to make it happen.

How to get from Big Data to Big Value?

A lot of questions …

Which use case (business priority)?

Which data from which sources?

Is external data trustworthy, and always (free) available?

How to transform data into high quality?

How to explore data and discover its meaning?

What to look for? Which questions to ask?

Which actions to trigger? Are actions executable?

Which analytic methods?

How to visualize results effectively?

Which tools? How to use them?

How to ensure data security and privacy?

What about data retention and deletion?

Which skills?

Creative ideas, deep knowledge of data, tools and intended results.


Manufacturing

Energy

Maintenance

Agriculture

Big Data matters to every industry

Big Data

Healthcare Transportation

New opportunities, new values for enterprises and society.

Retail Finance

Public Sector …


Big Data infrastructure challenges

Big Data requires new solution approaches.

Exponential growth of

data

Keep processing time constant while volumes increase

Store and process large data volumes at affordable cost

Ad-hoc queries and real-time demands

Process continuously generated event streams


Distributed Parallel Processing

Concept

Distribute data and I/O to many nodes

Local server storage

Move computing to where data resides

Shared nothing architecture

Data replication to several nodes

Benefits

High performance, fast results

Unlimited scalability

Fault-tolerance

Cost-effective (standard servers with OSS)

Examples

Task

Master

Slaves DFS

Data

Task

Data

Task

Data

Control jobs and data


Applications

Distilled Essence

What it is

Consolidated result of data transformation

Usually much smaller than initial data volumes

All essential information for use case included

Base for analytics

Where it is stored

Local disks in server cluster

External HDD storage

AFA

…

What if immediacy matters?

Distilled Essence


In-Memory Platform

How to ensure business continuity?

Applications

Distilled Essence

RAM

RAM

RAM

IM Platform

Concept

Resident (distributed) data store

Scale-up, scale-out

Benefits

Fast data access

Fast decisions

Performance as you grow


In-Memory Platform

Applications

Distilled Essence

RAM

RAM

RAM

IM Platform

Snapshots

Change log

Persistence

Concept


Scale-up, scale-out

Persistent storage for automatic recovery

Meet real-time demands, ensure business continuity.

Benefits

Fast data access

Fast decisions



In-Memory Platform

Concept


Scale-up, scale-out

Persistent storage for automatic recovery

Data mirror for ultra-fast recovery

Applications

Distilled Essence

RAM

RAM

RAM

IM Platform

Snapshots

Change log

Persistence

RAM

RAM

RAM

Mirror

Meet real-time demands, ensure business continuity.

Benefits

Fast data access

Fast decisions



Column-oriented DB

Row-oriented data store

Column-oriented data store

Many rows (records)

Queries often relate to small number of columns only

Inefficient to read all rows

Access only relevant columns

Reduced I/O

Fast analysis

High compression

Column contains only 1 data type

Often only few values per column (to store)

Faster results despite large data size.

Row-oriented storeKey Region Product Sales Vendor

0 Europe Mice 750 Tigerfoot1 America Mice 1100 Lionhead2 America Rabbits 250 Tigerfoot3 Asia Rats 2000 Lionhead4 Europe Rats 2000 Tigerfoot

Column-oriented storeKey Region Product Sales Vendor

0 Europe Mice 750 Tigerfoot1 America Mice 1100 Lionhead2 America Rabbits 250 Tigerfoot3 Asia Rats 2000 Lionhead4 Europe Rats 2000 Tigerfoot

SELECT SUM (Sales) GROUP BY Region

Data required

Data read, but not required


Databases – Designed for Big Data

Flexible data model

Designed to be distributed

Schema-less (simple structure)

Easily cope with new data types

Change format of data without app disruption

Ease of app development

Data automatically spread across servers

Support distributed query execution

Designed for scale-out

Add server to cluster on demand

Standard server HW

Integrated caching

Fault tolerance

Reduce latency, increase throughput

Improve app responsiveness

Data replication (across cluster / DC)

Automatic recovery from failure

Various flavors

Graph DB

Key-value stores

Document stores

Columnar stores

NoSQL databases help overcome the limitations of RDBMS.


Complex Event Processing (CEP)

Concept

Critical parameters

Collect and analyze continuously generated data

Rules for logical and temporal correlation

Event pattern matching

Trigger action in real-time

Throughput: 10K’s to M’s of events per sec

Latency: µs to ms (time from event input to output)

Event input

Ru

les

State transition

Apply Event output

Control CEP Engine

Sensor


Complex Event Processing (CEP)

Concept

Critical parameters

Collect and analyze continuously generated data

Rules for logical and temporal correlation

Event pattern matching

Trigger action in real-time

Throughput: 10K’s to M’s of events per sec

Latency: µs to ms (time from event input to output)

Streaming

Event input

Ru

les

State transition

Apply Event output

Control CEP Engine

Routing

Options for acceleration

Data in RAM (long time window)

Parallelization (large streams, many / complex rules)

CEP requires parallel processing and in-memory technologies.

Sensor Sensor Sensor


μsec msec sec min hrs

CEP (Complex Event Processing)

Response time

Conventional technologies (e.g. RDBMS)

In-Memory

GB

TB

PB

Dat

a vo

lum

e Big Data technologies at a glance

Select the right technology depending on volume and time requirements.

Distributed Parallel Processing


Un- / Semi-/ Poly-

structured data

Big Data infrastructure master plan

Data Sources Analytics Platform Access

Extract, Collect Clean, Transform Decide, Act Analyze, Visualize

Consolidated data Distilled essence Applied knowledge Various data

Dat

a at

res

t D

ata

in m

otio

n

DB

DW


Un- / Semi-/ Poly-

structured data





Dat

a at

res

t D

ata

in m

otio

n

DB

DW Distributed

Parallel Processing

Apps Services Queries

….

Visualization Reporting

Notification


Un- / Semi-/ Poly-

structured data





Dat

a at

res

t D

ata

in m

otio

n

DB

DW Distributed

Parallel Processing


….


Notification

In-Memory

DB / DW

NoSQL


Un- / Semi-/ Poly-

structured data





Dat

a at

res

t D

ata

in m

otio

n

DB

DW Distributed

Parallel Processing


….


Notification

In-Memory

DB / DW

NoSQL

CEP


Un- / Semi-/ Poly-

structured data






….


Notification

Distributed Parallel

Processing

CEP

Dat

a at

res

t D

ata

in m

otio

n

In-Memory

DB / DW

NoSQL

DB

DW


How can help

Optimum solution for every situation

Combination of technologies

Infrastructure products

SW (OSS, Fujitsu, ISV)

Integrated Systems

End-to-end services

Assessment, consulting

Solution design

Deployment, integration, maintenance

Attractive financing options

Sourcing options

Self-managed (on-premise)

Managed services (on- / off-premise)

Fujitsu Cloud

One-stop shop for Big Data: Reduce complexity, time and risk.

…

Experience across

industries

http://mediaportal.ts.fujitsu.com/pages/view.php?ref=28576&search=!collection3&offset=96&order_by=field12&sort=DESC&archive=0&k=


Integrated Systems for Big Data Infrastructures

PRIMEFLEX for Hadoop PRIMEFLEX for SAP HANA …

Hadoop / Datameer Services

Pre-integrated, pre-tested, ready-to-run: Fast deployment, short time to production.

Analytical apps / templates

Datameer (Visual Analytics)

Hadoop (HDFS, MapReduce)

Linux OS

PRIMERGY RX / CX

Brocade switches

Fujitsu HANA Services

SAP HANA DB

Linux OS

PRIMERGY / PRIMEQUEST

Persistent storage

NW switches (multi-node only)


Summary

Big Data offers enormous potential for business value

If you ignore it, your competitor won’t !

It changes the way companies make decisions, do business, succeed or fail

New ideas and use cases

New analytics

New tools

New skills

New infrastructure concepts

Fujitsu – A one-stop shop for Big Data

Enabler, making your dreams come true

From Big Data to Big Value? Have a word with FUJITSU.

fujitsu world tour 2015 · refine value e s you need an ... 3 asia rats 2000 lionhead 4 europe rats...

Documents