© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Enterprise Operational SQL on Hadoop – Trafodion Overview Rohit Jain

Distinguished & Chief Technologist

Strategic & Emerging Technologies

Enterprise Database Solutions

2 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Agenda

Directions in Hadoop

Directions in NoSQL

The case for Enterprise Operational SQL on Hadoop

HP’s Enterprise Operational SQL on Hadoop Solution – Trafodion

3 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Directions in Hadoop …

Text / Document

Images

Social media eMail Mobile

Video Audio

Unstructured

Big Data Volume

Elastic Scalability

Characteristics

Lower cost Software Server & storage

Lower cost

Enterprise Integrated

Hadoop Data Lake

Real-time Analytics Internet of things

Storm

Machine Learning

MapReduce Stinger

BI & Analytics Open Source & proprietary Eco-system

Enterprise Readiness & Manageability

HP Data Services Manager

Apache Yarn Kiji

Now available on the Vertica Marketplace

4 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

… and what’s missing

Integration of structured, semi-structured, and unstructured support

Operational transactional workloads

Using Hadoop for all operational SQL needs – Not Only Big Data

Item id Description Cost Price … Structured

Type Display Size Resolution Brand Model 3D …

… ISBN Author Publish Date Format Dept

TV

Book

…

Semi- structured

Queries

SELECT all TVs WHERE Price > 2000 and Type = ‘Plasma’ and Display Size > ‘50’ and customer sentiment is very positive

Unstructured Image …

Review …

Open distributed HDFS structures

HBase & Hive Free at last!

Capture data directly into open file structures

Accessible for reporting & analytics with no latency

What’s missing

Add item BEGIN WORK INSERT item into Trafodion table ITEM (item_id, desc, cost, price, …) INSERT item attributes for TV or book into HBase table ITEM_ATTR as col-value pairs using item_id END WORK

Transaction

Backend operational workloads Order tracking, supply chain, inventory control, …

Orders needing transactional support

across Order and Order Detail

5 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Directions in NoSQL … and what’s missing

Full ACID transactions

SQL querying capability

Joins

Data in open HDFS except for HBase

Variety: Semi-structured & Unstructured

• Key-value (Riak, Redis) • Document – JSON / BSON

(MongoDB, Couchbase) • Column families (Cassandra, HBase) • Graph (Neo4j, Giraph, Titan)

Schema Flexibility

Velocity

Low Latency

High Availability

Big Data Volume

Elastic Scalability

Characteristics

Lower cost Software Server & storage

Lower cost

In-memory

Distributed across data centers

Eventual consistency

What’s missing

6 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

The Case for Enterprise Operational SQL-on-Hadoop

• Sector Road Map: SQL-on-Hadoop platforms in 2013 – Joseph Turian, March 20, 2013

– If a strong player or two emerges in the category, it will completely shake up the big data and database landscape. If Hadoop were operational, it could be used to power websites and store transactions. Traditional SQL databases would no longer be necessary. The data stack would be significantly simplified.

– An operational database offers write access, not just read access, to data. However, there are other key features for an operational database: concurrency, interactive write speed, and distributed transactional support (guarantees about data consistency). Currently no existing SQL-on-Hadoop solution satisfies these requirements.

• 5 Reasons Hadoop is Kicking Can and Taking Names – Mike Gualtieri, October 22, 2013

– #5 The future of Hadoop is real-time and transactional. The key commercial vendors are focusing on fast SQL access, real-time streaming, and manageability features that enterprises demand. The groundwork is being laid for an eruption in data management technologies as Hadoop sneaks its way into the transactional database market.

• The Future of Hadoop: What Happened & What's Possible? – Doug Cutting, Oct 30 2013

– So I think the prediction we can make here is that it is inevitable that we will see just about every kind of workload be moved to this platform – even Online Transaction Processing.

7 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Trafodion – Enterprise Operational SQL on Hadoop

Structured OLTP through EDW

• Open distributed HDFS • Semi-structured & unstructured support • Schema flexibility • Elastic scalability • Automatic rebalancing • Replication for High Availability (k-safety) • Disaster Recovery (via MapR) • Column level access control • Column level encryption • Lower cost – cheap storage & servers • Space quotas (via MapR) • Huge open source & proprietary eco-system • Versioning snapshot support & incremental

data replication • Cloud enabled HP Cloud Services OpenStack • Industry trend towards Enterprise Hadoop Lake

OLTP K/V & document stores

Unstructured analytics

• One of the most powerful database engines in the industry for OLTP and EDW

• Full ANSI SQL support • Full ACID transactional support for multi-

row, multi- table, & multi-region updates • Support for nested loop, merge, hash joins • Structured tables, indexes, views • Incremental equal height histograms for

better execution plans • Efficient data flow architecture • Referential Integrity, Triggers, … • Grant/Revoke Security support • UDFs for Complex Event processing • Workload Management • Enterprise class Monitoring & Manageability

NonStop SQL/MX Neoview SeaQuest

Trafodion

Can join Trafodion, HBase, Hive tables in a single statement

• Compound primary keys • Encoding column names for compaction • Salting to spread updates

OLTP and ODS on Hadoop

8 HP PRIVATE © Copyright 2013 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Private Beta program launched

Seeking early adopters for POCs

Available for internal evaluation: Send email to Project.Trafodion@HP.COM for download details

www.hp.com/go/trafodion

© Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.

Thank You