overcoming the 5 biggest challenges in data mart consolidation

1 April 8, 2023© Kalido I Kalido Confidential I April 8, 2023

Overcoming the 5 Biggest Challenges in Data Mart ConsolidationKalido WebcastJanuary 31, 2012


Logistics

Attendees will be on mute for the call

Type your questions into the Questions box

Webcast is being recorded and will be available for replay

Request a copy of today’s slides by sending an email to: [email protected]

Join the conversation online by using the #Kalido hashtag. Follow us @kalido !

mailto:[email protected]


Today’s Speakers

John EvansDirector of Product Marketing, Kalido

Patrick MullinsMaster Principal Sales Consultant, Oracle

Lovan ChettySenior Manager, Product Management, Kalido


Discussion Topics

Data Mart Consolidation Issues

The 5 Challenges

How Kalido and Oracle Exadata Enable Data Mart Consolidation

Q&A


Why Data Mart Migration & Consolidation?

Data marts are expensive and spread across the organization– 59% of companies maintain up to 30 data marts– $1.5 – $2 million annually to maintain a single mart– 35% – 70% of those costs are redundant

Customers can improve information consistency, create more complete analytics and save money

CIOs should be aware that data marts will emerge continuously in the organization. They should advise the business intelligence and data warehouse teams to plan for ongoing data mart consolidation and demand that a strategy for accomplishing it is in place.

-- Gartner, Data Warehousing Trends for the CIO, 2011-2012


Traditional Warehousing Takes Too Long

Business Event

Time to Deliver

Make Decision

Requirements & Analysis

Testing

Release to Production

Modeling & Design

Data Integration

BusinessValue

Data Access/BI

Traditi

onal

Traditional


Shorten the Cycle, Maximize Business Value

Business Event

Time to Deliver

Make Decision


Testing

Modeling & Design

Data Integration

Traditi

onal

Kalid

o

Time to ValueBenefit

BusinessValue

Business ValueBenefit

Data Access/BI


Kalido

Traditional


Mart Consolidation Options: Lift and Shift

Migrate existing marts onto a high performance platform

Improve query speed and end-user response time

Manage increasing volumes of existing data

Same poorly-constructed and

inflexible marts, just running on a faster

platform


Mart Consolidation Options: Integrate into EDW


Improve query speed and end-user response time

Manage increasing volumes of existing data

Use traditional tools and approach to merge marts

Significant time to analyze and re-architect

Diversity of marts can lengthen time to build

No value delivered for months or longer

Same poorly-constructed and

inflexible marts, just running on a faster

platform


Ideal Approach to Mart Consolidation


Use agile tools and approach to merge marts

Exploit existing assets to accelerate reverse engineering

Consistently deliver business value as you build the warehouse,

retiring marts as you go, on an agile foundation

for future growth

Focus on solving business problems, not overcoming technical hurdles

Improve query speed, end-user response and load time

Manage increasing volumes of existing and new data


Top 5 Challenges

Reusing existing mart assets and refactoring the model

Untangling all the data integration connections

Data duplication between and within marts

Referential integrity

Controlling costs and preparing for change

The number of data migration and conversion projects is on the increase as organizations focus on IT modernization, cost optimization and merger/acquisition initiatives.

-- Gartner, Risks and Challenges in Data Migrations and Conversions


New Modeling & Design Improvements

Exploits your current logical and physical models and taxonomies to build a new more agile data warehouse

Enables reverse engineering

Converts technical names and labels to business-friendly terms

– Leverages existing business glossary and abbreviations document


Demo


What You Will See

Read a physical model from an existing data mart

Refactor the model into a business information model

Deploy the model

Initial population of the model from existing data mart


Kalido Enables Delivery In 90 Days or Less

OperationsProcess AutomationTask Execution & Monitoring

Deployment and Migration Archiving

Restore for Model and Data Undo Loads

Audit and Logging Job Definition with Dependency

BI Delivery

Metadata Management for BOBJ

Native XLS Pivot Table Generation

Native QlikView Generation

Metadata Management for MSAS

Metadata Management for COGN

Report-Time Formula Management

Testing

Built-in Integrity Checking

Aggregate Task Results

Excel Integration for User reconciliation

Data rollback and batch reload for system test

User Interface for data browsing & troubleshooting


Version Management

Object level Change Management

Model Migration

Generic Export/ Import for Data Migration

Object Level Dependencyfor Migration Versions

Model Comparison Report

Business Information Model Driven Automation

Data Integration

Data Validation

Suspense and Exception Handling

Data Sourcing and Field Mapping

Delta Detection

System Key Management

Code Management and Lookup

Currency and Units of Measure

Contra Processing

Data Export & Purging Post Processing Housekeeping

Modeling

Star and Snowflake Schema

Physical Schema Management

Slowly Changing Dimensions

Data Mart and Aggregates

Data Load and Index Management

Rollup Path Awareness

Incremental Summary Generation

Convert Existing Logical Models

Name & Label Management


Oracle Exadata Support

Combines Kalido’s business-driven automation with Exadata’s extreme data warehousing performance

Tuned and optimized out-of-the-box

Record performance for Kalido on any platform to date

– 4 to 50 times faster vs. traditional relational databases

– Significantly faster than other high-performance platforms

Unparalleled time-to-value for any data warehouse project

17 | © 2011 Oracle Corporation |

Oracle Exadata Database Machine

One architecture for…

• Data Warehousing

• OLTP

• Database Consolidation

Exadata is Oracle’s strategic database platform for ALL Oracle Database

workloads


Exadata ArchitectureA complete system: compute, storage, networking

• Database Cluster– Intel-based database servers– Oracle Linux or Solaris 11 – Oracle Database 11g– 10 Gig Ethernet (to data center)

• Storage Grid– Intel-based storage servers– Up to 504 terabytes raw disk– 5.3 terabytes Flash storage– Exadata Storage Server Software

• InfiniBand Network– Internal connectivity ( 40 Gb/sec )


Exadata Innovations

• Intelligent storage– Scale-out InfiniBand storage

– Smart Scan query offload

+ ++

• Hybrid Columnar Compression– 10x compression for warehouses– 15x compression for archives

• Smart PCI Flash Cache– Accelerates random I/O up to 30x– Triples data scan rate

Data remains compressed

for scans and in Flash

Benefits Cascade to Copies

compress

primary DB

standbytest

devbackup

uncompressed


SQL Query Throughput

IBM XIV NetApp 6080

IBM DS8700 Hitachi USP V

EMC VMAX Exadata Disk Exadata Flash

2.56

911 *

25

75Query ThroughputGigabytes per Second

* Undisclosed by vendor

50,000 IOPS

1,500,000IOPS (I/Os per second)


Exadata Delivers Extreme Consolidation

• Large Memory – Many databases can be consolidated

• Extreme Performance– OLTP, DW, data mining, batch, reporting, loading, backups,

files in the database

– Encryption, compression

• Workload Management– Manage SLAs via Quality of Service (QoS)

– CPU and I/O resource management

– Instance caging

Shrink data center costs, increase system utilization and promote

application integration


Pre-built and Optimized Out-of-the-BoxP

erf

orm

an

ce

Ac

hie

ve

me

nt

Pe

rfo

rma

nc

e A

ch

iev

em

en

t

Time (Days)

Time (Months)

100%

Measure, diagnose, tune and

reconfigure

Test & debug failure modes

Assemble dozens of

components

Multi-vendor finger

pointing

Custom Configuration


The Exadata Difference

Exadata DB Machine Custom Configuration

Storage scans & filters data Storage just ships blocks

Storage offloads DB* DB-unaware storage

Flash caches relevant data No DB-aware flash management

40 Gb/sec network 8 – 10 Gb/sec network

Pre-built for DB workload Assembled by customer

Redundancy built-in Build-your-own HA

Compression built-in Compression optional

Workload mgmt built-in Workload mgmt optional

* Backups, compression, decryption, data mining

Exadata is not a general-purpose system, it’s a Database Machine

168 CPU

cores in storage


Customer Overview

Mid-sized health insurance payer

Faced with rising health plan costs

Reduce costs through improved analysis

Dramatically increase both the scale and performance

Extensibility, seamless migration and compatibility with existing 11g-based data warehouse

More effective data management related to members, providers and claims


Top 5 Challenges Met

Reusing existing mart assets and refactoring the model

Untangling all the data integration connections

Data duplication between and within marts

Referential integrity

Controlling costs and preparing for change


Shorten the Cycle, Maximize Business Value

Business Event

Time to Deliver

Make Decision


Testing

Modeling & Design

Data Integration

Traditi

onal

Kalid

o

Time to ValueBenefit

BusinessValue

Business ValueBenefit

Data Access/BI


Kalido

Traditional


Enable Faster and Easier Data Mart Migration

Source: customer benchmark

Tim

e &

Mo

ney

Traditional Data Warehouse Approach

KalidoTime To Value Zone


Key Benefits from Mart Migration/Consolidation

Benefits for IT Users– Better control and governance over analytics across the

organization – no “shadow IT”– Accelerates data mart consolidation– Better responsiveness to business needs– Reduces TCO

Benefits for Business Users– Improved business decisions through enhanced

consistency of information– Significantly improved ability for the business to respond to

change– Accelerates the drive to become an agile business


Q&A


Next Steps

Attendees will receive our whitepaper on “The Next Generation of Data Integration for Data Warehousing”

Learn more about Kalido on Exadata at Independent Health – tune in to webcast on February 7 at 11am Easternhttp://info.kalido.com/healthcare_webinar.html

Download Kalido Business Information Modeler http://www.kalido.com/business-modeling-community.htm

Read our blog about Kalido Information Engine http://blog.kalido.com/category/information-engine/

Contact us! +1.781.202.3200, press 1

http://info.kalido.com/healthcare_webinar.html

http://www.kalido.com/business-modeling-community.htm

http://www.kalido.com/business-modeling-community.htm

http://blog.kalido.com/category/information-engine/


Thank you for attending!

overcoming the 5 biggest challenges in data mart consolidation

Technology

improve query speed

mart consolidation options

high performance platform

data marts

benefittraditional time

benefit time

2011 oracle corporation

model untangling