change data capture - migration data replication

© 2013 IBM Corporation

Change Data Capture - MigrationData Replication

Georg Sehrt – IBM InfoSphere Data Replication Change Data Capture CDC

[email protected]

© 2013 IBM Corporation2

FinancialServices

Government

Retail

Telecom-munications

Real-time integration requirements are everywhere:

Multi-channel sales Real-time inventory checkingGift registry updates

Benefits eligibility verificationSecurity threat detection

On-line bankingFraud detection

First call resolution Cross-sell/up-sell

Customer retention


customers

>

IBM InfoSphere Data Replication Has


Zero Downtime Server Migrations

Perform system upgrades while testing new applications with real-time production data

Data Synchronization and Replication

Up-to-date information flowing between applications to ensure most current version is always available

Provide continuous data protection and availability of systems during planned and unplanned downtime

High Availability and Disaster Recovery

Event-Driven SOA and Real Time Analytics

Detect and react to business events as they occur and adapt to change faster through real-time analytics

IBM InfoSphere Data Replication


� Consolidate more servers per core

� Run more applications/software at less expense

� Manage more virtual servers with fewer people

� Deploy new servers and applications faster

� Absorb workload spikes more easily

� Spend less on disaster recovery

� Occupy less floor space

� Save on energy

z/VM® z/VM z/VM

Linux

Servers

Linux

Servers

Linux

Servers

Shared EverythingInfrastructure

(CPU, Memory, Network,

Adapters, Crypto, Devices)

Helping you“Do More with Less”

IBM Enterprise Linux ServerSaving Money and Reducing Complexity

A refrigerator size box versus vs. a full room of servers.

The differences are quantum.


Case : A Large Telco – Application Integration

AmdocsBilling

Oracle Re-Do Log

Transaction 1

Transaction 2

Transaction 3

Transaction 4

Transaction 5

Transaction 6

Transaction 7

Transaction 8

Transaction 9

Transaction 10

Scrape

Customer

Address

Billing

HP Server

Oracle 9.2.05

Customer, Address, Memo, Service, Billing

etc…

Transaction 1

Transaction 2

Transaction 3

Transaction 4

Transaction 1

Transaction 1

Transaction 7

Transaction 8

Queue 1

Transaction 10

Queue 2

Queue 3

TIBCO ESB

XML Document

XML Document

XML Document

SiebelCRM

Oracle 11

Other Application

Other Application

* Real time visibility of Customer Billing Informat ion in CRM system (Service Requirement)* Minimal impact on operational system * Flexible to changing data requirements* Minimum Latency (1100 transactions per second wit h no latency)* Scalability

Business / Technical Drives

db size = 11 terabytes

40 million trans/day

Redo generation peak at 1 GB/min

TIBCO API

InfoSphereCDC

InfoSphereCDC

Oracle 11BI

ELS


Use Case – A Large Telco

� Introduce innovative concepts to win new business a nd retain customer base without overloading mission-critical systems

� Extremely large billing system, no extra bandwidth for additional processing

– Database already running 10000 queries per second– Tables contain up to 500 million rows

� Monitor customer account data in real-time to enabl e proactive action, additional revenue opportunities and unique custome r service

� IBM InfoSphere Change Data Capture detects changes in the billing system in real-time and sends a message to their CRM system.

�Prepaid customers notified when running low on minutes

•Prevents customers from losing service

�Provide immediate service to new customers•Newly activated phones can be used right away•Can start billing new customers instantly

� Peak performance of billing system through real-time Change Data Capture

� By tying the CRM to the billing system and using Event Detection, this company improves its business agility and productivity

� Event detection to increase revenue opportunities, improve service, mitigate risk to enable prompt response to potential compliance and fraud issues

� Quick and easy activation of new customers to boost sales

� Increased customer satisfaction

Challenge

Solution

Benefits

“Better business awareness means better business decisions, better customer service and better sales. IBM InfoSphere Change Data Capture gives us real-time insight into our operations, helping us to attract new business and maintain our leadership position.”


IBM InfoSphere Data Replication

Low Impact Real-Time Trusted

Data Delivery for the Enterprise

Plattform / Database Migration


�High performance/functionality

�No Programming – Configuration via GUI

� Information from native log

�Non intrusive to applications and databases

Smarter


Simpler

� Best of breed UI

� Built in monitoring

� Management easily integrated into existing IT operations

� Mobile ready


InfoSphere Change Data Capture

Journal LogRedo/Archive Logs

Source EngineAnd Metadata

Target EngineAnd Metadata

TCP/IP

Java-based GUIfor admin & monitoring

Database(Oracle, DB2, SQL Server,Teradata, etc.)

ETL (DataStage, others)

JMS (MQ, others)

Web Services

Targets

Flat files

Information Server

(DataStage, QualityStage, etc.)

Sources

Oracle

SQL Server

Informix

Sybase

DB2

� Key Benefits:– Low impact– Flexible implementation– Heterogeneous platform support

– Easy to use– Additional opportunity to leverage existing ETL and Data Quality

investments through InfoSphere DataStage and QualityStage


InfoSphere

CDC

Implementation Topologies

Uni-Directional Cascade

2-Way Multi-Thread

Bi-Directional Local

Remote Capture

DistributionConsolidation


IBM i

IBM System z

IBM System p

HP PA-RISC

Intel / AMD (x86/x64)

Sun SPARC

DB2 i

Oracle

DB2 z/OS

DB2 LUW

Information Server IBM i OS

z/OS

AIX

HP-UX

Solaris

MS Windows

Red Hat, SUSE Linux

DATABASESSource & Target

TARGETS OPERATINGSYSTEMS

HARDWAREPLATFORMS

MESSAGE QUEUE

JMS

MQ Series

WebMethods

BEA

TIBCO

Netezza

HP ItaniumMS SQL Server

DB2 UDBSybase

Teradata *

Cognos Now!

Informix

SolidDB

Greenplum**

MySQL**

Classic z

Intel Itanium


Remote configuration with Oracle source

CDC Server

Source Server Target Server

Scraper

redolog

Metadata

ApplySource TablesMetadata

Target Tables

1. Scraper on source, Apply on target


Scraper

redolog

Metadata

Apply

Source TablesMetadata

Target Tables

2. Scraper and Apply on target *


Scraper

redolog

Metadata

Apply

Source TablesMetadata

Target Tables

3. Scraper and Apply on CDC server *


Scraper

redolog

Metadata

ApplySource TablesMetadata

Target Tables

4. Scraper and Apply on source

* For configurations 2 & 3, optimal using shared SA N for logs


Oracle Configuration

� dmconfigurets interface supports advanced Oracle configuration settings

� Tool provides early detection of basic accessibility and connectivity problems

� Configuration of system parameters post instance creation not required


Monitoring - Refresh Status

� Summary view of the percentage of the tables that a re refreshed and ability to drill down for detailed view of refresh status


Monitoring - Performance View

� Provides subscription and table-level monitoring

� Exposes a large collection of statistics grouped by component

� Provides graphical and table presentation

� Trend analysis

� Supports data export

� Provides busy table identification


� Consolidate more servers per core

� Run more applications/software at less expense

� Manage more virtual servers with fewer people

� Deploy new servers and applications faster

� Absorb workload spikes more easily

� Spend less on disaster recovery

� Occupy less floor space

� Save on energy

z/VM® z/VM z/VM

Linux

Servers

Linux

Servers

Linux

Servers

Shared EverythingInfrastructure

(CPU, Memory, Network,

Adapters, Crypto, Devices)

Helping you“Do More with Less”

IBM Enterprise Linux ServerSaving Money and Reducing Complexity

A refrigerator size box versus vs. a full room of servers.

The differences are quantum.


Examples of Oracle Solutions deployed on IBM zEnterprise System

Oracle solution landscape on a single zEnterprise Sy stem

� Run Oracle database and Oracle E-Business Suite on highly available and reliable zEnterprise 196 (z196) or zEnterprise z114 (z114)

� Run certified Oracle solutions including PeopleSoft and Siebel application and web tiers on AIX on zEnterprise BladeCenter Extension (zBX)

� Fully benefit from zEnterprise, the first hybrid sy stem, and manage all your Oracle solution servers with the zEnterprise Unified Resource Manag er

EC12/ z196 / z114

Select IBM Blades

Blade HW Resources

Optimizers

IBM

Sm

art

Ana

lytic

s O

ptim

izer

z HW Resources

z/OS

ELS (zeroDowntime HW )

Linux on System x

AIX on POWER7 D

ataP

ower

Fut

ure

Offe

ring

Fut

ure

Offe

ring

Blade Virtualization Blade Virtualization

System z PR/SM

z/TPFz/VSE

Linuxon

System z

Certified Oracle

Solutions

z/VM

IBM

DB2 V10

zBX / pureSystem

Oracle E-Business

Suite

Oracle DB11g R2 and

Fusion Middleware

Linuxon

System z


zEnterprise Linux Server hybrid

ELS Smarter Solution für Ihr Anwendungsdesign

Sys

tem

z H

ardw

are

Man

agem

ent C

onso

le (

HM

C)

mit

Uni

fied

Res

ourc

e M

anag

er

zBX

Select IBM Blades

Blade HW Resources

Optimizers

IBM

Sm

art A

naly

tics

Opt

imiz

er

z HW Resources

Support Element

FedoraaufELS

z/VM

Private High Speed Data Network IEDN

Private Management Network INMNUnified Resource Manager

Private data network (IEDN)

1 All statements regarding IBM future direction and intent are subject to change or withdrawal without notice, and represents goals and objectives only.

Customer Network Customer Network

Enterprise Linux Server

Linux auf Intel 1

AIX auf POWER7

Dat

aPow

er1

Fut

ure

Offe

ring

Fut

ure

Offe

ring

Blade Virtualization Blade Virtualization

System z PR/SM

Linux auf ELS

21

HMC

Debian auf ELS

z/VM

z/VM

SuS

E

Red hat

Windows auf Intel

© 2013 IBM Corporation22

Acknowledgements and Disclaimers

Availability : References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

© Copyright IBM Corporation 2013. All rights reserved.U.S. Government Users Restricted Rights –Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM, the IBM logo, ibm.com, InfoSphere, InfoSphere Business Glossary, InfoSphere Data Architect, InfoSphere DataStage, InfoSphere FastTrack, InfoSphere Information Analyzer, InfoSphere Information Services Director, InfoSphere Metadata Workbench, and InfoSphere QualityStage are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

Other company, product, or service names may be trademarks or service marks of others.

change data capture - migration data replication

Documents