goldengate a live introduction - doag

67
PATRICK HURLEY GOLDENGATE A LIVE INTRODUCTION 1

Upload: others

Post on 10-Jan-2022

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: GOLDENGATE A LIVE INTRODUCTION - DOAG

PATRICK HURLEY

GOLDENGATE

A LIVE INTRODUCTION

1

Page 2: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. |

• Global systems integrator focused on the Oracle platform

• Consultants average 15+ years of Oracle experience

Elite

Expertise

Oracle Specializations*

• Oracle Exadata

• Oracle Exalogic

• Oracle Database

• Oracle GoldenGate

• Oracle Data Integrator

• Oracle Data Warehouse

Thought

Leadership

Success

Our consultants have been published in multiple subject areas

and additional online resources that demonstrate Accenture’s

experience and expertise with the OES platform

Oracle Engineered Systems Numbers

• 700+ Oracle Engineered Systems which AEG have configured, patched or supported.

• 100+ AEG resources which have an average 15+ years of Oracle experience

• AEG Support across 9 countries

• 200 Oracle Engineered Systems (Exadata/Exalogic etc) currently under management directly by AEG

• 200+ customers in either the AEG Managed Services program or remoteDBA program

• 50,000 Accenture Oracle IDC resources that can be leveraged for Level 1 & Level 2 support

*https://www.accenture.com/us-en/service-oracle-diamond-partner

ACCENTURE ENKITEC GROUPCAPABILITIES OVERVIEW

• Oracle Real Application Cluster

• Oracle Performance Tuning

• Oracle Database Security

• Worldwide specialist in Engineered Systems implementations

• 14 Oracle ACE members, recognized by Oracle for their technical expertise

Copyright © 2017 Accenture All rights reserved 2

Page 3: GOLDENGATE A LIVE INTRODUCTION - DOAG

PATRICK HURLEY

GOLDENGATE

A LIVE INTRODUCTION

GoldenGate moves data:

Regular Data Movementbetween operational databasesdata warehouse/BI/reportingdata to/from:

flat filesJava message busesBig Data (Hadoop, Kafka etc.)

Upgrades/migrationszero/minimal downtime (depends on app)

HeterogeneousCloudBi-directional (if migrating can go back to original database)

3

Page 4: GOLDENGATE A LIVE INTRODUCTION - DOAG

DR (should really be Data Guard)High Availability (should really be RAC)

1995: data replication, Tandem computers, banks (cashpoints/ATMs to mainframes)data integrity/zero data lossspeedheterogeneous2009: Oracle

Between many relational databases (Oracle, SQL Server, DB2, Teradata etc.) and other source/targets (Hadoop, Kafka, flat files etc.)

I'm showing how to do Oracle to Oracle

Structure• install• set up replication• key points about this type of replication

3

Page 5: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 4

CHECK FOR UNSUPPORTED FEATURES

select * from dba_goldengate_support_mode;

Classic Extract Scripts (also useful for Integrated Extract):

Individual schema: 1296168.1 (support.oracle.com)

Whole database: 1298562.1 (support.oracle.com)

select * from dba_goldengate_support_mode;

11.2.0.4 onwards

unsupported data types

tables with no primary or unique keys

unsupported data types (caveat: may be supported by Integrated Extract)

4

Page 6: GOLDENGATE A LIVE INTRODUCTION - DOAG

PATCH ORACLE DATABASES

5

Page 7: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 6

PATCH ORACLE DATABASE SOFTWARE HOME

1557031.1 (support.oracle.com)

required and recommended

6

Page 8: GOLDENGATE A LIVE INTRODUCTION - DOAG

DOWNLOAD GOLDENGATE

7

Page 9: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 8

DOWNLOAD GOLDENGATE

oracle.com/technetwork/middleware/goldengate:

123010_fbo_ggs_Linux_x64_shiphome.zip

Older version of GoldenGate for older databases (MOS, Certifications tab)

8

Page 10: GOLDENGATE A LIVE INTRODUCTION - DOAG

INSTALL GOLDENGATE

easiest on same server and os user as databasecan be own server or app server

9

Page 11: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 10

INTERACTIVE INSTALLATION11G DATABASE

$ mkdir /u01/app/oracle/product/ggsycamore

$ unzip 123010_fbo_ggs_Linux_x64_shiphome.zip

$ cd fbo_ggs_Linux_x64_shiphome/Disk1

$ ./runInstaller

install previously unzip, now OUI (in inventory)

10

Page 12: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 11

INTERACTIVE INSTALLATION11G DATABASE

11

Page 13: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 12

INTERACTIVE INSTALLATION11G DATABASE

12

Page 14: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 13

INTERACTIVE INSTALLATION11G DATABASE

13

Page 15: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 14

INTERACTIVE INSTALLATION11G DATABASE

14

Page 16: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 15

INTERACTIVE INSTALLATION11G DATABASE

15

Page 17: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 16

INTERACTIVE INSTALLATION11G DATABASE

$ cat /u01/app/oraInventory/ContentsXML/inventory.xml

<?xml version="1.0" standalone="yes" ?><!-- Copyright (c) 1999, 2017, Oracle and/or its affiliates.All rights reserved. --><!-- Do not modify the contents of this file by hand. --><INVENTORY><VERSION_INFO>

<SAVED_WITH>12.2.0.1.4</SAVED_WITH><MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>

</VERSION_INFO><HOME_LIST><HOME NAME="OraDb11g_home1" LOC="/u01/app/oracle/product/11.2.0/dbhome_1" TYPE="O" IDX="1"/><HOME NAME="OraHome1" LOC="/u01/app/oracle/product/ggsycamore" TYPE="O" IDX="2"/></HOME_LIST><COMPOSITEHOME_LIST></COMPOSITEHOME_LIST></INVENTORY>

16

Page 18: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 17

SILENT INSTALLATION12C DATABASE

$ mkdir /u01/app/oracle/product/ggpoplar

$ unzip 123010_fbo_ggs_Linux_x64_shiphome.zip

$ vi poplar.rsp

oracle.install.responseFileVersion=/oracle/install/rspfmt_ogginstall_response_schema_v12_1_2INSTALL_OPTION=ORA12cSOFTWARE_LOCATION=/u01/app/oracle/product/ggpoplarSTART_MANAGER=trueMANAGER_PORT=7809DATABASE_LOCATION=/u01/app/oracle/product/12.2.0/dbhome_1INVENTORY_LOCATION=/u01/app/oraInventoryUNIX_GROUP_NAME=oinstall

17

Page 19: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 18

SILENT INSTALLATION12C DATABASE

$ fbo_ggs_Linux_x64_shiphome/Disk1/runInstaller -silent -nowait -responseFile ~/poplar.rsp

18

Page 20: GOLDENGATE A LIVE INTRODUCTION - DOAG

PATCH GOLDENGATE

opatch for patches/PSUs

19

Page 21: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 20

PATCH GOLDENGATE

1645495.1 (support.oracle.com)

20

Page 22: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 21

PATCH GOLDENGATE

2193391.1 (support.oracle.com)

21

Page 23: GOLDENGATE A LIVE INTRODUCTION - DOAG

CHECK AND PREPARE DATABASES

22

Page 24: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 23

PREPARE SOURCE DATABASE

SQL> select name, log_mode, supplemental_log_data_min, force_logging from v$database;

NAME LOG_MODE SUPPLEME FOR--------- ------------ -------- ---SKYLARK NOARCHIVELOG NO NO

SQL> shutdown immediate

SQL> startup mount

SQL> alter database archivelog;

SQL> alter database open;

archive log mode – in case of large lag/outage

Production database will usually already be in archive log mode

23

Page 25: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 24

PREPARE SOURCE DATABASE

SQL> alter database add supplemental log data;

SQL> alter database force logging;

SQL> alter system switch logfile;

SQL> select name, log_mode, supplemental_log_data_min, force_logging from v$database;

NAME LOG_MODE SUPPLEME FOR--------- ------------ -------- ---SKYLARK ARCHIVELOG YES YES

SQL> alter system set enable_goldengate_replication=true scope=both;

enable_goldengate_replication parameter available from Oracle 11.2.0.4 onwards

minimal database level logging (e.g. row-chaining information)

enable_goldengate_replication:

• 11.2.0.4 and above

• database services used by GoldenGate

24

Page 26: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 25

PREPARE TARGET DATABASE

SQL> select name, log_mode from v$database;

NAME LOG_MODE--------- ------------PENGUIN NOARCHIVELOG

SQL> shutdown immediate

SQL> startup mount

SQL> alter database archivelog;

SQL> alter database open;

25

Page 27: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 26

PREPARE TARGET DATABASE

SQL> select name, log_mode from v$database;

NAME LOG_MODE--------- ------------PENGUIN ARCHIVELOG

SQL> alter system set enable_goldengate_replication=true scope=both;

26

Page 28: GOLDENGATE A LIVE INTRODUCTION - DOAG

CONFIGURE GOLDENGATE

27

Page 29: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 28

GOLDENGATE

HeterogeneousTarget (or Source) could be CloudNot database Data PumpClassic - heterogeneous; uses database log files (Oracle: redo) and SQLIntegrated - Oracle specific• connected database processes• supports more data types• Extract (11.2.0.3) and Replicat (11.2.0.4)• Streams

• deprecated• streams pool• Extract Capture• Replicat Apply

Integrated/non-integrated: can mix; can switch

28

Page 30: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 29

EXTRACT

redo data: reading redo logs (classic) or logminer (integrated)

trail files

29

Page 31: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 30

CONFIGURE EXTRACT

SQL> create tablespace ggate datafile '/u01/app/oracle/oradata/SKYLARK/ggate01.dbf’size 20m autoextend on;

SQL> create user ggate identified by ggate default tablespace ggate;

SQL> grant create session, connect, resource, alter any table, alter system to ggate;

SQL> grant select any dictionary to ggate;

SQL> exec dbms_goldengate_auth.grant_admin_privilege('ggate','capture');

permissions depend on database version and type of capture:

DBA permission was needed for classic?

‘grant select any dictionary to ggate’ prevents:GoldenGate 12.2: ORA-00942 on startup of extract on sys.tab$GoldenGate 12.3: "OGG-10468 Could not find definition for SCOTT.BONUS. Error:

OCI Err" on add schematrandata scott

dbms_goldengate_auth.grant_admin_privilege('ggate','capture’) - integrated

more privileges needed for DDL support if 11.2.0.3 or earlier

30

Page 32: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 31

CONFIGURE EXTRACT

$ cd /u01/app/oracle/product/ggsycamore

$ ./ggsci

GGSCI (sycamore1) 1> dblogin userid ggate

GGSCI (sycamore1) 2> add schematrandata scott

2017-10-10 12:19:44 INFO OGG-01788 SCHEMATRANDATA has been added on schema "scott".2017-10-10 12:19:44 INFO OGG-01976 SCHEMATRANDATA for scheduling columns has been added on schema "scott".

GGSCI - GoldenGate Software Command Interpreter

supplemental logging of tables:

• whole schema, alternative is add trandata for individual tables

• main purpose is primary/unique key logging, so row can be found on

target database

31

Page 33: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 32

CONFIGURE EXTRACT

GGSCI (sycamore1) 3> edit params exscott

EXTRACT exscottUSERID ggate, PASSWORD ggateEXTTRAIL ./dirdat/aaTABLE scott.*;

GGSCI (sycamore1) 4> add extract exscott, integrated tranlog, begin now

GGSCI (sycamore1) 5> add exttrail ./dirdat/aa, extract exscott

GGSCI (sycamore1) 6> register extract exscott database

8 character names

Methods to avoid clear password (Wallet, credential store)

Trail files:

• default location ./dirdat

• 2 char prefix, 6 digitsRegister - integrated, e.g. if do not do this extract cannot connect to logminer (also classic - RMAN to keep archive logs)

32

Page 34: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 33

CONFIGURE EXTRACT

GGSCI (sycamore1) 7> start extract exscott

GGSCI (sycamore1) 8> info all

Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGEXTRACT RUNNING EXSCOTT 00:00:00 00:01:25

33

Page 35: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 34

DATA PUMP

trail files, local to remote

may not use Data Pump:

• extract to replicat

• less resilient, data loss during an outage

collector on target

34

Page 36: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 35

CONFIGURE DATA PUMP

GGSCI (sycamore1) 9> edit params dpscott

EXTRACT dpscottPASSTHRURMTHOST poplar1.localdomain, mgrport 7809RMTTRAIL ./dirdat/bbTABLE scott.*;

GGSCI (sycamore1) 10> add extract dpscott, exttrailsource ./dirdat/aa

GGSCI (sycamore1) 11> add rmttrail ./dirdat/bb, extract dpscott

passthru - no transformations, so no source database for table definitions

needed

35

Page 37: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 36

CONFIGURE DATA PUMP

GGSCI (sycamore1) 12> start extract dpscott

GGSCI (sycamore1) 13> info all

Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGEXTRACT RUNNING DPSCOTT 00:00:00 00:03:49EXTRACT RUNNING EXSCOTT 00:00:05 00:00:03

36

Page 38: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 37

INSTANTIATE TARGET DATABASE

Instantiation methods:

• GoldenGate initial load - not Oracle specific

• Oracle Database Data Pump

• RMAN (recovery, duplicate from backup or active database)

• Transportable Tablespaces

37

Page 39: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 38

INSTANTIATE TARGET DATABASE

On source database server:

SQL> select dbms_flashback.get_system_change_number from dual;

GET_SYSTEM_CHANGE_NUMBER------------------------

453114

$ vi scott_exp.par

dumpfile=scott.dmplogfile=scott_exp.logdirectory=exportsflashback_scn=453114schemas=scott

$ expdp system parfile=scott_exp.par

$ scp scott.dmp poplar1:

38

Page 40: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 39

INSTANTIATE TARGET DATABASE

On target database server:

$ vi scott_imp.par

dumpfile=scott.dmplogfile=scott_imp.logdirectory=exports

$ impdp system parfile=scott_imp.par

39

Page 41: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 40

REPLICAT

replicat to database: could be via OCI/SQL or connected processes

(integrated)

40

Page 42: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 41

CONFIGURE REPLICAT

SQL> create tablespace ggate datafile '/u01/app/oracle/oradata/PENGUIN/ggate01.dbf’size 20m autoextend on;

SQL> create user ggate identified by ggate default tablespace ggate;

SQL> grant create session, connect, resource, dba to ggate;

SQL> exec dbms_goldengate_auth.grant_admin_privilege('ggate’,’apply');

permissions depend on database version; DBA needed for DDL and sequence

support

41

Page 43: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 42

CONFIGURE REPLICAT

$ cd /u01/app/oracle/product/ggpoplar

$ ./ggsci

GGSCI (poplar1) 1> dblogin userid ggate

GGSCI (poplar1) 2> edit params rpscott

REPLICAT rpscottUSERID ggate, PASSWORD ggateASSUMETARGETDEFSMAP scott.*, TARGET scott.*;

GGSCI (poplar1) 3> add replicat rpscott integrated, exttrail ./dirdat/bb

ASSUMETARGETDEFS - source and target have identical column structure

42

Page 44: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 43

START REPLICAT

GGSCI (poplar1) 4> info all

Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGREPLICAT STOPPED RPSCOTT 00:00:00 00:01:26

GGSCI (poplar1) 5> start replicat rpscott, aftercsn 453114

GGSCI (poplar1) 6> info all

Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGREPLICAT RUNNING RPSCOTT 00:00:07 00:00:09

System Change Number - Commit Sequence Number

Capture starts from when it was registered, so need to start replicat after csn

43

Page 45: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 44

GOLDENGATE

Replication has been configured and is now running from source to target

44

Page 46: GOLDENGATE A LIVE INTRODUCTION - DOAG

MONITOR GOLDENGATE STATUS

45

Page 47: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 46

EXTRACT STATUS

GGSCI (sycamore1) 1> info extract exscott

EXTRACT EXSCOTT Last Started 2017-10-10 12:23 Status RUNNINGCheckpoint Lag 00:00:03 (updated 00:00:03 ago)Process ID 19367Log Read Checkpoint Oracle Integrated Redo Logs

2017-10-10 15:31:23SCN 0.461602 (461602)

46

Page 48: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 47

EXTRACT STATUS

GGSCI (sycamore1) 2> info extract exscott, detail

EXTRACT EXSCOTT Last Started 2017-10-10 12:23 Status RUNNINGCheckpoint Lag 00:00:04 (updated 00:00:03 ago)Process ID 19367Log Read Checkpoint Oracle Integrated Redo Logs

2017-10-10 15:33:23SCN 0.461791 (461791)

Target Extract Trails:Trail Name Seqno RBA Max MB Trail Type./dirdat/aa 0 834639 500 EXTTRAIL

Integrated Extract outbound server first scn: 0.422356 (422356)Integrated Extract outbound server filtering start scn: 0.422356 (422356)

Extract Source Begin EndNot Available 2017-10-10 12:21 2017-10-10 15:33Not Available * Initialized * 2017-10-10 12:21Not Available * Initialized * 2017-10-10 12:21Not Available * Initialized * 2017-10-10 12:21

Current directory /u01/app/oracle/product/ggsycamore

Report file /u01/app/oracle/product/ggsycamore/dirrpt/EXSCOTT.rptParameter file /u01/app/oracle/product/ggsycamore/dirprm/exscott.prmCheckpoint file /u01/app/oracle/product/ggsycamore/dirchk/EXSCOTT.cpeProcess file /u01/app/oracle/product/ggsycamore/dirpcs/EXSCOTT.pceError log /u01/app/oracle/product/ggsycamore/ggserr.log

Relative Byte Address (RBA) - position in the trail file

47

Page 49: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 48

DATA PUMP STATUS

GGSCI (sycamore1) 3> info extract dpscott

EXTRACT DPSCOTT Last Started 2017-10-10 12:43 Status RUNNINGCheckpoint Lag 00:00:00 (updated 00:00:05 ago)Process ID 20520Log Read Checkpoint File /u01/app/oracle/product/ggsycamore/dirdat/aa000000000

2017-10-10 15:36:50.000000 RBA 849812

48

Page 50: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 49

DATA PUMP STATUS

GGSCI (sycamore1) 4> info extract dpscott, detail

EXTRACT DPSCOTT Last Started 2017-10-10 12:43 Status RUNNINGCheckpoint Lag 00:00:00 (updated 00:00:06 ago)Process ID 20520Log Read Checkpoint File /u01/app/oracle/product/ggsycamore/dirdat/aa000000000

2017-10-10 15:38:30.000000 RBA 857065Target Extract Trails:Trail Name Seqno RBA Max MB Trail Type./dirdat/bb 0 857143 500 RMTTRAIL

Extract Source Begin End/u01/app/oracle/product/ggsycamore/dirdat/aa000000000 * Initialized * 2017-10-10 15:38/u01/app/oracle/product/ggsycamore/dirdat/aa000000000 * Initialized * First Record ./dirdat/aa000000000 * Initialized * First Record

Current directory /u01/app/oracle/product/ggsycamore

Report file /u01/app/oracle/product/ggsycamore/dirrpt/DPSCOTT.rptParameter file /u01/app/oracle/product/ggsycamore/dirprm/dpscott.prmCheckpoint file /u01/app/oracle/product/ggsycamore/dirchk/DPSCOTT.cpeProcess file /u01/app/oracle/product/ggsycamore/dirpcs/DPSCOTT.pceError log /u01/app/oracle/product/ggsycamore/ggserr.log

49

Page 51: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 50

TRAIL FILE STATUS

GGSCI (sycamore1) 5> info exttrail *

Extract Trail: ./dirdat/bbSeqno Length: 9

Flip Seqno Length: noExtract: DPSCOTTSeqno: 0RBA: 871868

File Size: 500M

Extract Trail: ./dirdat/aaSeqno Length: 9

Flip Seqno Length: noExtract: EXSCOTTSeqno: 0RBA: 872230

File Size: 500M

50

Page 52: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 51

STATISTICS FOR EXTRACT

GGSCI (sycamore1) 6> stats extract exscott

Sending STATS request to EXTRACT EXSCOTT ...Start of Statistics at 2017-10-10 15:43:53.Output to ./dirdat/aa:Extracting from SCOTT.PRODUCT to SCOTT.PRODUCT:*** Total statistics since 2017-10-10 12:23:57 ***

Total inserts 0.00Total updates 3997.00Total deletes 0.00Total discards 0.00Total operations 3997.00

*** Daily statistics since 2017-10-10 12:23:57 ***Total inserts 0.00Total updates 3997.00Total deletes 0.00Total discards 0.00Total operations 3997.00

*** Hourly statistics since 2017-10-10 15:00:00 ***Total inserts 0.00Total updates 877.00Total deletes 0.00Total discards 0.00Total operations 877.00

*** Latest statistics since 2017-10-10 12:23:57 ***Total inserts 0.00Total updates 3997.00Total deletes 0.00Total discards 0.00Total operations 3997.00

End of Statistics.

51

Page 53: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 52

REPORT FOR EXTRACTGGSCI (sycamore1) 7> view report exscott

***********************************************************************Oracle GoldenGate Capture for Oracle

Version 12.3.0.1.0 OGGCORE_12.3.0.1.0_PLATFORMS_170721.0154_FBOLinux, x64, 64bit (optimized), Oracle 11g on Jul 21 2017 20:34:23

Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.Starting at 2017-10-10 12:22:53

***********************************************************************

Operating System Version:LinuxVersion #2 SMP Fri Oct 21 14:23:10 PDT 2016, Release 4.1.12-61.1.16.el6uek.x86_64Node: sycamore1.localdomainMachine: x86_64

soft limit hard limitAddress Space Size : unlimited unlimitedHeap Size : unlimited unlimitedFile Size : unlimited unlimitedCPU Time : unlimited unlimited

………

************************************************************************* Run Time Messages *************************************************************************2017-10-10 12:23:21 INFO OGG-03522 Setting session time zone to source database time zone 'GMT'.2017-10-10 12:23:56 INFO OGG-06508 Wildcard MAP (TABLE) resolved (entry scott.*): TABLE "SCOTT"."PRODUCT".2017-10-10 12:23:57 INFO OGG-06509 Using the following key columns for source table SCOTT.PRODUCT: PRD_ID.2017-10-10 12:23:57 INFO OGG-06508 Wildcard MAP (TABLE) resolved (entry scott.*): TABLE "SCOTT"."PRODUCT".2017-10-10 12:23:57 INFO OGG-06509 Using the following key columns for source table SCOTT.PRODUCT: PRD_ID.2017-10-10 15:43:53 INFO OGG-01021 Command received from GGSCI: STATS.

52

Page 54: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 53

REPLICAT STATUS

GGSCI (poplar1) 1> info replicat rpscott

REPLICAT RPSCOTT Last Started 2017-10-10 15:23 Status RUNNINGINTEGRATEDCheckpoint Lag 00:00:09 (updated 00:00:10 ago)Process ID 31073Log Read Checkpoint File /u01/app/oracle/product/ggpoplar/dirdat/bb000000000

2017-10-10 16:19:11.680713 RBA 1041343

53

Page 55: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 54

REPLICAT STATUS

GGSCI (poplar1) 2> info replicat rpscott, detail

REPLICAT RPSCOTT Last Started 2017-10-10 15:23 Status RUNNINGINTEGRATEDCheckpoint Lag 00:00:00 (updated 00:00:06 ago)Process ID 31073Log Read Checkpoint File /u01/app/oracle/product/ggpoplar/dirdat/bb000000000

2017-10-10 16:20:49.641431 RBA 1049261INTEGRATED ReplicatDBLOGIN Provided, inbound server name is OGG$RPSCOTT in ATTACHED state

Current Log BSN value: (no data)Low Watermark CSN value: 467372(All source transactions prior to this scn have been applied)High Watermark CSN value: 467374(Some source transactions between this scn and the low watermark may have been applied)

Extract Source Begin End/u01/app/oracle/product/ggpoplar/dirdat/bb000000000 * Initialized * 2017-10-10 16:20/u01/app/oracle/product/ggpoplar/dirdat/bb000000000 * Initialized * First Record./dirdat/bb000000000 * Initialized * First Record

Current directory /u01/app/oracle/product/ggpoplarReport file /u01/app/oracle/product/ggpoplar/dirrpt/RPSCOTT.rptParameter file /u01/app/oracle/product/ggpoplar/dirprm/rpscott.prmCheckpoint file /u01/app/oracle/product/ggpoplar/dirchk/RPSCOTT.cprProcess file /u01/app/oracle/product/ggpoplar/dirpcs/RPSCOTT.pcrError log /u01/app/oracle/product/ggpoplar/ggserr.log

54

Page 56: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 55

REPLICAT LAG

GGSCI (poplar1) 3> lag replicat rpscott

Sending GETLAG request to REPLICAT RPSCOTT ...Last record lag 4 seconds.Low watermark lag: 8.High watermark lag: 5.Low watermark position: 468156.High watermark position: 468158.At EOF, no more records to process

55

Page 57: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 56

STOP AND START AN EXTRACT OR REPLICAT

GGSCI (sycamore1) 8> stop extract exscott

Sending STOP request to EXTRACT EXSCOTT ...Request processed.

GGSCI (sycamore1) 9> info all

Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGEXTRACT RUNNING DPSCOTT 00:00:00 00:00:01EXTRACT STOPPED EXSCOTT 00:00:02 00:00:07

GGSCI (sycamore1) 10> start extract exscott

Sending START request to MANAGER ...EXTRACT EXSCOTT starting

Log file:

/u01/app/oracle/product/ggsycamore/ggserr.log

56

Page 58: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 57

ORACLE DATABASE DICTIONARY VIEWS

Source Database:

SQL> select * from dba_capture;

Target Database:

SQL> select * from dba_apply;

57

Page 59: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 58

HEALTH CHECK FOR INTEGRATED EXTRACT AND REPLICAT

1448324.1 (support.oracle.com)

58

Page 60: GOLDENGATE A LIVE INTRODUCTION - DOAG

LOGICAL REPLICATION

Logical not Physical Replication (cf. Physical Data Guard or storage replication), where do not need to worry about contentsSQL not blocksThat's how it can:• cross database type• cross platform• cross endian• character set conversion• subset of database/tables• subset of rows: filtering on source (capture) or target (apply)• transform

59

Page 61: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 60

ALTERNATIVE LOGICAL REPLICATION PRODUCTS

also streams

60

Page 62: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 61

LOGICAL REPLICATION

Some data types are not supported

Must be able to uniquely identify a row

Set based operations become row-by-row operations

blog.dbvisit.com/3-fundamentals-of-oracle-replication-part-1-of-3

Uniquely identify:

• PK/UK, otherwise concatenates all columns - may or may not guarantee

uniqueness, or KEYCOLS

• hence extra supplemental logging - PK/UK data into the redo logs

Set based row-by-row:

• Redo: block changes, not SQL

61

Page 63: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 62

SET BASED OPERATIONS BECOME ROW-BY-ROW OPERATIONS

Execute a multiple row update statement on source database:

SQL> update scott.emp set hiredate = sysdate;

14 rows updated.

Statement uses table access full:---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | UPDATE STATEMENT | | | | 3 (100)| || 1 | UPDATE | EMP | | | | || 2 | TABLE ACCESS FULL| EMP | 14 | 168 | 3 (0)| 00:00:01 |---------------------------------------------------------------------------

62

Page 64: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 63

SET BASED OPERATIONS BECOME ROW-BY-ROW OPERATIONS

On target database GoldenGate Replicat executes statement below, many times:

UPDATE /*+ restrict_all_ref_cons */ "SCOTT"."EMP" SET "HIREDATE"=:1 WHERE "EMPNO"=:2

Statement uses index access:------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------| 0 | UPDATE STATEMENT | | | | 1 (100)| | 1 | UPDATE | EMP | | | | |* 2 | INDEX UNIQUE SCAN| PK_EMP | 1 | 12 | 0 (0)| ------------------------------------------------------------------

Predicate Information (identified by operation id): ---------------------------------------------------

2 - access("EMPNO"=:2)

63

Page 65: GOLDENGATE A LIVE INTRODUCTION - DOAG

PATRICK HURLEY

GOLDENGATE

A LIVE INTRODUCTION

Covered• install• set up replication• had to discard more content than I have included:

• filtering (table or row)• handling replication errors• parallelism (performance)

• key points about logical replication

Whet appetite, get started

64

Page 66: GOLDENGATE A LIVE INTRODUCTION - DOAG

Copyright © 2017 Accenture All rights reserved. | 65

ORACLE GOLDENGATE 12C IMPLEMENTER’S GUIDE

65

Page 67: GOLDENGATE A LIVE INTRODUCTION - DOAG

bit.ly/ggateintro

@phurley

[email protected]

66