goldengate a live introduction - doag
TRANSCRIPT
PATRICK HURLEY
GOLDENGATE
A LIVE INTRODUCTION
1
Copyright © 2017 Accenture All rights reserved. |
• Global systems integrator focused on the Oracle platform
• Consultants average 15+ years of Oracle experience
Elite
Expertise
Oracle Specializations*
• Oracle Exadata
• Oracle Exalogic
• Oracle Database
• Oracle GoldenGate
• Oracle Data Integrator
• Oracle Data Warehouse
Thought
Leadership
Success
Our consultants have been published in multiple subject areas
and additional online resources that demonstrate Accenture’s
experience and expertise with the OES platform
Oracle Engineered Systems Numbers
• 700+ Oracle Engineered Systems which AEG have configured, patched or supported.
• 100+ AEG resources which have an average 15+ years of Oracle experience
• AEG Support across 9 countries
• 200 Oracle Engineered Systems (Exadata/Exalogic etc) currently under management directly by AEG
• 200+ customers in either the AEG Managed Services program or remoteDBA program
• 50,000 Accenture Oracle IDC resources that can be leveraged for Level 1 & Level 2 support
*https://www.accenture.com/us-en/service-oracle-diamond-partner
ACCENTURE ENKITEC GROUPCAPABILITIES OVERVIEW
• Oracle Real Application Cluster
• Oracle Performance Tuning
• Oracle Database Security
• Worldwide specialist in Engineered Systems implementations
• 14 Oracle ACE members, recognized by Oracle for their technical expertise
Copyright © 2017 Accenture All rights reserved 2
PATRICK HURLEY
GOLDENGATE
A LIVE INTRODUCTION
GoldenGate moves data:
Regular Data Movementbetween operational databasesdata warehouse/BI/reportingdata to/from:
flat filesJava message busesBig Data (Hadoop, Kafka etc.)
Upgrades/migrationszero/minimal downtime (depends on app)
HeterogeneousCloudBi-directional (if migrating can go back to original database)
3
DR (should really be Data Guard)High Availability (should really be RAC)
1995: data replication, Tandem computers, banks (cashpoints/ATMs to mainframes)data integrity/zero data lossspeedheterogeneous2009: Oracle
Between many relational databases (Oracle, SQL Server, DB2, Teradata etc.) and other source/targets (Hadoop, Kafka, flat files etc.)
I'm showing how to do Oracle to Oracle
Structure• install• set up replication• key points about this type of replication
3
Copyright © 2017 Accenture All rights reserved. | 4
CHECK FOR UNSUPPORTED FEATURES
select * from dba_goldengate_support_mode;
Classic Extract Scripts (also useful for Integrated Extract):
Individual schema: 1296168.1 (support.oracle.com)
Whole database: 1298562.1 (support.oracle.com)
select * from dba_goldengate_support_mode;
11.2.0.4 onwards
unsupported data types
tables with no primary or unique keys
unsupported data types (caveat: may be supported by Integrated Extract)
4
PATCH ORACLE DATABASES
5
Copyright © 2017 Accenture All rights reserved. | 6
PATCH ORACLE DATABASE SOFTWARE HOME
1557031.1 (support.oracle.com)
required and recommended
6
DOWNLOAD GOLDENGATE
7
Copyright © 2017 Accenture All rights reserved. | 8
DOWNLOAD GOLDENGATE
oracle.com/technetwork/middleware/goldengate:
123010_fbo_ggs_Linux_x64_shiphome.zip
Older version of GoldenGate for older databases (MOS, Certifications tab)
8
INSTALL GOLDENGATE
easiest on same server and os user as databasecan be own server or app server
9
Copyright © 2017 Accenture All rights reserved. | 10
INTERACTIVE INSTALLATION11G DATABASE
$ mkdir /u01/app/oracle/product/ggsycamore
$ unzip 123010_fbo_ggs_Linux_x64_shiphome.zip
$ cd fbo_ggs_Linux_x64_shiphome/Disk1
$ ./runInstaller
install previously unzip, now OUI (in inventory)
10
Copyright © 2017 Accenture All rights reserved. | 11
INTERACTIVE INSTALLATION11G DATABASE
11
Copyright © 2017 Accenture All rights reserved. | 12
INTERACTIVE INSTALLATION11G DATABASE
12
Copyright © 2017 Accenture All rights reserved. | 13
INTERACTIVE INSTALLATION11G DATABASE
13
Copyright © 2017 Accenture All rights reserved. | 14
INTERACTIVE INSTALLATION11G DATABASE
14
Copyright © 2017 Accenture All rights reserved. | 15
INTERACTIVE INSTALLATION11G DATABASE
15
Copyright © 2017 Accenture All rights reserved. | 16
INTERACTIVE INSTALLATION11G DATABASE
$ cat /u01/app/oraInventory/ContentsXML/inventory.xml
<?xml version="1.0" standalone="yes" ?><!-- Copyright (c) 1999, 2017, Oracle and/or its affiliates.All rights reserved. --><!-- Do not modify the contents of this file by hand. --><INVENTORY><VERSION_INFO>
<SAVED_WITH>12.2.0.1.4</SAVED_WITH><MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
</VERSION_INFO><HOME_LIST><HOME NAME="OraDb11g_home1" LOC="/u01/app/oracle/product/11.2.0/dbhome_1" TYPE="O" IDX="1"/><HOME NAME="OraHome1" LOC="/u01/app/oracle/product/ggsycamore" TYPE="O" IDX="2"/></HOME_LIST><COMPOSITEHOME_LIST></COMPOSITEHOME_LIST></INVENTORY>
16
Copyright © 2017 Accenture All rights reserved. | 17
SILENT INSTALLATION12C DATABASE
$ mkdir /u01/app/oracle/product/ggpoplar
$ unzip 123010_fbo_ggs_Linux_x64_shiphome.zip
$ vi poplar.rsp
oracle.install.responseFileVersion=/oracle/install/rspfmt_ogginstall_response_schema_v12_1_2INSTALL_OPTION=ORA12cSOFTWARE_LOCATION=/u01/app/oracle/product/ggpoplarSTART_MANAGER=trueMANAGER_PORT=7809DATABASE_LOCATION=/u01/app/oracle/product/12.2.0/dbhome_1INVENTORY_LOCATION=/u01/app/oraInventoryUNIX_GROUP_NAME=oinstall
17
Copyright © 2017 Accenture All rights reserved. | 18
SILENT INSTALLATION12C DATABASE
$ fbo_ggs_Linux_x64_shiphome/Disk1/runInstaller -silent -nowait -responseFile ~/poplar.rsp
18
PATCH GOLDENGATE
opatch for patches/PSUs
19
Copyright © 2017 Accenture All rights reserved. | 20
PATCH GOLDENGATE
1645495.1 (support.oracle.com)
20
Copyright © 2017 Accenture All rights reserved. | 21
PATCH GOLDENGATE
2193391.1 (support.oracle.com)
21
CHECK AND PREPARE DATABASES
22
Copyright © 2017 Accenture All rights reserved. | 23
PREPARE SOURCE DATABASE
SQL> select name, log_mode, supplemental_log_data_min, force_logging from v$database;
NAME LOG_MODE SUPPLEME FOR--------- ------------ -------- ---SKYLARK NOARCHIVELOG NO NO
SQL> shutdown immediate
SQL> startup mount
SQL> alter database archivelog;
SQL> alter database open;
archive log mode – in case of large lag/outage
Production database will usually already be in archive log mode
23
Copyright © 2017 Accenture All rights reserved. | 24
PREPARE SOURCE DATABASE
SQL> alter database add supplemental log data;
SQL> alter database force logging;
SQL> alter system switch logfile;
SQL> select name, log_mode, supplemental_log_data_min, force_logging from v$database;
NAME LOG_MODE SUPPLEME FOR--------- ------------ -------- ---SKYLARK ARCHIVELOG YES YES
SQL> alter system set enable_goldengate_replication=true scope=both;
enable_goldengate_replication parameter available from Oracle 11.2.0.4 onwards
minimal database level logging (e.g. row-chaining information)
enable_goldengate_replication:
• 11.2.0.4 and above
• database services used by GoldenGate
24
Copyright © 2017 Accenture All rights reserved. | 25
PREPARE TARGET DATABASE
SQL> select name, log_mode from v$database;
NAME LOG_MODE--------- ------------PENGUIN NOARCHIVELOG
SQL> shutdown immediate
SQL> startup mount
SQL> alter database archivelog;
SQL> alter database open;
25
Copyright © 2017 Accenture All rights reserved. | 26
PREPARE TARGET DATABASE
SQL> select name, log_mode from v$database;
NAME LOG_MODE--------- ------------PENGUIN ARCHIVELOG
SQL> alter system set enable_goldengate_replication=true scope=both;
26
CONFIGURE GOLDENGATE
27
Copyright © 2017 Accenture All rights reserved. | 28
GOLDENGATE
HeterogeneousTarget (or Source) could be CloudNot database Data PumpClassic - heterogeneous; uses database log files (Oracle: redo) and SQLIntegrated - Oracle specific• connected database processes• supports more data types• Extract (11.2.0.3) and Replicat (11.2.0.4)• Streams
• deprecated• streams pool• Extract Capture• Replicat Apply
Integrated/non-integrated: can mix; can switch
28
Copyright © 2017 Accenture All rights reserved. | 29
EXTRACT
redo data: reading redo logs (classic) or logminer (integrated)
trail files
29
Copyright © 2017 Accenture All rights reserved. | 30
CONFIGURE EXTRACT
SQL> create tablespace ggate datafile '/u01/app/oracle/oradata/SKYLARK/ggate01.dbf’size 20m autoextend on;
SQL> create user ggate identified by ggate default tablespace ggate;
SQL> grant create session, connect, resource, alter any table, alter system to ggate;
SQL> grant select any dictionary to ggate;
SQL> exec dbms_goldengate_auth.grant_admin_privilege('ggate','capture');
permissions depend on database version and type of capture:
DBA permission was needed for classic?
‘grant select any dictionary to ggate’ prevents:GoldenGate 12.2: ORA-00942 on startup of extract on sys.tab$GoldenGate 12.3: "OGG-10468 Could not find definition for SCOTT.BONUS. Error:
OCI Err" on add schematrandata scott
dbms_goldengate_auth.grant_admin_privilege('ggate','capture’) - integrated
more privileges needed for DDL support if 11.2.0.3 or earlier
30
Copyright © 2017 Accenture All rights reserved. | 31
CONFIGURE EXTRACT
$ cd /u01/app/oracle/product/ggsycamore
$ ./ggsci
GGSCI (sycamore1) 1> dblogin userid ggate
GGSCI (sycamore1) 2> add schematrandata scott
2017-10-10 12:19:44 INFO OGG-01788 SCHEMATRANDATA has been added on schema "scott".2017-10-10 12:19:44 INFO OGG-01976 SCHEMATRANDATA for scheduling columns has been added on schema "scott".
GGSCI - GoldenGate Software Command Interpreter
supplemental logging of tables:
• whole schema, alternative is add trandata for individual tables
• main purpose is primary/unique key logging, so row can be found on
target database
31
Copyright © 2017 Accenture All rights reserved. | 32
CONFIGURE EXTRACT
GGSCI (sycamore1) 3> edit params exscott
EXTRACT exscottUSERID ggate, PASSWORD ggateEXTTRAIL ./dirdat/aaTABLE scott.*;
GGSCI (sycamore1) 4> add extract exscott, integrated tranlog, begin now
GGSCI (sycamore1) 5> add exttrail ./dirdat/aa, extract exscott
GGSCI (sycamore1) 6> register extract exscott database
8 character names
Methods to avoid clear password (Wallet, credential store)
Trail files:
• default location ./dirdat
• 2 char prefix, 6 digitsRegister - integrated, e.g. if do not do this extract cannot connect to logminer (also classic - RMAN to keep archive logs)
32
Copyright © 2017 Accenture All rights reserved. | 33
CONFIGURE EXTRACT
GGSCI (sycamore1) 7> start extract exscott
GGSCI (sycamore1) 8> info all
Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGEXTRACT RUNNING EXSCOTT 00:00:00 00:01:25
33
Copyright © 2017 Accenture All rights reserved. | 34
DATA PUMP
trail files, local to remote
may not use Data Pump:
• extract to replicat
• less resilient, data loss during an outage
collector on target
34
Copyright © 2017 Accenture All rights reserved. | 35
CONFIGURE DATA PUMP
GGSCI (sycamore1) 9> edit params dpscott
EXTRACT dpscottPASSTHRURMTHOST poplar1.localdomain, mgrport 7809RMTTRAIL ./dirdat/bbTABLE scott.*;
GGSCI (sycamore1) 10> add extract dpscott, exttrailsource ./dirdat/aa
GGSCI (sycamore1) 11> add rmttrail ./dirdat/bb, extract dpscott
passthru - no transformations, so no source database for table definitions
needed
35
Copyright © 2017 Accenture All rights reserved. | 36
CONFIGURE DATA PUMP
GGSCI (sycamore1) 12> start extract dpscott
GGSCI (sycamore1) 13> info all
Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGEXTRACT RUNNING DPSCOTT 00:00:00 00:03:49EXTRACT RUNNING EXSCOTT 00:00:05 00:00:03
36
Copyright © 2017 Accenture All rights reserved. | 37
INSTANTIATE TARGET DATABASE
Instantiation methods:
• GoldenGate initial load - not Oracle specific
• Oracle Database Data Pump
• RMAN (recovery, duplicate from backup or active database)
• Transportable Tablespaces
37
Copyright © 2017 Accenture All rights reserved. | 38
INSTANTIATE TARGET DATABASE
On source database server:
SQL> select dbms_flashback.get_system_change_number from dual;
GET_SYSTEM_CHANGE_NUMBER------------------------
453114
$ vi scott_exp.par
dumpfile=scott.dmplogfile=scott_exp.logdirectory=exportsflashback_scn=453114schemas=scott
$ expdp system parfile=scott_exp.par
$ scp scott.dmp poplar1:
38
Copyright © 2017 Accenture All rights reserved. | 39
INSTANTIATE TARGET DATABASE
On target database server:
$ vi scott_imp.par
dumpfile=scott.dmplogfile=scott_imp.logdirectory=exports
$ impdp system parfile=scott_imp.par
39
Copyright © 2017 Accenture All rights reserved. | 40
REPLICAT
replicat to database: could be via OCI/SQL or connected processes
(integrated)
40
Copyright © 2017 Accenture All rights reserved. | 41
CONFIGURE REPLICAT
SQL> create tablespace ggate datafile '/u01/app/oracle/oradata/PENGUIN/ggate01.dbf’size 20m autoextend on;
SQL> create user ggate identified by ggate default tablespace ggate;
SQL> grant create session, connect, resource, dba to ggate;
SQL> exec dbms_goldengate_auth.grant_admin_privilege('ggate’,’apply');
permissions depend on database version; DBA needed for DDL and sequence
support
41
Copyright © 2017 Accenture All rights reserved. | 42
CONFIGURE REPLICAT
$ cd /u01/app/oracle/product/ggpoplar
$ ./ggsci
GGSCI (poplar1) 1> dblogin userid ggate
GGSCI (poplar1) 2> edit params rpscott
REPLICAT rpscottUSERID ggate, PASSWORD ggateASSUMETARGETDEFSMAP scott.*, TARGET scott.*;
GGSCI (poplar1) 3> add replicat rpscott integrated, exttrail ./dirdat/bb
ASSUMETARGETDEFS - source and target have identical column structure
42
Copyright © 2017 Accenture All rights reserved. | 43
START REPLICAT
GGSCI (poplar1) 4> info all
Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGREPLICAT STOPPED RPSCOTT 00:00:00 00:01:26
GGSCI (poplar1) 5> start replicat rpscott, aftercsn 453114
GGSCI (poplar1) 6> info all
Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGREPLICAT RUNNING RPSCOTT 00:00:07 00:00:09
System Change Number - Commit Sequence Number
Capture starts from when it was registered, so need to start replicat after csn
43
Copyright © 2017 Accenture All rights reserved. | 44
GOLDENGATE
Replication has been configured and is now running from source to target
44
MONITOR GOLDENGATE STATUS
45
Copyright © 2017 Accenture All rights reserved. | 46
EXTRACT STATUS
GGSCI (sycamore1) 1> info extract exscott
EXTRACT EXSCOTT Last Started 2017-10-10 12:23 Status RUNNINGCheckpoint Lag 00:00:03 (updated 00:00:03 ago)Process ID 19367Log Read Checkpoint Oracle Integrated Redo Logs
2017-10-10 15:31:23SCN 0.461602 (461602)
46
Copyright © 2017 Accenture All rights reserved. | 47
EXTRACT STATUS
GGSCI (sycamore1) 2> info extract exscott, detail
EXTRACT EXSCOTT Last Started 2017-10-10 12:23 Status RUNNINGCheckpoint Lag 00:00:04 (updated 00:00:03 ago)Process ID 19367Log Read Checkpoint Oracle Integrated Redo Logs
2017-10-10 15:33:23SCN 0.461791 (461791)
Target Extract Trails:Trail Name Seqno RBA Max MB Trail Type./dirdat/aa 0 834639 500 EXTTRAIL
Integrated Extract outbound server first scn: 0.422356 (422356)Integrated Extract outbound server filtering start scn: 0.422356 (422356)
Extract Source Begin EndNot Available 2017-10-10 12:21 2017-10-10 15:33Not Available * Initialized * 2017-10-10 12:21Not Available * Initialized * 2017-10-10 12:21Not Available * Initialized * 2017-10-10 12:21
Current directory /u01/app/oracle/product/ggsycamore
Report file /u01/app/oracle/product/ggsycamore/dirrpt/EXSCOTT.rptParameter file /u01/app/oracle/product/ggsycamore/dirprm/exscott.prmCheckpoint file /u01/app/oracle/product/ggsycamore/dirchk/EXSCOTT.cpeProcess file /u01/app/oracle/product/ggsycamore/dirpcs/EXSCOTT.pceError log /u01/app/oracle/product/ggsycamore/ggserr.log
Relative Byte Address (RBA) - position in the trail file
47
Copyright © 2017 Accenture All rights reserved. | 48
DATA PUMP STATUS
GGSCI (sycamore1) 3> info extract dpscott
EXTRACT DPSCOTT Last Started 2017-10-10 12:43 Status RUNNINGCheckpoint Lag 00:00:00 (updated 00:00:05 ago)Process ID 20520Log Read Checkpoint File /u01/app/oracle/product/ggsycamore/dirdat/aa000000000
2017-10-10 15:36:50.000000 RBA 849812
48
Copyright © 2017 Accenture All rights reserved. | 49
DATA PUMP STATUS
GGSCI (sycamore1) 4> info extract dpscott, detail
EXTRACT DPSCOTT Last Started 2017-10-10 12:43 Status RUNNINGCheckpoint Lag 00:00:00 (updated 00:00:06 ago)Process ID 20520Log Read Checkpoint File /u01/app/oracle/product/ggsycamore/dirdat/aa000000000
2017-10-10 15:38:30.000000 RBA 857065Target Extract Trails:Trail Name Seqno RBA Max MB Trail Type./dirdat/bb 0 857143 500 RMTTRAIL
Extract Source Begin End/u01/app/oracle/product/ggsycamore/dirdat/aa000000000 * Initialized * 2017-10-10 15:38/u01/app/oracle/product/ggsycamore/dirdat/aa000000000 * Initialized * First Record ./dirdat/aa000000000 * Initialized * First Record
Current directory /u01/app/oracle/product/ggsycamore
Report file /u01/app/oracle/product/ggsycamore/dirrpt/DPSCOTT.rptParameter file /u01/app/oracle/product/ggsycamore/dirprm/dpscott.prmCheckpoint file /u01/app/oracle/product/ggsycamore/dirchk/DPSCOTT.cpeProcess file /u01/app/oracle/product/ggsycamore/dirpcs/DPSCOTT.pceError log /u01/app/oracle/product/ggsycamore/ggserr.log
49
Copyright © 2017 Accenture All rights reserved. | 50
TRAIL FILE STATUS
GGSCI (sycamore1) 5> info exttrail *
Extract Trail: ./dirdat/bbSeqno Length: 9
Flip Seqno Length: noExtract: DPSCOTTSeqno: 0RBA: 871868
File Size: 500M
Extract Trail: ./dirdat/aaSeqno Length: 9
Flip Seqno Length: noExtract: EXSCOTTSeqno: 0RBA: 872230
File Size: 500M
50
Copyright © 2017 Accenture All rights reserved. | 51
STATISTICS FOR EXTRACT
GGSCI (sycamore1) 6> stats extract exscott
Sending STATS request to EXTRACT EXSCOTT ...Start of Statistics at 2017-10-10 15:43:53.Output to ./dirdat/aa:Extracting from SCOTT.PRODUCT to SCOTT.PRODUCT:*** Total statistics since 2017-10-10 12:23:57 ***
Total inserts 0.00Total updates 3997.00Total deletes 0.00Total discards 0.00Total operations 3997.00
*** Daily statistics since 2017-10-10 12:23:57 ***Total inserts 0.00Total updates 3997.00Total deletes 0.00Total discards 0.00Total operations 3997.00
*** Hourly statistics since 2017-10-10 15:00:00 ***Total inserts 0.00Total updates 877.00Total deletes 0.00Total discards 0.00Total operations 877.00
*** Latest statistics since 2017-10-10 12:23:57 ***Total inserts 0.00Total updates 3997.00Total deletes 0.00Total discards 0.00Total operations 3997.00
End of Statistics.
51
Copyright © 2017 Accenture All rights reserved. | 52
REPORT FOR EXTRACTGGSCI (sycamore1) 7> view report exscott
***********************************************************************Oracle GoldenGate Capture for Oracle
Version 12.3.0.1.0 OGGCORE_12.3.0.1.0_PLATFORMS_170721.0154_FBOLinux, x64, 64bit (optimized), Oracle 11g on Jul 21 2017 20:34:23
Copyright (C) 1995, 2017, Oracle and/or its affiliates. All rights reserved.Starting at 2017-10-10 12:22:53
***********************************************************************
Operating System Version:LinuxVersion #2 SMP Fri Oct 21 14:23:10 PDT 2016, Release 4.1.12-61.1.16.el6uek.x86_64Node: sycamore1.localdomainMachine: x86_64
soft limit hard limitAddress Space Size : unlimited unlimitedHeap Size : unlimited unlimitedFile Size : unlimited unlimitedCPU Time : unlimited unlimited
………
************************************************************************* Run Time Messages *************************************************************************2017-10-10 12:23:21 INFO OGG-03522 Setting session time zone to source database time zone 'GMT'.2017-10-10 12:23:56 INFO OGG-06508 Wildcard MAP (TABLE) resolved (entry scott.*): TABLE "SCOTT"."PRODUCT".2017-10-10 12:23:57 INFO OGG-06509 Using the following key columns for source table SCOTT.PRODUCT: PRD_ID.2017-10-10 12:23:57 INFO OGG-06508 Wildcard MAP (TABLE) resolved (entry scott.*): TABLE "SCOTT"."PRODUCT".2017-10-10 12:23:57 INFO OGG-06509 Using the following key columns for source table SCOTT.PRODUCT: PRD_ID.2017-10-10 15:43:53 INFO OGG-01021 Command received from GGSCI: STATS.
52
Copyright © 2017 Accenture All rights reserved. | 53
REPLICAT STATUS
GGSCI (poplar1) 1> info replicat rpscott
REPLICAT RPSCOTT Last Started 2017-10-10 15:23 Status RUNNINGINTEGRATEDCheckpoint Lag 00:00:09 (updated 00:00:10 ago)Process ID 31073Log Read Checkpoint File /u01/app/oracle/product/ggpoplar/dirdat/bb000000000
2017-10-10 16:19:11.680713 RBA 1041343
53
Copyright © 2017 Accenture All rights reserved. | 54
REPLICAT STATUS
GGSCI (poplar1) 2> info replicat rpscott, detail
REPLICAT RPSCOTT Last Started 2017-10-10 15:23 Status RUNNINGINTEGRATEDCheckpoint Lag 00:00:00 (updated 00:00:06 ago)Process ID 31073Log Read Checkpoint File /u01/app/oracle/product/ggpoplar/dirdat/bb000000000
2017-10-10 16:20:49.641431 RBA 1049261INTEGRATED ReplicatDBLOGIN Provided, inbound server name is OGG$RPSCOTT in ATTACHED state
Current Log BSN value: (no data)Low Watermark CSN value: 467372(All source transactions prior to this scn have been applied)High Watermark CSN value: 467374(Some source transactions between this scn and the low watermark may have been applied)
Extract Source Begin End/u01/app/oracle/product/ggpoplar/dirdat/bb000000000 * Initialized * 2017-10-10 16:20/u01/app/oracle/product/ggpoplar/dirdat/bb000000000 * Initialized * First Record./dirdat/bb000000000 * Initialized * First Record
Current directory /u01/app/oracle/product/ggpoplarReport file /u01/app/oracle/product/ggpoplar/dirrpt/RPSCOTT.rptParameter file /u01/app/oracle/product/ggpoplar/dirprm/rpscott.prmCheckpoint file /u01/app/oracle/product/ggpoplar/dirchk/RPSCOTT.cprProcess file /u01/app/oracle/product/ggpoplar/dirpcs/RPSCOTT.pcrError log /u01/app/oracle/product/ggpoplar/ggserr.log
54
Copyright © 2017 Accenture All rights reserved. | 55
REPLICAT LAG
GGSCI (poplar1) 3> lag replicat rpscott
Sending GETLAG request to REPLICAT RPSCOTT ...Last record lag 4 seconds.Low watermark lag: 8.High watermark lag: 5.Low watermark position: 468156.High watermark position: 468158.At EOF, no more records to process
55
Copyright © 2017 Accenture All rights reserved. | 56
STOP AND START AN EXTRACT OR REPLICAT
GGSCI (sycamore1) 8> stop extract exscott
Sending STOP request to EXTRACT EXSCOTT ...Request processed.
GGSCI (sycamore1) 9> info all
Program Status Group Lag at Chkpt Time Since ChkptMANAGER RUNNINGEXTRACT RUNNING DPSCOTT 00:00:00 00:00:01EXTRACT STOPPED EXSCOTT 00:00:02 00:00:07
GGSCI (sycamore1) 10> start extract exscott
Sending START request to MANAGER ...EXTRACT EXSCOTT starting
Log file:
/u01/app/oracle/product/ggsycamore/ggserr.log
56
Copyright © 2017 Accenture All rights reserved. | 57
ORACLE DATABASE DICTIONARY VIEWS
Source Database:
SQL> select * from dba_capture;
Target Database:
SQL> select * from dba_apply;
57
Copyright © 2017 Accenture All rights reserved. | 58
HEALTH CHECK FOR INTEGRATED EXTRACT AND REPLICAT
1448324.1 (support.oracle.com)
58
LOGICAL REPLICATION
Logical not Physical Replication (cf. Physical Data Guard or storage replication), where do not need to worry about contentsSQL not blocksThat's how it can:• cross database type• cross platform• cross endian• character set conversion• subset of database/tables• subset of rows: filtering on source (capture) or target (apply)• transform
59
Copyright © 2017 Accenture All rights reserved. | 60
ALTERNATIVE LOGICAL REPLICATION PRODUCTS
also streams
60
Copyright © 2017 Accenture All rights reserved. | 61
LOGICAL REPLICATION
Some data types are not supported
Must be able to uniquely identify a row
Set based operations become row-by-row operations
blog.dbvisit.com/3-fundamentals-of-oracle-replication-part-1-of-3
Uniquely identify:
• PK/UK, otherwise concatenates all columns - may or may not guarantee
uniqueness, or KEYCOLS
• hence extra supplemental logging - PK/UK data into the redo logs
Set based row-by-row:
• Redo: block changes, not SQL
61
Copyright © 2017 Accenture All rights reserved. | 62
SET BASED OPERATIONS BECOME ROW-BY-ROW OPERATIONS
Execute a multiple row update statement on source database:
SQL> update scott.emp set hiredate = sysdate;
14 rows updated.
Statement uses table access full:---------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------| 0 | UPDATE STATEMENT | | | | 3 (100)| || 1 | UPDATE | EMP | | | | || 2 | TABLE ACCESS FULL| EMP | 14 | 168 | 3 (0)| 00:00:01 |---------------------------------------------------------------------------
62
Copyright © 2017 Accenture All rights reserved. | 63
SET BASED OPERATIONS BECOME ROW-BY-ROW OPERATIONS
On target database GoldenGate Replicat executes statement below, many times:
UPDATE /*+ restrict_all_ref_cons */ "SCOTT"."EMP" SET "HIREDATE"=:1 WHERE "EMPNO"=:2
Statement uses index access:------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| ------------------------------------------------------------------| 0 | UPDATE STATEMENT | | | | 1 (100)| | 1 | UPDATE | EMP | | | | |* 2 | INDEX UNIQUE SCAN| PK_EMP | 1 | 12 | 0 (0)| ------------------------------------------------------------------
Predicate Information (identified by operation id): ---------------------------------------------------
2 - access("EMPNO"=:2)
63
PATRICK HURLEY
GOLDENGATE
A LIVE INTRODUCTION
Covered• install• set up replication• had to discard more content than I have included:
• filtering (table or row)• handling replication errors• parallelism (performance)
• key points about logical replication
Whet appetite, get started
64
Copyright © 2017 Accenture All rights reserved. | 65
ORACLE GOLDENGATE 12C IMPLEMENTER’S GUIDE
65