managing oracle streams using enterprise manager grid control
DESCRIPTION
Presentation for Central Florida Oracle Users Group on how to manage Oracle Streams using Oracle Grid Control in 10gTRANSCRIPT
Managing Oracle Managing Oracle Streams 10g Using Streams 10g Using Oracle Grid ControlOracle Grid Control
Scott R Baker8/24/2009
This presentation can be viewed at:http://www.slideshare.net/scottb411
Scott R BakerScott R Baker Fall 1999: - Graduated from the University of Central Fall 1999: - Graduated from the University of Central
Florida with BS Management information SystemsFlorida with BS Management information Systems Fall 1999 – August 2005: Senior Support Engineer Fall 1999 – August 2005: Senior Support Engineer
with Oracle Corporation (Orlando)with Oracle Corporation (Orlando) 2002 – Oracle Certified DBA in 8i2002 – Oracle Certified DBA in 8i August 2005 - May 2009: Senior Oracle DBA; DBA August 2005 - May 2009: Senior Oracle DBA; DBA
Manager with Bonnier Corporation (Formerly World Manager with Bonnier Corporation (Formerly World Publications)Publications)
May 2009 - Present: Development Manager - (DBA / May 2009 - Present: Development Manager - (DBA / Quality Assurance / Support) with Bonnier Quality Assurance / Support) with Bonnier CorporationCorporation
Bonnier CorporationBonnier Corporation Bonnier Corporation is one of the largest Bonnier Corporation is one of the largest
consumer-publishing groups in America, and with consumer-publishing groups in America, and with nearly 50 special-interest magazines and related nearly 50 special-interest magazines and related multimedia projects and events, it is the leading multimedia projects and events, it is the leading media company serving passionate, highly media company serving passionate, highly engaged audiences. With over 1,000 employees engaged audiences. With over 1,000 employees and more than $350 million in annual revenue, and more than $350 million in annual revenue, Bonnier Corp. ranks in the top 10 nationally Bonnier Corp. ranks in the top 10 nationally among publishing companies. among publishing companies.
Headquarters are in Winter Park, FLHeadquarters are in Winter Park, FL
http://www.bonniercorp.comhttp://www.bonniercorp.com
Bonnier Corporation’s Use of Bonnier Corporation’s Use of Oracle StreamsOracle Streams
Bonnier Corporation’s use of Oracle Bonnier Corporation’s use of Oracle Streams was featured in Oracle Magazine Streams was featured in Oracle Magazine in June 2008:in June 2008:– http://www.oracle.com/technology/oramag/orachttp://www.oracle.com/technology/oramag/orac
le/08-jul/o48linuxsupport.htmlle/08-jul/o48linuxsupport.html
In December 2008, Oracle released a case In December 2008, Oracle released a case study of our use of Oracle Streams at study of our use of Oracle Streams at Bonnier:Bonnier:– http://www.oracle.com/technology/deploy/availhttp://www.oracle.com/technology/deploy/avail
ability/pdf/bonnier_casestudy.pdfability/pdf/bonnier_casestudy.pdf
Bonnier Corporation’s Use of Bonnier Corporation’s Use of Oracle StreamsOracle Streams
Streams Replication Streams Replication OverviewOverview
Streams ReplicationStreams Replication Logical Replication vs Physical ReplicationLogical Replication vs Physical Replication
– Streams is logical-based replicationStreams is logical-based replication
Rule-Based ReplicationRule-Based Replication– Positive Rules of ReplicationPositive Rules of Replication
What do you want to replicate?What do you want to replicate?– Negative Rules of ReplicationNegative Rules of Replication
What don’t you want to replicate?What don’t you want to replicate?
Streams AdministratorStreams Administrator– Streams environment self-contained within it’s Streams environment self-contained within it’s
own schema and administratorown schema and administrator
Streams ReplicationStreams ReplicationProcesses:Processes:
Capture: A Streams capture process mines redo logs to create one or more Capture: A Streams capture process mines redo logs to create one or more logical change records (LCRs) and queues them to a Capture Queue. An logical change records (LCRs) and queues them to a Capture Queue. An LCR is a message with a specific format that describes a database changeLCR is a message with a specific format that describes a database change
Propagation: Streams propagation propagates the staged LCR to another Propagation: Streams propagation propagates the staged LCR to another queue residing in the destination database where apply will occur.queue residing in the destination database where apply will occur.
Apply: Once the LCR has reached the destination database, a Streams Apply: Once the LCR has reached the destination database, a Streams apply process consumes the change by applying the LCR to the shared apply process consumes the change by applying the LCR to the shared database object.database object.
Source: Oracle Streams Concepts and AdministrationSource: Oracle Streams Concepts and Administration
Source: Oracle® Database High Availability Overview 10Source: Oracle® Database High Availability Overview 10gg Release 2 (10.2) Release 2 (10.2)
Oracle Streams Oracle Streams DemonstrationDemonstration
PrerequisitesPrerequisites
Database must be in archivelog modeDatabase must be in archivelog mode– archive log list;archive log list;
Undo_retention must be set to 86400Undo_retention must be set to 86400– select name, value from v$parameter where name like '%globalselect name, value from v$parameter where name like '%global
%';%';– alter system set global_names=TRUE scope=both;alter system set global_names=TRUE scope=both;
Global_name must be set to trueGlobal_name must be set to true– select name, value from v$parameter where name like '%undoselect name, value from v$parameter where name like '%undo
%';%';– alter system set undo_retention=86400 scope=both;alter system set undo_retention=86400 scope=both;
Global Names must be used for each databaseGlobal Names must be used for each database
From each database:From each database:
SQL> select * from global_name;SQL> select * from global_name;
GLOBAL_NAME (Source)GLOBAL_NAME (Source)----------------------------------------------------------------------------------------------------------------------------------------------------RUNI.DOMAINRUNI.DOMAIN
GLOBAL_NAME (Destination)GLOBAL_NAME (Destination)----------------------------------------------------------------------------------------------------------------------------------------------------TICKET.DOMAINTICKET.DOMAIN
Create streams administrator in both the source Create streams administrator in both the source and destination databaseand destination database
> CREATE USER strmadmin identified by streams> CREATE USER strmadmin identified by streams
default tablespace USERSdefault tablespace USERS
temporary tablespace TEMP;temporary tablespace TEMP;
>grant dba to strmadmin;>grant dba to strmadmin;
>alter user strmadmin quota unlimited on USERS;>alter user strmadmin quota unlimited on USERS;
Configure streams administrator in Configure streams administrator in both environments:both environments:
BEGINBEGIN
DBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE(DBMS_STREAMS_AUTH.GRANT_ADMIN_PRIVILEGE(
grantee => 'strmadmin',grantee => 'strmadmin',
grant_privileges => TRUE);grant_privileges => TRUE);
END;END;
//
select * from dba_streams_administrator;select * from dba_streams_administrator;
Create a database link from the source to destination and Create a database link from the source to destination and vice-versavice-versa
On Source:On Source:
create database link ticket.DOMAIN connect to strmadmin create database link ticket.DOMAIN connect to strmadmin identified by streams using 'TICKET';identified by streams using 'TICKET';
select * from [email protected] * from [email protected]
On Destination:On Destination:
create database link RUNI.DOMAIN connect to strmadmin create database link RUNI.DOMAIN connect to strmadmin identified by streams using 'RUNI';identified by streams using 'RUNI';
select * from [email protected] * from [email protected]
Directories must be setup for export and import Directories must be setup for export and import – ($ ls -al | grep exp_dir)($ ls -al | grep exp_dir)
In /home/oracle:In /home/oracle:
[oracle]$ mkdir src_exp_dir[oracle]$ mkdir src_exp_dir[oracle]$ chmod 777 src_exp_dir[oracle]$ chmod 777 src_exp_dir[oracle]$ mkdir dst_exp_dir[oracle]$ mkdir dst_exp_dir[oracle]$ chmod 777 dst_exp_dir[oracle]$ chmod 777 dst_exp_dir
On source as strmadmin user:On source as strmadmin user:--------------------------------------------------------SQL> connect strmadmin/streamsSQL> connect strmadmin/streams>create directory src_exp_dir as '/home/oracle/src_exp_dir';>create directory src_exp_dir as '/home/oracle/src_exp_dir';
On destination as strmadmin user:On destination as strmadmin user:------------------------------------------------------------------SQL> connect strmadmin/streamsSQL> connect strmadmin/streams>create directory dst_exp_dir as '/home/oracle/dst_exp_dir';>create directory dst_exp_dir as '/home/oracle/dst_exp_dir';
Begin Using Enterprise Begin Using Enterprise ManagerManager
Log out of Enterprise Manager for the source Log out of Enterprise Manager for the source database and log back into the source database database and log back into the source database as the strmadmin useras the strmadmin user
In Enterprise Manager, Click on the Maintenance TabIn Enterprise Manager, Click on the Maintenance Tab– Source (Runi) / Destination (Ticket)Source (Runi) / Destination (Ticket)
Under Data Movement, click on the “Setup” link under StreamsUnder Data Movement, click on the “Setup” link under Streams
Select: Select:
Streams Global, Schema, Table and Subset Streams Global, Schema, Table and Subset Replication WizardReplication Wizard
Provide strmadmin for source/destinationProvide strmadmin for source/destination
For this demonstration, we will select “Schema For this demonstration, we will select “Schema Rule”Rule”
Important: Go to next slide before hitting the “Next” button
Leave Processes to their default Leave Processes to their default valuesvalues– CaptureCapture– PropagatePropagate– ApplyApply
Important: Go to next slide before hitting the “Next” button
Use flashlights to select directory objectsUse flashlights to select directory objects
These are the directories we created in our prerequisites
Important: Go to next slide before hitting the “Next” button
Choose to propagate DDLChoose to propagate DDL
Capturing DDL changes will apply structural changes to tables and indexes to the destination database. If you create a new table, it will be created in the destination database as well.
Click the “Next” button in the Wizard
Include the Schema “SCOTT” by using the Include the Schema “SCOTT” by using the flashlight or typing in directly.flashlight or typing in directly.
Click on the “Exclude Tables” Button at the bottom of the page. Proceed to next slide
Use the Flashlight to exclude the table Use the Flashlight to exclude the table “Bonus” in the Scott/Tiger schema“Bonus” in the Scott/Tiger schema
Review the replication settings that you Review the replication settings that you have made.have made.
Streams RulesStreams Rules
Selecting the Scott/Tiger schema to Selecting the Scott/Tiger schema to replicate was a replicate was a positivepositive rule rule
Choosing to not replicate the BONUS Choosing to not replicate the BONUS table from the Scott/Tiger schema table from the Scott/Tiger schema was a was a negativenegative rule. rule.
You need to submit the job in order for the You need to submit the job in order for the replication to take place.replication to take place.
Make sure that you provide the host credentials prior to submitting the job. This is the Oracle account that you authenticate to the database server as.
We are now Streaming!We are now Streaming!In the source database, execute the following:
SQL> CREATE TABLE SCOTT.streams_demo( demo_pk NUMBER, demo NUMBER);
SQL> ALTER TABLE SCOTT.STREAMS_DEMO ADD ( CONSTRAINT STREAMS_DEMO_PK PRIMARY KEY (demo_pk));
SQL> insert into scott.streams_demo values (5,5);SQL> insert into scott.bonus values ('demo','teacher',200, 10);SQL> commit;
Verify the table and record is showing up in the streams_demo destination database but not the bonus table (Negative Rule).
Let’s Break StreamsLet’s Break Streams On destination database, execute the On destination database, execute the
following:following:
SQL> update scott.streams_demo set demo = 6;SQL> update scott.streams_demo set demo = 6;SQL> commit;SQL> commit;
On the source database, execute the On the source database, execute the following:following:
SQL> update scott.streams_demo set demo = 7 SQL> update scott.streams_demo set demo = 7 where demo=5;where demo=5;
SQL> commit;SQL> commit;
On the destination database In Enterprise Manager, Click on the Maintenance TabOn the destination database In Enterprise Manager, Click on the Maintenance Tab– Destination (Ticket)Destination (Ticket)
Under Data Movement, click on the “Management” link under StreamsUnder Data Movement, click on the “Management” link under Streams
We can see that the Apply process is We can see that the Apply process is having errors.having errors.
Click on the “Apply” link to look into it
We can see that the Apply process We can see that the Apply process has a status of “ABORTED”has a status of “ABORTED”
Click on the “Errors” button to look into the problem.
Click on the icon for “View Error LCRs”Click on the icon for “View Error LCRs”
The operation failed on an update. Click on the “Compare Value” button.
We can see what changed and what the different values are
Let’s correct the record in the destination database:Let’s correct the record in the destination database:
SQL> update scott.streams_demo set demo = 7 where SQL> update scott.streams_demo set demo = 7 where demo_pk = 5;demo_pk = 5;
SQL> commit;SQL> commit;
Select the record and click on the “DELETE” button
In the Source database, insert a new record
SQL> insert into scott.streams_demo values (8,8);SQL> commit;
Check the destination database for Check the destination database for the new record.the new record.
What happened?What happened?
Why did it happen?Why did it happen?
The Apply process is still abortedThe Apply process is still aborted
Click on the “Start” button to enable it
Check the STREAMS_DEMO table on the destination database again. The record now shows up. It was saved in the queue until the apply process was enabled.
By Default, the streams apply process will stop all By Default, the streams apply process will stop all replication until the problem record(s) are cleaned replication until the problem record(s) are cleaned up. If you want replication to continue even if up. If you want replication to continue even if there is a problem, click on the “Edit” button for there is a problem, click on the “Edit” button for the apply process…the apply process…
… … and change the parameter DISABLE_ON_ERROR and change the parameter DISABLE_ON_ERROR from “Y” to “N” and click “Apply”.from “Y” to “N” and click “Apply”.
In the previous scenario, the new record would have been inserted, regardless of the previous error message, and the apply process would not have been aborted.
Go back to the source database and click Go back to the source database and click on the Streams “Management” linkon the Streams “Management” link
Under the Caputure section, click on the link Under the Caputure section, click on the link under the “Negative Rule Set” columnunder the “Negative Rule Set” column
Click on the “Create” button
Select “Table Rule” and both DML and DDL ChangesSelect “Table Rule” and both DML and DDL Changes
EnterEnter– Schema: SCOTTSchema: SCOTT– Table: STREAMS_DEMOTable: STREAMS_DEMO
Click on the “Go” button and “OK”Click on the “Go” button and “OK”
Under the Search Box, select Table for the Rule Under the Search Box, select Table for the Rule Type and click on the “Go” buttonType and click on the “Go” button
We can see that there is now a negative DML rule and DDL rule for the BONUS and STREAMS_DEMO table.
Insert a new record into the STREAMS_DEMO table Insert a new record into the STREAMS_DEMO table on the source database:on the source database:
SQL> insert into scott.streams_demo values (9,9);SQL> commit;
As expected, the record does not get copied over.
Remove StreamsRemove Streams
From the Streams Dashboard, you From the Streams Dashboard, you can completely remove your streams can completely remove your streams environment by clicking on the environment by clicking on the “Remove Streams Configuration” link “Remove Streams Configuration” link shown below.shown below.
Good to KnowGood to Know Streams HealthcheckStreams Healthcheck
– Metalink Doc ID:Metalink Doc ID: 273674.1 Streams Configuration Report 273674.1 Streams Configuration Report and Health Check Scriptand Health Check Script (Equivalent to AWR/Statspack report (Equivalent to AWR/Statspack report built to diagnose Streams issues).built to diagnose Streams issues).
Error Logs not showing up in Enterprise ManagerError Logs not showing up in Enterprise Manager– If you are having errors for the streams processes but the logs If you are having errors for the streams processes but the logs
aren’t showing up in Enterprise Manager, log out of Enterprise aren’t showing up in Enterprise Manager, log out of Enterprise Manager and log back in. Enterprise Manager doesn’t always Manager and log back in. Enterprise Manager doesn’t always show the latest logs unless you log out and back in.show the latest logs unless you log out and back in.
SequencesSequences– If you are setting up master to master replication, be sure If you are setting up master to master replication, be sure
sequences are even numbers on source and odd numbers on sequences are even numbers on source and odd numbers on destination or vice-versa.destination or vice-versa.
Oracle TextOracle Text– The Oracle text index will break if using streams when it is The Oracle text index will break if using streams when it is
refreshed. Manage the indexes locally.refreshed. Manage the indexes locally.
Questions?Questions? Scott BakerScott Baker
Email: Email: [email protected]@yahoo.comLinked In: Linked In: http://www.linkedin.com/in/scottbakerprofilehttp://www.linkedin.com/in/scottbakerprofile
This presentation can be viewed at:http://www.slideshare.net/scottb411