Expert Oracle GoldenGate

Ben Prusinski

Steve PhillipsRichard Chung

Apress*

Contents

About the Authors xvii

About the Technical Reviewer xviii

Acknowledgments xix

Chapter 1: Introduction .....1

Distributed Processing and Replication 1

Oracle Basic Replication 2

Oracle Advanced Replication 2

Oracle Streams Replication 3

Evolution and Oracle GoldenGate 4

Summary 4

Chapter 2: Installation 5

Downloading the Software 5

Downloading from Oracle E-Delivery 5

Downloading from OTN 9

Understanding Your Environment 12

Reviewing the Install Instructions 14

Installing Goldengate. 16

General System Requirements 16

Memory Requirements 16

Disk Space Requirements 17

Network Requirements 17

Operating System Requirements 18

Requirements for Microsoft Windows Clustered Environments 18

Installing Goldengate on Windows 18

Installing GoldenGate 11g on Linux and UNIX 20

Environment Settings for Oracle and Goldengate on Linux and UNIX 20

GoldenGate and Oracle RAC Considerations 21

Installing Goldengate for Microsoft SQL Server on Windows 21

Installing Goldengate for Teradata on Windows and UNIX 21

Installing Goldengate for Sybase on Windows and UNIX 23

Installing GoldenGate for IBM DB2 UDB on Windows and UNIX 23

Installing Oracle GoldenGate Director 11g 24

System Requirements 25

Installing Oracle GoldenGate Director Server 27

Grant Database Privileges and Credentials to Oracle GoldenGate Director Server Schema 27

Install Oracle GoldenGate Director 27

Installing Oracle GoldenGate Veridata 27

GoldenGate Veridata Agent System Requirements 28

GoldenGate Veridata Agent Disk Requirements 28

GoldenGate Veridata Agent Memory Requirements 28

GoldenGate Veridata Agent Database Privileges 28

GoldenGate Veridata Server System Requirements 29

Install Oracle Goldengate Veridata 31

Summary 31

Chapter 3: Architecture —-33

Typical GoldenGate Flow 33

GoldenGate Components 34

Source Database 34

Capture (Local Extract) Process 35

Source Trail 36

Data Pump 36

Network 38

Collector 38

Remote Trail 38

Delivery (Replicat) 39

Target Database 39

Manager 40

Topologies and Use Cases 40

One-Way Replication 40

Bidirectional Replication 43

Broadcast Replication 44

Integration Replication 45

Tools and Utilities 46

GGSCI 46

DEFGEN 46

Logdump 46

Reverse. 46

Veridata 46

Director 47

Summary 47

Chapter 4: Basic Replication 49

Overview 49

Prerequisites for Setting Up Replication 49

Requirements 50

One-Way Replication Topology 52

Basic Replication Steps 52

Starting the Extract 54

Verifying Database-Level Supplemental Logging 54

Enabling Database-Level Supplemental Logging 54

Enabling Table-Level Supplemental Logging 55

Disabling Triggers and Cascade-Delete Constraints 57

Verifying the Manager Status 58

Configuring the Local Extract 59

Adding the Extract 63

Starting and Stopping the Extract 63

Verifying the Extract 63

Starting the Data Pump 65

Configuring the Data Pump 65

Adding the Data Pump 67

Starting and Stopping the Data Pump 68

Verifying the Data Pump 68

Loading with GoldenGate 70

Prerequisites for the GoldenGate Initial Load 70

Configuring the Initial-Load Extract 71

Adding the Initial-Load Extract 71

Configuring the Initial-Load Replicat 72

Adding the Initial-Load Replicat 74

Starting the GoldenGate Initial Load 74

Verifying the Initial Load 74

Loading with DBMS Utilities 75

Prerequisites for Loading with DBMS Utilities 76

Steps for Loading with DBMS Utilities 76

Starting the Replicat 76

Configuring the Replicat 77

Adding the Replicat 81

Starting and Stopping the Replicat 81

Verifying the Replicat 82

Summary 83

Chapter 5: Advanced Features 85

Enhancing the Replication Configuration 85

Enhancing Extract and Replicat Reporting 86

Reporting Discarded Records 89

Purging Old Trail Files 89

Adding Automatic Process Startup and Restart 90

Adding a Checkpoint Table 90

Making the Replication More Secure 92

Encrypting Passwords 92

Encrypting the Trail Files 93

Adding Data Filtering and Mapping 94

Filtering Tables 94

Filtering Columns 95

Filtering Rows 96

Mapping Columns 97

Transforming Columns 98

Oracle-Specific DBMS Configuration Options.. 99

Configuring for Oracle RAC 100

Configuring for Oracle ASM 102

Adding Oracle DDL Replication 103

Adding Bidirectional Replication 106

Excluding Transactions for Bidirectional Replication 108

Handling Conflict Resolution for Bidirectional Replication 109

Summary 11°

Chapter 6: Heterogeneous Replication 111

Microsoft SQL Server to Oracle Replication 111

Preparing the Oracle GoldenGate Environments 113

Initial Data Load Completion for Oracle GoldenGate Environments 113

Source Oracle Database Configuration 113

Target Microsoft SQL Server Database Configuration 115

Creating Sample Microsoft SQL Server Database 118

Configure Change Data Capture on Source 121

Verifying Operational Readiness 124

Summary 125

Chapter 7: Tuning 127

Tuning Methodology 127

Defining the Performance Requirements 128

Creating a Performance Baseline 129

Evaluating the Current Performance 131

Determining the Problem 132

Designing and Implementing a Solution 134

Using Parallel Extracts and Replicats 134

Implementing Parallel Extracts and Replicats with Table Filtering 135

Implementing Parallel Extracts and Replicats Using Key Ranges 141

Using BATCHSQL 146

Using GROUPTRANSOPS 149

Tuning Disk Storage 149

Tuning the Network 150

Tuning the RMTHOST Parameter 151

Tuning the Database 152

Summary 152

Chapter 8: Monitoring Oracle GoldenGate 153

Designing a Monitoring Strategy 153

Why Monitoring the Extract Is Important 155

Getting the Maximum Threshold 155

Which Processes to Monitor in the GoldenGate Environment 155

Monitoring All Running Processes 156

Monitoring the Detail Extract 156

Checking the Current Oracle Online Redo Log 157

Checking the Archived Log 157

Monitoring the GoldenGate Rate and Redo Log Consumption Rate 158

Monitoring Lags for Each Group 159

Viewing Event and Error Logs 162

Automating Monitoring 162

Checking GoldenGate Process Scripts 162

Monitoring Lag Scripts 163

Checking Memory and CPU Scripts 168

Checking Disk Space 168

Summary 169

Chapter 9: Oracle GoldenGate Veridata 171

Veridata Components 171

GoldenGate Veridata Server 172

GoldenGate Veridata Web 173

GoldenGate Veridata Repository 173

GoldenGate Veridata Agent, Java, and C-Code 173

GoldenGate Veridata CLI (Vericom) 173

How Veridata Comparison Works 173

How Veridata Can Help You 174

Setting Up the Veridata Compares 175

Creating Database Connections 175

Setting Up Tables and Data Scripts 176

Creating a Group 177

Creating Compare Pairs 177

Creating a Job 178

Creating a Profile 179

Running the Veridata Job 179

Improving Performance and Reducing Overhead 180

Excluding Columns 180

Tuning Profiles Settings 180

Disabling the Confirm Out of Sync Step 180

Increasing the Number of Threads 180

Compares Methods 180

Right Trim on Character Fields 181

Comparing Incremental Data for Large Tables 181

Comparing GoldenGate Real-Time Replication Data 182

Comparing Different Column Types and Compare Formats 182

Using Performance Statistics 183

Using Vericom Command Line 185

Setting Up Role-Based Security 186

Summary 188

Chapter 10: GoldenGate Director 189

Director Components 189

GoldenGate GGSCI Instances 190

Director Administrator 190

Director Server and Database 190

Director Web 191

Director Client 192

Setting Up the Data Source 192

Modifying the Manager Parameter file 193

Modifying the Parameter File Using the Built-in Editor 193

Modifying the Parameter File Using the GUI 194

Setting Up the Initial Load 195

Adding an Initial Load Task 196

Creating a One-Way Replication 200

Adding a Data Pump Process 204

Additional Director Features and Tricks 206

Alter Extract or Replicat RUN Options 206

Changing the Trail-File Size 206

Extracting Tranlogoptions 206

Generating Definition Files 207

Finding Parameters or Attributes in Director 209

Advanced Mapping 210

Alerts 213

Summary 215

Chapter 11: Troubleshooting Oracle GoldenGate .217

Common Problems and Solutions 217

Oracle GoldenGate Process Failures 218

Oracle GoldenGate Extract Process Failures 219

Oracle GoldenGate Process Failures Without Report Diagnostics 221

Oracle GoldenGate Trail File Problems 221

Trail Files that Don't Empty 221

Trail Files that Don't Roll Over 222

Trail File Purging Issues 222

Trail Files that Purge Too Soon 223

Oracle GoldenGate Error Log Analysis 223

Understanding the Oracle GoldenGate Discard File 223

Discard File Not Created 224

Discard File Is Too Big 224

Can't Open the Discard File 225

Using Trace Commands with Oracle GoldenGate 225

Oracle GoldenGate Process Tracing with TLTRACE 225

Using TRACE Parameters with Oracle GoldenGate 226

Troubleshooting Case Study with Oracle GoldenGate 226

Oracle GoldenGate Configuration Issues 226

Incorrect Software Versions with Oracle GoldenGate 227

Database Availability Issues 227

Missing Oracle GoldenGate Process Groups 228

Missing Oracle GoldenGate Trail Files 228

Oracle GoldenGate Parameter File Configuration Issues 229

Operating System Configuration Issues with Oracle GoldenGate 230

Network Configuration Issues with Oracle GoldenGate 231

Network Data-Transfer Issues 232

Oracle Database Issues with GoldenGate 234

Extract Can't Access Oracle Database Archive and Redo Logs 235

Extract Failure Conditions Due to Oracle Source Database Issues 235

Data-Pump Errors 236

Replicat Errors on the Oracle Database Target System 236

Replicat Hangs on the Target System 236

Replicat Experiences an Abend Failure on the Target System 236

Replicat Fails on Large Transactions 238

Incompatible Record Errors with a Replicat 238

Data-Synchronization Issues 238

Tables Missing Key Constraints 239

Character Set Configuration Issues 239

Missing-Column Errors 239

Fetch Failures 239

Summary 240

Chapter 12: Disaster Recovery Replication 241

Prerequisites 241

Requirements 242

Disaster Recovery Replication Topology 243

Setup 244

Configuring the Local Extract for Disaster Recovery 245

Configuring the Data Pump for Disaster Recovery 246

Configuring the Replicat for Disaster Recovery 247

Configuring the Standby Extract 248

Configuring the Standby Data Pump 249

Configuring the Standby Replicat 250

Performing a Planned Switchover 251

Performing an Unplanned Failover 256

Summary 262

Chapter 13: Zero-Downtime Migration Replication 263

Prerequisites 263

Requirements 264

Zero-Downtime Migration Topology 265

Setup 266

Configuring the Local Extract for Zero-Downtime Migration 267

Configuring the Data Pump for Zero-Downtime Migration 269

Configuring the Replicat for Zero-Downtime Migration 270

Configuring the Fallback Local Extract for Zero-Downtime Migration 272

Configuring the Fallback Data-Pump for Zero-Downtime Migration 272

Configuring the Fallback Replicat for Zero-Downtime Migration 273

Performing the Migration Cutover 274

Performing the Migration Fallback 276

Summary 278

Chapter 14: Tips and Tricks .279

Requirements and Planning 279

Knowing the Business Objectives 279

Understanding the Requirements 280

Determining the Topology 281

Installation and Setup 281

Creating Dedicated Users 281

Encrypting Passwords 282

Creating a Dedicated Installation Directory 282

Using a Checkpoint Table 282

Verifying the Character Sets 283

Developing Naming Standards 283

Using a Data Pump 283

Management and Monitoring 284

Using GGSCI Command Shortcuts 284

Using OBEY Files 285

Generating Interim Statistics 286

Using a Discard File 286

Reporting Regularly on Process Health 286

Purging Old Trail Files Regularly 287

Automatically Starting Processes 287

Performance 287

Running Performance Tests 287

Limiting the Number of Extracts 288

Using Passthru Mode for Data Pumps 288

Using Parallel Replicats 288

Using the Fastest Available Storage 288

Tuning the Database 289

Summary 289

Appendix: Additional Technical Resources for the

Oracle GoldenGate Administrator 291

References for Further Reading 291

Quick Guide to Oracle GoldenGate Commands 292

ADD 292

GGSCI 293

HELP 293

INFO 294

SEND 295

STATUS 295

Logdump Commands and Syntax for Troubleshooting 296

Accessing the Logdump Utility 296

Getting Help with Logdump Syntax 296

HISTORY 299

Opening GoldenGate Trail Files with Logdump 299

Index 301