Managing Change with Real Application Managing Change with Real Application Testing and Snapshot StandbyTesting and Snapshot Standby

Barry HodgesBarry HodgesSenior Solution Architect, Sales Consulting, Oracle NZSenior Solution Architect, Sales Consulting, Oracle NZ

                     

““The art of progress is to preserve order The art of progress is to preserve order amid change.”amid change.”

Alfred North Whitehead

British Mathematician and Philosopher, 1861-1947

IT ChallengesIT Challenges

Motivation for changeMotivation for change

Database TierDatabase Tier

Database Upgrades,

Patches, Schema, Parameters,

RAC nodes, Interconnect,

OS Platforms, OS Upgrades,

CPU, Memory, etc

Application Upgrades,

Object Modifications, etc

Application TierApplication Tier

SAN Upgrades,

ASM, more disk, etc

Storage TierStorage TierStorage Tier

Database Upgrade Process: StepsDatabase Upgrade Process: Steps

1. Gather information about PROD environment

2. Determine upgrade path & method

3. Prepare B/R strategy, clone and setup Test DB

4. Establish performance baseline before upgrade

5. Develop tests for database & applications

6. Test upgraded database & applications

7. Check performance before & after upgrade

8. Remediate regressions

9. Go Live!

Majority of effort spent in developing tests, detecting and remediating regressions

Current Testing MethodologyCurrent Testing Methodology

PRODUCTION TEST

Need to provision the whole stack

BA’s id top n business

transactions

Test Scripts need to be constantly

rewritten as data changes

Simulated load 80/20?

Simulate load by over

throttling?

Can we trust the simulated

load?

What is the correct

throughput?

Can we account for selects?

Requires specialised application knowledge

Database Upgrade Process: StepsDatabase Upgrade Process: Steps

1. Gather information about PROD environment

2. Determine upgrade path & method

3. Prepare B/R strategy, clone and setup Test DB

4. Establish performance baseline before upgrade

5. Develop tests for database & applications

6. Test upgraded database & applications

7. Check performance before & after upgrade

8. Remediate regressions

9. Go Live!

Majority of effort spent in developing tests, detecting and remediating regressions

Real Application TestingReal Application Testing provides realistic testing for real-world systems

Database ReplayDatabase Replay• Replay actual production database workload in test

environment • Identify, analyse and fix potential instabilities before making

changes to production

• Capture Workload in Production– Capture full production workload– Move the captured workload to test system

• Replay Workload in Test– Make the desired changes in test system– Replay workload with with production load, timing & concurrency

characteristics– Honor commit ordering

• Analyze & Report– Errors– Data divergence – Performance divergence

SQL Performance AnalyzerSQL Performance Analyzer• Identify SQL performance regression before end-users are

impacted • Identify, analyse and fix potential instabilities before making

changes to production

• Re-execute SQL queries in Test– Make the desired changes in test system– Statements can be re-executed as often as is necessary in the

test environment

• Analyze & Report– Highlight improvements– Highlight regressions– Highlight highest impact statements– Use SQL Tuning Advisor to tune regressed statements

• Capture SQL in Production– Capture SQL workload in production including statistics & binds– Move the SQL Tuning Set to test system

SQLSQL

Un

sup

po

rte

d

Supported ChangesSupported Changes

Database Upgrades (9i, 10g 11g),

Patches, Schema, Parameters,

RAC nodes, Interconnect,

OS Platforms, OS Upgrades,

CPU, Memory

Application Upgrades,

Some Object Modifications

Database TierDatabase Tier

Application TierApplication Tier

Storage TierStorage Tier

SAN Upgrades,

ASM, more disk, etc

Su

pp

ort

ed

Current Testing MethodologyCurrent Testing Methodology

DB Replay: 2 weeks

LoadRunner: 30 weeks

Total Testing Time

0

20

40

60

80

Tim

e Tak

en (D

ays)

Install & Setup UnderstandApplication

Usage

Identify KeyTransactions

GenerateWorkload

Run Test

DB Replay

LoadRunner

Install & Setup Understand Application

Usage

Identify Key Transactions

Generate Workload

Run Test

Eff

ort

(D

ays

)

Real Application Testing DemoReal Application Testing DemoProduction (wgn1)

Standby (wgn1sby)

Dell OPTIPLEX 755

Intel(R) Dual Core(TM)2 Duo CPU @ 2.33GHz

4 GB RAM

Oracle Enterprise Linux 5.1

Oracle 11.1.0.6

Real Application Testing DemoReal Application Testing DemoProduction (wgn1) Standby (wgn1sby)

Data Guard

Real Application Testing DemoReal Application Testing Demo

DGMGRL> convert database 'sid' to snapshot standby;

Standby (wgn1sby)Production (wgn1)

Data Guard

Real Application Testing DemoReal Application Testing Demo

Capture

Standby (wgn1sby)Production (wgn1)

SQLSQL

Data Guard

Real Application Testing DemoReal Application Testing Demo

Prepare

Standby (wgn1sby)Production (wgn1)

Data Guard

Real Application Testing DemoReal Application Testing Demo

Replay

Standby (wgn1sby)Production (wgn1)

SQLSQL

Data Guard

Real Application Testing DemoReal Application Testing Demo

Report & Analyze

Standby (wgn1sby)Production (wgn1)

Data Guard

Real Application Testing DemoReal Application Testing Demo

DGMGRL> convert database 'sid' to physical standby;

Standby (wgn1sby)Production (wgn1)

Data Guard

Value PropositionValue Proposition

NoYes

Integrated with AWR, ASH, SQL Tuning Advisor & SQL

Plan Management

Transfer resolutions to

production

No not integrated with

optimizer

YesTest with

production context

No require explicit polling &

external storage

YesCapture with low

overhead

3rd Party

SQL Testing Tools

Oracle

DB Replay & SPA

Technology that can only be built by Oracle

Value PropositionValue Proposition

To:From:

Complete workflowsPartial workflows

Low riskHigh risk

AutomatedManual intensive

Production workloadsArtificial workloads

Days of developmentMonths of development

DaysWeeks

Database TierComplete Stack

• Oracle.com:http://www.oracle.com/database/real-application-testing.html

• Oracle Real Application Testing on OTN: http://www.oracle.com/technology/products/manageability/database/index.html

• Oracle Real Application Testing white papers: http://www.oracle.com/technology/products/database/oracle11g/pdf/real-application-testing-whitepaper.pdf

http://www.oracle.com/technology/products/manageability/database/pdf/ow07/db_replay_white_paper_ow07_1.pdf

http://www.oracle.com/technology/products/manageability/database/pdf/ow07/spa_white_paper_ow07.pdf

Resources