delphix for dbas by jonathan lewis

Saving time and reducing risk with Delphix

Jonathan Lewisjonathanlewis.wordpress.com

www.jlcomp.demon.co.uk

Delphix for DBAs2 / 36

Jonathan Lewis © 2013 - 2014

Who am I ?Independent Consultant

31+ years in IT26+ using Oracle

Strategy, Design, Review,Briefings, Educational,Trouble-shooting

Oracle author of the year 2006Select Editor’s choice 2007UKOUG Inspiring Presenter 2011ODTUG 2012 Best Presenter (d/b)UKOUG Inspiring Presenter 2012UKOUG Lifetime Award (IPA) 2013Member of the Oak Table NetworkOracle ACE DirectorO1 visa for USA



Topics

• Testing requirements

• Virtual Databases

• Basic DxFS

• Oracle Backups

• Possible uses



Testing is difficult (a)

Last night this report took 45 minutes instead of 3.

Why ? Make sure it doesn't do it again tonight.

It's not instrumented properly.

It didn't get captured in any AWR snapshots.There are fragments in historic ASH

It seems to be running as expected right now

Maybe it was resource contention - any clues in AWR.

Maybe it was a change in plan - how to prove it.

What's the fastest way to eliminate possibilities ?



Testing is difficult (b)

I believe I can make this part of the batch run six times faster - when can I test it ?

You can have pre-prod at 3:00 am the day after tomorrow.

But it's going to take 2 hours to prime the data correctly

(provided nothing goes wrong while we're priming it)

Give me 5 minutes and you can have a copy of prod as it was last night at the correct point in the batch

… and the same for the previous night

… and the night before



Testing is difficult (c)

The best place to test is on productionRight data, right stats, right hardware, right activity

End-users may disagree.If nothing else competition for resources is a threat

Next best is a "good" copy of productionStill unlikely to emulate performance characteristics

Usually adequate for "sanity" checks

How "good", how often, how quickly, how private ?

The best way to avoid big mistakes in development is to test on a carefully timed full-scale copy of the production data.



Testing is difficult (d)

Warning:

Even if you had a disk-based clone of production on exactly the same hardware across the entire stack you still have to worry about the impact of:

No matter how good your test system there are always imponderables that require some careful thought when you try to interpret test results.

Basic concurrency effects

Oracle-related resource contention

Read-consistency

Historic caching effects

Other people's tests

The target is to maximise your level of confidence



Virtual DB

Avoids duplicating data storage

One full, read-only copyCould be a "logical snapshot" of production

Probably better to use a copy

Plus N modifying instances"local" copies of blocks the instance has modified

If you have only one development project a very recent backup may be adequate, but if you have many projects it's best to keep them separate.



Options (a)

Production DB

It is possible to have a solution that takes a "logical copy" of the production system on the same SAN, and then writes modified blocks to a separate area

Dev Delta"Logical copy"

ProductionServer

DevelopmentServer

Storagedevice



Options (b)

Production DB

You could use "split mirror" technology to make a fast, cheap, local copy of Production and associate multiple deltas with it.

Dev1 Delta Dev2 DeltaSplit Mirror



Options (c)

Production DB

You could take a copy of production to another storage device, and then use the "delta" technology there. This helps to protect your production system.

Dev1 Delta Dev2 DeltaBackup Copy



Minimum impact on production systemDelphix uses rman backups with log transport

Ease of creating/discarding a virtual databaseVery simple once target machine is setup

Speed of refreshing "primary" copyThe ideal would be a "rolling-history" mechanism

Minimum impact on production system

Ease of creating/discarding a virtual database

Speed of refreshing "primary" copy

Considerations



My SandboxWindows 7, VMWare Server 2.0 - 16GB RAM, 4 CPU, 2 x 500GB drives

OEL 5.6Oracle 11.2.0.4

Production: s/w and databasewith "delphix" a/c

Delphix 3.0

Backups and virtual databases

OEL 5.6Oracle 11.2.0.4

Development:s/w and config fileswith "delphix" a/c



Basic architecture

Independent system - running DxFS.

Acts as a network file server for vDBs

Registers source and destination machinesTakes rman backups from source machines

Uses transparent compression to save space

Keeps rolling backups using minimum space

Mounts backups on destination machinesUses DxFS technology to present virtual DBs

Best effect - when a small fraction of the database changes



DxFS

a b c d e f g h i

Data (i.e. filesystem blocks)

128 entries per block

Variable number of sectors per block

Set the DxFS block size to match the database block size. DxFS blocks are transparently compressed to the smallest number of sectors possible.

t0

Metadata(associates logical block with physical location)



a b c d e f g h i

"Updates"

b' c'

Data only becomes visible when root block is written

You don't overwrite blocks on disc with a new version, you write a new copy and maintain the index (and you don't overwrite index blocks either).

t0 t1



a d e f g h i

Multi-version

b' c'

This is the latest state of our file system - but we don't have to release the old versions of the blocks we "changed".

t1



Aging data

Blocks that can be freed when we don’t want the older snapshot any more

a d e f g h i

b' c'

b c

Over time we may "overwrite" many of the most recent blocks, and be able to free old copies of those blocks, keeping a lot of the oldest original blocks

t1t0



Working with Oracle

• Start with an rman backup

• Take regular incremental backups– Delphix has custom code to apply them efficiently

• For a vDB pick a recovery point– Delphix code mounts the file system, configures the

target, and sets up the instance.

Time for a demonstration: (identify source and target machines, identify source d/b, create first backup, create vDB).



Traditional Backups

Level 0

Level 1

Data Redo

Level 1

Data Redo

Level 1

Data Redo

Redo



Delphix calls rmanCONNECT TARGET /SET ECHO ON;SET ENCRYPTION OFF;RUN {

SET NOCFAU;ALLOCATE CHANNEL DLPX_CHANNEL_0 DEVICE TYPE sbt … ;ALLOCATE CHANNEL DLPX_CHANNEL_1 DEVICE TYPE sbt

PARMS='SBT_LIBRARY=/home/oracle/toolkit/Delphix_.../libobk.so,ENV=(

DELPHIX_CONFIG_DIR=/home/oracle/toolkit/Delphix_.../lib64,

DELPHIX_URL=http://192.168.91.167:8341/context/servlet,DELPHIX_HID=SNL.1.17.F76CB2C54939EE8FF470FA

)'

;SET COMMAND ID TO 'SNL.1.17.F76CB2C54939EE8FF470FA';BACKUP NOEXCLUDE FORCE COPIES 1 AS BACKUPSET

FILESPERSET 5 TAG 'DLPX201306181730521301FE51F5B5' ARCHIVELOG FROM SCN 6726134

;}exit



Delphix Backups

Level 0

Level 1

Data Redo

Level 1

Data Redo

Level 1

Data Redo

Redo

Delphix unpacks the rman backups as they are generated and applies them (excluding redo) to the level 0 backup.



Delphix Backups

DxFS doesn't overwrite changed blocks, though, it writes them to a separate location and creates some new index entries.



Delphix Backups



Pick your version

Pick an index to create your vDB

Rolling forward creates some new (f/s) blocks and pointers, and a new root block.Every new vDB gets its own "index root block".

Redo Redo Redo

Delphix rolls forward (and back) from the associated redo

Redo



Rolling history

Redo

To delete history Delphix can do an optimised walk of "old" indexes and delete any blocks for which a newer block has been created.



Options

• Snapsync– Take regular rman backups "from SCN"– vDBs open at granularity of snapshots

• Logsync– Take copies of archived redo log– vDBs roll forward and open at any desired SCN– Frequent snapsync operations reduce time to roll forward – Can do "pre-provisioning" (continuous rollforward)



Solutions (a)

Last night this report took 45 minutes instead of 3.

Why ? Make sure it doesn't do it again tonight.

Create a vDB as at report start time last night.

It's got yesterday's data and yesterday's stats

Run the report.

Does it take 2 hours or 3 minutes ?



Solutions (b)

I believe I can make this part of the batch run six times faster - when can I test it ?

Create a vDB as at the right moment from last night.

Run the old code

Create a vDB as at the right moment from last night.

Run the new code

Check results match, check workload

Rinse and repeat for every night last week / month.



Ideas (a)

• Stats Tests– Create a vDB to generate / fake different statistics

• E.g. histograms, global partition code strategies

– Run test-suite

– Transfer stats to production if appropriate

• Index tests:– Create a vDB to gauge the side effects of index changes

– Some idea of time to build.



Ideas (b)

• Upgrades / Patches:– Create a vDB from production

– Create a new Oracle Home for new s/w version

– Run upgrade

• Single instance conversion to RAC– you install the clusterware etc.

– Delphix will give you the database and instances



Ideas (c)

• Hand-off to QA– Create a vDB

– Apply and test development changes

– Create vDB from development vDB for QA to test

– Continue development

• Rinse and repeat

• Drop "spare" QA vDBs



Ideas (d)

• Obfuscation:– Create a vDB from production

– Run obfusaction code.

– Create vDB(s) from obfuscated vDB.

• Probably more restrictive – But time for obfuscation needed only once per "generation"

– (obfuscate once, use many)



Performance

• Under any circumstances, changes in discs, CPU, memory, and users make it difficult to compare performance. – Check statistics - SQL by reads, segments by reads

• Sharing (cached) blocks on the Delphix may make some I/O faster.

• CoW may make tablescans / IFFS slower

• Effects on writes may vary with scatter



Conclusion

• Questions to ask about virtual databases– How much extra workload (at source)

– How much space needed (and where)

– Simplicity of generation

– Simplicity / overhead of refresh

• Versatility– One big backup is good for everything !

• Interpretation of performance

delphix for dbas by jonathan lewis

Software

usajonathan lewis

dxfsjonathan lewis

copyjonathan lewis

delphix fordbas3

delphix fordbas11

delphix fordbas10

delphix fordbas9

delphix fordbas6