oracle database in-memory - doag deutsche · pdf fileoracle database in-memory: ... product...

OracleDatabaseIn-Memory

AndyRivenesDatabaseIn-MemoryProductManagerSystemsTechnologyGroupDOAG- November2016

EnablingReal-TimeAnalytics

SafeHarborStatementThefollowingisintendedtooutlineourgeneralproductdirection.Itisintendedforinformationpurposesonly,andmaynotbeincorporatedintoanycontract.Itisnotacommitmenttodeliveranymaterial,code,orfunctionality,andshouldnotberelieduponinmakingpurchasingdecisions.Thedevelopment,release,andtimingofanyfeaturesorfunctionalitydescribedforOracle’sproductsremainsatthesolediscretionofOracle.

Date Title Location SpeakerTuesdayNov15th11:00-11:45

OracleDatabaseIn-MemoryOption

Seoul AndyRivenesProductManager,Oracle

WednesdayNov16th08:00-08:45

OracleDatabaseIn-Memory:ADeepDiveandaFuturePreview

Tokio TirthankarLahiri–VicePresident,Oracle

WednesdayNov19th11:00-11:45

UsingADOwithDatabaseIn-MemoryandStorage-Based Tiering

Tokio GreggChristmanAndyRivenesProductManagers,Oracle

OracleDatabaseIn-MemoryScheduleforDOAG

WhatisDatabaseIn-Memory

OracleDatabase In-MemoryGoals

Real-TimeAnalytics TrivialtoImplement

NoApplicationChangesNotLimitedbyMemory

AccelerateMixedWorkload

AnalyticsTransactions

RunanalyticsonOperationalSystems

EnableReal-TimeBusinessDecisions

Real-TimeAnalytics100X

Risk-Free

ProvenScale-Out,Availability,Security

Breakthrough:DualFormatDatabase

• BOTH rowandcolumnformatsforsametable

• Simultaneouslyactiveandtransactionally consistent

• Analytics&reportingusenewin-memoryColumnformat

• OLTPusesprovenrowformat

BufferCacheNewIn-MemoryColumnStore

SALES SALESRow

FormatColumnFormat

Howeasyisittogetstarted

OracleDatabase12c• DatabaseIn-MemoryisanoptionforOracleDatabase12cEnterpriseEdition

• DatabaseIn-Memoryisincludedinthefirstpatchset (12.1.0.2)for12.1

• UsethelatestDatabaseProactiveBundlePatch–NotPSUsorCPUs

• Noadditionalstepsrequired

Note:DatabaseIn-Memoryisnot enabledbydefault

Oracle Database In-Memory: Simple to Implement

1. Configure Memory Capacity• inmemory_size = XXX GB

2. Configure tables or partitions to be in memory• alter table | partition … inmemory;

3. Later drop analytic indexes to speed up OLTP

OracleIn-MemoryAdvisor• NewIn-MemoryAdvisor• AnalyzesexistingDBworkloadviaAWR&ASHrepositories

• ProvideslistofobjectsthatwouldbenefitmostfrombeingpopulatedintoIMcolumnstore

Note:Diagnostic&TuningPacksrequired

Howdoesitwork

OracleIn-MemoryColumnarTechnology

PureIn-MemoryColumnar• Purein-memorycolumnformat

• Enableforsubsetofdatabase

• Cheaptomaintain– nologgingorIO• AllowsefficientOLTP

• Nochangetodiskformat

• Builtseamlessly intoOracleDatabase• Appearsasanewstoragetype• Transparent toApplications• AllEnterpriseFeatureswork..

• Availability– RAC,Flashback,DataGuard,etc.• Security– Encryption,Auditing,etc.

In-MemoryArea:NewStaticAreawithinSGA

SystemGlobalAreaSGA

BufferCache

SharedPool

LogBuffer

LargePool

In-MemoryAreaOther

• ContainsdatainthenewIn-MemoryColumnFormat

• ControlledbyINMEMORY_SIZEparameter- Minimumsizeof100MB

• SGA_TARGETmustbelargeenoughtoaccommodatethisarea

CompositionofIn-MemoryArea

• Containstwosubpools:– IMCUpool:StoresInMemoryCompressionUnits(IMCUs)

– SMUpool:StoresSnapshotMetadataUnits(SMUs)

• IMCUscontaincolumnformatteddata

• SMUscontainmetadataandtransactionalinformation

SMU SMU

IMCU IMCU

IMCUIMCU

IMCU IMCU

IMCUIMCU

InMemoryArea

ColumnFormatData

Metadata

CompositionofIn-MemoryCompressionUnit(IMCU)

• Unitofcolumnstoreallocation– Columnarrepresentationofalargenumberofrowsfromanobject

– Rowsfromoneormoretableextents

• Actualsizedependsonsizeofrows,compressionfactor,etc.

• EachcolumnstoredasaseparatecontiguousColumn CompressionUnit(columnCU)– Rowids alsostoredasaColumnCU

IMCUheader

Extent#13Blocks20-120

ROWIDEMPID NAME

DEPT SALARYColumnCUs

Population:Compression• Objectscompressedduringpopulation

• ControlledbyMEMCOMPRESSsubclause

• 2xto20xcompressiontypical

• Multiplelevelsofcompression

- FORDML

- FORQUERY(Default)

- FORCAPACITY

• OrderinwhichobjectsarepopulatedcontrolledbyPRIORITYsubclause:ALTER TABLE sales INMEMORY PRIORITY HIGH;

• Levels:- CRITICAL>HIGH>MEDIUM>LOW

- Controlsorder(notspeed)ofpopulate

• DefaultPRIORITYisNONE- Populateonlyonfirstaccess

• Populationcompletedbynewsetofbackgroundprocesses

ora_w001_orcl

• Numberofprocessescontrolledbyparameter:INMEMORY_MAX_POPULATE_SERVERS

Population:Priority

WhyAreAnalyticQueriesFasterInTheIn-MemoryColumnStore?

ScanningandfilteringdatamoreefficientlyReal-TimeAnalytics

StorageIndexesCompression

Scan&filterdataincompressedformat

Accessonlythecolumnsyouneed

ColumnarFormat SIMDVectorProcessing

ProcessmultiplecolumnvaluesinasingleCPUinstruction

VectorRegister

Loadmultipleregionvalues

VectorCompareallvaluesan1cycle

Min1Max3

Min4Max7

Min8Max12

Pruneoutanyunnecessarydatafromthecolumn

• Speedofmemory• ScanandFilteronlytheneededColumns

• VectorInstructions

Real-timeAnalyticsImprovesallaspectsofanalyticqueries

DataScans

VectorRegisterCPU

In-MemoryAggregation

•CreateIn-MemoryReportOutlinethatisPopulatedduringFastScan

•RunsReportsInstantly

•ConvertStarJoinsinto10XFasterColumnScans

•Searchlargetableforvaluesthatmatchsmalltable

HASHJOIN

TableA TableB

STATE=CA

What’sNew

• Availablenow– Exadata ExpressCloudService

• Comingsoon– DatabaseCloudServices– Exadata CloudMachine

Announcing OracleDatabase12cRelease2onOracleCloud

OracleispresentingfeaturesforOracleDatabase12cRelease2onOracleCloud. WewillannounceavailabilityoftheOn-Prem releasesometimeafterOpenWorld.

• Analyticquerieshavecomplexjoinswithnofilterpredicatesspecified

• JoinGroupspecifiescolumnsusedtojointables– Columnsharecompressiondictionary– Joinsoccuron dictionaryvaluesratherthandata

• Enables2-3xfasterjoinprocessing

Real-TimeAnalytics:FasterIn-MemoryJoins

VEHICLENAM

Example:FindsalespriceofeachVehicleSalesVehicle

NAMEisjoincolumnN

CREATE INMEMORY JOIN GROUP V_name_jg(VEHICLES(NAME),SALES(NAME));

• Analyticqueriescontaincomplexexpressions– Originallyevaluatedforeveryrow

• Expressions pre-computedandcachedin-memory–Userdefinedviavirtualcolumns–Or expressionsautomaticdetected

• AllIn-Memoryoptimizationsapply• 3-5x fastercomplexqueries

Real-TimeAnalytics:In-MemoryExpressions

Net=Price+Price*Tax

In-MemoryColumnStore

Example:ComputetotalsalespriceTax

Price+PriceXTax

Price+Price*Tax

• FullJSONdocumentspopulatedusinganoptimizedbinaryformat

• AdditionalexpressionscanbecreatedonJSONcolumns(e.g.JSON_VALUE) &storedincolumnstore

• QueriesonJSONcontentorexpressionsautomaticallydirectedtoIn-Memoryformat- e.g.Findmovieswheremovie.name

contains“Jurassic”

• 60x performancegainsobserved

Multi-ModelAnalytics:In-MemoryJSON

Relational

In-MemoryColumStore

In-MemoryVirtualColumns

In-MemoryJSONFormat

{"Theater":"AMC 15","Movie":”Sully","Time“:2016-09-09T18:45:00","Tickets":{

"Adults":2}

Relational Virtual JSON

• AutomaticDataOptimizationextendedtoIn-Memory

• In-Memorypoliciesallowoptimaluseofcolumnstorebasedonheatmap- Requiresonlyinmemory option,not

advancedcompression

• Policiescanbedefinedto:- BringdataintotheIMcolumnstore- Increasecompressionlevelsasdatacools

- EvictcolddatafromIMcolumnstore27

Sales_Q3

Sales_Q2

Sales_Q4

In-MemoryColumnStore

Sales_Q1

Automation:In-MemoryDataAutoPopulationPolicies

HowdoesitimpactOLTPenvironments

• ComplexOLTPisSlowedbyAnalyticIndexes

• Insertingonerowintoatablerequiresupdating10-20analyticindexes:Slow!

• ColumnStoreReplacesAnalyticIndexes

• Fastanalyticsonany columns

• ColumnStorenotpersistentsoupdatecostismuchlower

DatabaseIn-MemoryAcceleratesMixedWorkloads

Table1– 3OLTP

Indexes

10– 20AnalyticIndexes

REPLACE

DatabaseIn-MemoryandotherOracleDatabasefeatures

DatabaseIn-Memory:ScalestoAnySize

Scale-Up

•Scale-UponlargeSMPs•NUMAOptimized

•Scale-OutAcrossServerstoGrowMemoryandCPUs

• In-MemoryQueriesParallelizedAcrossServers

Scale-Out CombinewithFlashandDisk

DISKPCIFLASHDRAM

ColdDataHottestData

ActiveData

•Easilyplacedataonmostcosteffectivetier

•SimultaneouslyAchieve:•Speed ofDRAM• I/OsofFlash•Cost ofDisk

Data Guard & GoldenGate

• Pure In-Memory format does not change Oracle’s storage format, logging, backup, recovery, etc.

• All Oracle’s proven availability technologies work transparently

• Protection from all failures- Node, site, corruption, human error, etc.

DatabaseIn-Memory:IndustrialStrengthAvailability

• Similar to storage mirroring

• Duplicate in-memory columns on another node- Enabled per table/partition

• e.g. only recent data

- Application transparent

• Downtime eliminated by using duplicate after failure

OnlyAvailableonEngineeredSystems

DatabaseIn-Memory:UniqueFaultTolerance

DatabaseIn-Memory:TrivialtoImplement

SimpletoConfigure

NocomplexsetupSetcolumnstoresize

DeclareIn-Memorytables

FullFunctionality

NoSQLrestrictions

100%Compatible

Noapplicationchanges

Nodatamigration

EasytoDeploy

HowarecustomersusingOracleDatabaseIn-Memory

AT&TWiFi – DataWarehouse• BusinessObjectsreports100X faster

• ETLprocessesimprovedby50% faster

• NochangestoSAPBusinessObjectsreports

Villeroy &Boch – SAPBW• SAPBWCOPAqueries30– 33X faster

• SAPTransactionlistqueries4– 4,800Xfaster

• Avoidedexpensive&riskyupgradetoS4/Hana

BOSCH – SAPCRM• Dropped allcustomindexes

• Analyticqueries2-20Xfaster,DML2-3X faster

• Nochangestoapplicationrequired

DieMobiliar – MixedWorkload• RemovalofNetezza system

• Analyticqueries50-200X faster

• Databasesizereduced considerably

DatabaseIn-MemoryReferences

In-MemoryUseCasesOLTP

• Real-timereportingdirectlyonOLTPsourcedata

• RemovesneedforseparateODS• Speedsdataextraction

DataWarehouse• Staging/ETL/Tempnotacandidate

• Writeonce,readonce• AllorasubsetofFoundationLayer

• Fortimesensitiveanalytics• PotentialtoreplaceAccessLayer

WherecanIgetmoreinformation

JointheConversation

https://twitter.com/TheInMemoryGuyhttps://blogs.oracle.com/in-memory/

WhitePapers(otn.com)• OracleDatabaseIn-MemoryWhitePaper• OracleDatabaseIn-MemoryAggregationPaper• WhentouseOracleDatabaseIn-Memory• OracleDatabaseIn-MemoryAdvisor

Videos• OracleDatabaseIn-MemoryYouTubeChannel• oracle.com- PoweringtheReal-TimeEnterprise- IndustryExpertsSharePerspectivesoracle.com/us/corporate/events/dbim/index.html

• YouTube- JuanLoaiza:SoftwareinSilicon

AdditionalQuestions• In-Memoryblog:blogs.oracle.com/In-Memory• Myemail:andy.rivenes@oracle.com

https://www.facebook.com/OracleDatabasehttp://www.oracle.com/goto/dbim.html

AdditionalResources

oracle database in-memory - doag deutsche · pdf fileoracle database in-memory: ... product...

Documents

reuse in oracle soa suite 12c - doag deutsche oracle ... ·...

oracle soa suite 11g performance tuning - doag

doag 2016 oracle logon security

oracle database appliance, doag regio berlin brandenburg,...

doag 2012 oracle endeca

high-performance oracle platforms - doag

safe harbor statement - doag deutsche oracle ... · •db...

sap auf oracle datenbank 12c release 1 - doag

oracle oem...

sap on oracle: development update - doag

oracle e-business suite release 12 - doag

best practice oracle database 12c upgrade - doag ·...

oracle no sql-doag-datenbank_konferenz_juni_2014

doag 2014 konrad-haefeli_in-memory-option-on-exadata

oracle engineered systems - doag

oracle cloud platform services - doag

doag oracle 12c new security features soe

doag 2010: mysql architekturen für oracle dba's

oracle data integrator 12c - doag

oracle hadoop doag-big-data_09_2014_gpi