Download - Oracle OpenWorld 2016. Big Data references
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CustomerPanelDataWarehouseandBigDataSessionID:CON6105
ReinerZimmermannSeniorDirector,DW&BigDataGlobalLeadersProgramDW&BigDataProductManagementSeptember21nd,2016
ConfidenRal–OracleInternal/Restricted/HighlyRestricted
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
JamesHewiWSeniorLeader-EnterpriseDataArchitecture&Strategy
KenHallSeniorInfrastructureArchitect
CarolinaBouvardDirectorofArchitecture,TransformaRonandBusinessIntelligence
GauravSinghSoluRonArchitect(DataWarehouseandBigData)
GunnarSinnBigDataPla_ormManager
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
JimHewiWSr.ManagerDataStrategy&Architecture
linkedin.com/in/JFHewiW
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
DataEnvironments
FinancialServicesFraudDetec8onScoringCardTransacRonsinRealTime
Applica8ons&WarehousingData&AnalyRcs
RevenueManagement&BillingDataGu
ard
DataGuard
Prod Standby Standby
FraudEngine
Prod DR
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
TechnicalDrivers• Mainframeoriented,sharedprocessingandtransacRonreporRngenvironment/notbuiltforlargescaleanalyRcs
• AdhocreporRngachallenge,someanalyRcqueriesonmainframetakinghourstocomplete
• ExisRngEDWonlysummarizeddata• LongdeliveryRmesfornewdataintegraRons,notkeepingupwithdemand
BusinessDriversScalingExis,ngAnaly,cs• Por_olioPerformanceAnalyRcs• OperaRonalProcessingOpRmizaRonAnalyRcs
• ClientAWriRonModelingNewCapabili,es• DataScienceR&DEnvironmentforRapidPrototyping
JourneywithaPurpose
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
Data&Analy8csEcosystem(DnA)
BigDataAppliance Exadata
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
Functional Use • Landing Zone for Raw Data • Data Transformation to RDBMS • Long Term Data Archival • Data Science Discovery & Advanced Analytics
Primary Technologies Used • HDFS, HIVE, Pig, Impala, Parquet, Sqoop, Oozie, HUE, Spark, Cloudera Manager
• R, Python, Scala • Oracle Big Data Connectors
Oracle-BigDataAppliance
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
6BigDataKeyLearnings
1. Deliverquickwinsforthebusiness2. Prototype!Thereare5waystodoonething3. Failfastandmoveon4. Thinkaboutgovernance&securityearly5. Notasilverbullet,BadDataissRllBadData6. TalentiscriRcalforsuccess
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
CarolinaBouvardNuñoArchitecture,TransformaRonandBusinessIntelligenceDirector
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
GauravSinghSoluRonArchitect(DataWarehouseandBigData)
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
WhoisEnergyAustraliaEnergyAustraliainvestsinenergygeneraRonandretailing
OwnandoperateamulRbilliondollarpor_oliosupplyingtheworld’slongestinterconnectedpowergrid
OneofAustralia'slargestandmostexperiencedenergyretailers,servicingover2.6millionhouseholdsandbusinessesacrossAustralia
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
PreTransformaRonAnalyRcalArchitecture
OracleConfidenRal–Internal/Restricted/HighlyRestricted 28
C1 NBVKinetiq B1MDMS
SAPERP
Market
NMM
M
MM
F
C&I
C&I
n/w payment (manual)
accruals
CC&I
CC&I
SAPBW
F
Operational
Analytical
RDW Intellect HDS CIDS EMDMEDW RGMSM
CBFC
CC&I
M
CB
F
M
MC
C&I
C
C&IF
C
B
C&I
BF
Quote
Digital
NMI discovery M
Bill payment history
B
C
C&I
C
Benriched
- Aspect- CentrePoint- Infra
S
B
B
BB
PMDS MADB & PFDB PARS
SAPBPC
HRIS Maximo
F
F
F
F
MW
Info-server
W
W
W
Campaign Tool
C
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
TransformaRonEnergyAustralia’sInformaRonManagement(IM)AugmentedDataWarehouseArchitecture
Source Store
Structured Data
Big Data Store(Distributed File Clusters)
Real
-tim
e Ev
ent
Cap
ture
Master Data ManagementData Quality
Metadata
Master Data Store
Meter Data
(Usage)
SDS
SDS
SDS
Customer
Analytical datasets
Feature vectors
Meter
Market
Other
SDS
.
.
Consume
Traditional Business Intelligence
Data Discovery
Self Service Legal / Compliance
Descriptive Analytics
Management Reports
Pattern DetectionData Visualisation
Downstream FeedsOperational
SystemsLegacy
WarehousesOther
Financial Reporting and AnalysisRisk
ManagementFinancial
planning and budgeting
Financial modelling
and forecasts
Operational Analytics
Channels
Web
Multichannel Campaign
Management
Real Time Decision Engine
Call CentreShopEmail
Digital Messaging Platform
SMS/ MMS
Mobile Apps
3rd Party Interfaces
Other
Advanced AnalyticsPrescriptive Analytics
Descriptive Analytics
Predictive Analytics
ProcurementPerformance Management
Ingest
Structured
C1
B1
Kinetiq
External
Retail Meter Data
Wholesale Meter Data
Wholesale Market Data
3rd Party Data
Semi-structured
EA Websites
EA Mobile Apps
Social Media
EA Call Center
Discovery Data Store
(Data Labs)
Big Data OperationalData Store
Analytical Data StoreSource
Data Store
Operational Systems
…etc
BIGovernance(acquisiRon,integraRon,transformaRon,needs-aligneddelivery,innovaRon)
IMGovernance(end-to-end,mulR-domain,strategicalignment,enterprisecontext)
GovernanceFoundaRon
TechnicalFoundaRon
BusinessAnalyRcalCapabiliRes
Key
AcceleratetechnicalfoundaRondevelopment
EstablishinformaRongovernance
SetupCustomermasterstore
TransiRonMeterdatatofuturestate
EstablishSourceDataStores
EstablishdatadiscoveryforadvancedanalyRcsusingtheHadoopEcosystem
DevelopnewbusinesscapabiliRes• CustomerandMeterAnalyRcs• ProducRvityandCostAnalyRcs• EnergyAssetmanagement
MigrateexisRngdatastores
Dataquality–customerandmeter
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
TechnicalSpecificaRons2x(¼+⅛)RackX5-2&X6-2(8TBspindle)Exadata(Primary+Secondary)
- PCIFlash:19.2TB
- DiskCapacity:54TB
- Cores:18
- Memory:512GB
- 2000StaffconnecRonsand2.5MillionCustomerconnecRons
- 18BillionRecords+
2xStarterRackX5-2BigDataAppliance
-6xComputeNodes
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
• 179 Years old this year • Countries of operation: ~70 • Countries where are brands are sold:
~180 • Consumers served by our brands: 5 billion (approximate)
• Last FY Revenue: $70.7 billion
AboutProcterandGamble…Business Description: The Procter & Gamble Company (P&G) is a leading manufacturer and marketer of branded consumer packaged goods across the world. P&G manufactures, markets and distributes premium quality, branded beauty, health, fabric, home, baby, family, and personal care products.
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
Source&DataApplicaRon DataAcquisiRon BaseDataStorage AnalyRcs Reports&Conclusion
Data Data InformaRon Knowledge Wisdom
RetailerData
ShopperData
MarketData
&more
InternalData
ExternalData
BigUniverseOnPREM
BigUniverseExernalCloud
DomainExpert
Analyst
EndUser
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
OpenSource
3
PLATFORM
LINUXOPERATINGSYSTEM
CLOUDERAMIDLAYER
CUSTOMOPEN
SOURCE
EDGENODES
CLUSTER
e.g….
DATASOURCESDATASOURCES
DWOR DWSAP EXT
CLUSTERs CLUSTERserverforstorageandprocessingORACLE ORACLE ORACLE
EDGE(UTILITY)NODES
ONPREM CLOUDA CLOUDB
Dedicatedservertoaccessclusterservices
BISERVER DedicatedserverforBIapplicaRons
CLOUDA CLOUDBONPREMBITOOLLAYER
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
AnalyRcalUseCases• SellingOpera8ons:IntegraRngstore,product&consumerdatatodevelopmodels&simulaRonswhichgenerate
revenueopportuniResinassortment,pricing*,promoRonandshoppingbehavior.
• ConsumerEngagement:OpRmizingconsumerengagement,purchaseintentandconversionbymodelingand
simulaRngtheironandoff-linebehaviordrivingupandcrosssellingopportuniRes.
• SupplyChainOpera8ons:Transformingoursupplynetwork,andoperaRonsthroughbigdataanalyRcstosave
inventorycosts,idenRfydriversofmissedcasesandincreaseoverallagilityandresponsiveness.
• BusinessManagement:ForecasRngresultsofnewiniRaRvesdaysintolaunch,assessingourabilitywintarget
consumers,anddrivingasignificantincreaseintradeandmediaefficiency.*pricingisatthesolediscreRonofretailers
Copyright©2016,Oracleand/oritsaffiliates.Allrightsreserved.|
JamesHewiWSeniorLeader-EnterpriseDataArchitecture&Strategy
KenHallSeniorInfrastructureArchitect
CarolinaBouvardDirectorofArchitecture,TransformaRonandBusinessIntelligence
GauravSinghSoluRonArchitect(DataWarehouseandBigData)
GunnarSinnBigDataPla_ormManager