verizon centralizes data into a data lake in real time for analytics
TRANSCRIPT
© 2017 Attunity
Sanjay Kumar
General Manager, Telecom - Hortonworks
Arvind RajagopalanDirector – Global Technology Services – Verizon
Jordan MartzDirector of Technology Solutions - Attunity
Guest Speakers
© 2017 Attunity
• Hortonworks - Hortonworks Data Platform (HDP) for the telecommunications market
• Verizon - How we centralized data into a data lake for analytics
• Attunity - Migrating real-time data
• All - Q&A
Agenda
5 ©HortonworksInc.2011– 2016.AllRightsReserved
TheNewLandscapeoftheTelecomIndustry
ServiceProvidersSocialMedia&Mobile:ExplosionofrichcustomerdatathroughSocialMedia
andMobileAppsforcustomersentiment&
Interests
CustomerExpectation:Withtheculturalimpactofwebandmobile,customersareexpectinggreater levels
ofserviceandresponsiveness
CompetitiveDifferentiation:Asotherserviceprovidersdeliver
similar levelsoftelecomserviceandcoverage,otherareasofservice levels
areneeded
NewDigitalEcosystem:GreatervalueofDataondigitalecosystemfor
CustomersandPartnersdrivingDataMonetization
InternetOfThings:ExplosionofdatafromIOTwithbenefitsaligned
withinsightnotcorrelated todata
volumes
6 ©HortonworksInc.2011– 2016.AllRightsReserved
Service Providers Areas of FocusCustomerExperience&Marketing- EnhanceEnd-to-end ExperienceofCustomer- BecomeTrustedPartnertoCustomer- Awarenessofcustomer’sneedswhenandwhereneeded
NewBusiness&ConsumerServices- NewDigital&InfrastructureServices- DataMonetization- M2M,IoT,Analytics-as-a service
NetworkOptimization- MovetoSoftwareDrivenNetworks- LeverageNetworkDataAssets- Selfoptimizingandprovisioning Customer
Network Service
7 ©HortonworksInc.2011– 2016.AllRightsReserved
DATA-CENTRIC
OldModel:App-Centric
Limitations:• Multiplecopiesofdata• Difficultcross-systemintegration• Upper-limitondatavolumes
beforeharmingperformance
NewModel:Data-Centric
Advantages:• Oneversionofthedata• Noneedforcross-appintegration• Systemscaleslinearly
APP-CENTRIC
App1 App2 App3 App4 App5 App6
AppCentricwillbreakdownwithx10,x100,x1000…NeedtoshifttoDataCentric
8 ©HortonworksInc.2011– 2016.AllRightsReserved
TheFutureofDataActionableIntelligence
D ATA I N M O T I O N
STORA
GEST
ORA
GE
GROUP2GROUP1
GROUP4GROUP3
D ATA AT R E S T
INTERNETOF
ANYTHING
CombiningData-in-MotionandData-at-RestpowersActionableIntelligence
9 ©HortonworksInc.2011– 2016.AllRightsReserved
StreamingInjestion
(HDF)
Streaming:NetworkProbes,ClickStream,Sensor,Location
Batch:CallDetailRecords
On-Line:CustomerSentiment
Unstructured:Txt,Pictures,Video,Voice2Text
Clickstream
Web&Social
Geolocation
EventRecords
CallLogs
Files,emails
ServerLogs
ExistingOSS/BSS/DWHApp1 App2 App3 App4 App5 App6
IOT Solutions
Analytic Services
Network Planning
OptimizationContextual Marketing
360 Customer Awareness
DataDrivenBusiness
DELIVERY
PersonalDataAnalysis&CustomerInsightServicesToCustomers&Partners
10 ©HortonworksInc.2011– 2016.AllRightsReserved
SocialMedia
SentimentCallCenterInteraction
Lifestyle&Interests
StreamingInjestion
(HDF)
Streaming:NetworkProbes,ClickStream,Sensor,Location
Batch:CallDetailRecords
On-Line:CustomerSentiment
Unstructured:Txt,Pictures,Video,Voice2Text
HowtoSeeWhattheCustomerSeesandRespondwithAwareness
– ViewfromTheCustomer(360degreeview)• QualityofServiceExperience (Networkdetails)• CallCenterdialog(voicetotextanalysis)• CustomerInteractionChannel(CRM,Billing)• Customer’sSentiment(whattheyaresayingonsocial
media)
– UnderstandingCurrentStateoftheCustomer• Allkeycustomermetricsattributesandreferences
withinadynamiccustomerprofile• NetPromoterScore(dynamicallycomputed)• ChurnScore• AppetiteforInformation&otherCustomerMetrics• TargetAdvertisingProfile
– NextBestActions• Contextawarepromotions• Churnriskresponse• Retail StorewithStateoftheCustomer• Omni-channelcorrelatedactions
QualityofService
viaNetwork
CustomerExperience
ViaBSS
11 ©HortonworksInc.2011– 2016.AllRightsReserved
BusinessFunctions
CustomerGo-to-market
CyberSecurity
Hadoop-as-a-service
M2M/IoTSolutions
PersonalIntelligence
IndustryBrokering
ITOperationsEDWDataOffload HDP
LandingZoneHDP
DataLakeReal-timeStreaming
HDPDataLake
360 CustomerHousehold
View
Dynamic Customer
Profile
Next BestAction
Location &Context Based
CEM- CustomerExperience
Target Audience
Profile
Context Aware Target
Marketing
Location & Context Based
Promotions- Marketing
ThreatDetection
Real-TimeThreat
Analytics
DynamicPattern
Detection- Security
Next BestNetwork
Investment
EquipmentPredictive
Maintenance
Customer Aware Network
Provisioning
- Network
Customer Sentiment &
Churn
Dynamic Network
Provisioning
NetworkDecommissioning
© 2017 Attunity
CRM
Streaming Injestion(HDF)
Call Center Voice to Text
Social Media Feeds (Twitter..)
CDRs or Network Probe
© 2017 Attunity
Customer Sentiment &
Churn
360 CustomerHousehold
View
Dynamic Customer
Profile
Next BestAction
Location &Context Based
CEM
Target Audience
Profile
Context Aware Target
Marketing
Location & Context Based
Promotions
© 2017 Attunity
Attunity Corporate Overview
Data Integration & Big Data Management Software
Accelerate data delivery and availability
Automate data readiness for analytics
Optimize data management with intelligence
§Hadoop & Big Data
§Databases & Data Warehouses
§On premise & in the Cloud
Solutions for: Global OfficesOverview
§2000 customers in 65 countries
§250 people and growing
§NASDAQ traded (ATTU)
© 2017 Attunity
Seamless integration with Hortonworks Connected Data platforms and solutions
HortonworksConnection
Hortonworks Solutions
Enterprise DataWarehouse Optimization
Cyber Security andThreat Management
Internet of Thingsand Streaming Analytics
Hortonworks ConnectionSubscription SupportSmartSensePremier SupportEducational ServicesProfessional ServicesCommunity Connection
CloudHortonworks Data CloudAWS HDInsight
Data CenterHortonworks Data Suite
HDFHDP
© 2017 Attunity
Real-time Data Ingest with Attunity Replicate
SOURC
ES
OLTP,ERP,CRMSystems
Documents,Emails
Web Logs,Click Streams
SocialNetworks
MachineGenerated
SensorData
GeolocationData
Attunity Replicate for HDP & HDF
Accelerate time-to-insights by delivering solutions faster, with fresh data, from many sources - Automated data ingest - Incremental data ingest (CDC)- Support for multiple sources
© 2017 Attunity
Attunity Replicate architecture
Transfer
TransformFilterBatch
CDC Incremental
In-Memory
File Channel
Batch
Hadoop
Files
RDBMS
Data Warehouse
Mainframe
Cloud
On-prem
Cloud
On-prem
Hadoop
Files
RDBMS
Data Warehouse
Kafka
Persistent Store
© 2017 Attunity*Supported under early access program
Attunity Replicate sources and targets
RDBMS
OracleSQL ServerDB2 iSeriesDB2 z/OSDB2 LUW MySQLPostgreSQLSybase ASEInformix
DW
ExadataTeradataNetezzaVertica
HortonworksClouderaMapR
Hadoop
DB2 for z/OSIMS/DBVSAMSQL M/PEnscribeRMSHP NonStop
Mainframe
AWS RDSSalesforce
Cloud
RDBMS
OracleSQL ServerDB2 LUWMySQLPostgreSQLSybase ASEInformix
DW
Microsoft PDWExadataTeradataNetezzaVerticaSybase IQAmazon RedshiftActian VectorSAP HANA
HortonworksClouderaMapRPivotalAmazon EMR
Hadoop
MongoDB
NoSQL
Amazon RDSAmazon RedshiftAmazon EMRGoogle Cloud SQLGoogle Cloud DataprocAzure SQL Data WarehouseAzure SQL Database
Cloud
Azure Event Hubs*Kafka
Messaging
Targets
Sources
SAP
ECC on OracleECC on SQLECC on DB2*
SAP
HANA
© 2017 Attunity
In Memory and File Optimized Data Transport
CDC for data-at-rest and data-in-motion
R1R1R2R1R2
R1R2
Batch CDC
Data WarehouseIngest-Merge
SQLn 2 1
SQL SQLTransactional CDC
Message Encoded CDC
Data Sources
AttunityReplicate–ChangeProcessing
CDC
Many Databases and Data
Warehouses
....
© 2017 Attunity
CDC
Demo: Data Streaming into Kafka è HDF è HDP
MSG
n 2 1
MSG MSGData Streaming
Transaction logs
In memory optimized metadata management and data transport
Bulk Load
MSG
n 2 1
MSG MSGData Streaming
Message broker
Message broker
© 2017 Attunity
Attunity Replicate for SAPUniversal, Real-Time and Simplified Data Integration
• Replicate your SAP application data in bulk or real-time for data analytics§ Documents, transactions and business data§ All core and industry-specific SAP modules
• Integrate real-time with all major targets§ DBs, data warehouses, Hadoop – cloud or on premises§ Decode SAP data from complex source structures§ Enable business usage on common data model
• Move external data into SAP HANA
Attunity ReplicateBulk Load
CDC
Core and Industry-Specific SAP Modules
RDBMS | EDW | HadoopOn Premises or Cloud
Hadoop Data Lake
© 2017 Attunity
Attunity Replicate Server
TransformFilter
Batch
CDC IncrementalIn-Memory
File Channel
Batch
Demo: Attunity Replicate
Persistent Store Extract relationships for Pool and Cluster Tables
RDBMS(Oracle, DB2, etc.)
Redo/ Archivelogsor
JournalFile
----------------Transparent
Tables
On Premises
Hadoop RDBMS
Data WarehouseKafka
Cloud
Attunity Replicate Agent for SAP
SAP ECC (Enterprise Central
Component)
Data Model Mapping
Pool/Cluster table RFC