how to create a real-time bag data application
TRANSCRIPT
-
7/30/2019 How to Create a Real-Time Bag Data Application
1/11
HowtoCreateaReal-meBigDataApplicaon
BigDataSiliconValley,April24th2013
1
-
7/30/2019 How to Create a Real-Time Bag Data Application
2/11
AboutMe
About4yearsintheSemiconductorspace NaonalSemiconductor(TexasInstruments) LTXCorporaon(LTXCadence) AppliedMaterials
About8yearsintheMapping/TrafficDataspace TeleAtlas(TomTom) Map24(NAVTEQ) INRIX
About5yearsintheStartupspace
WindSpring Skydera
(2)
-
7/30/2019 How to Create a Real-Time Bag Data Application
3/11
CreangAReal-meBigDataApp
WhatisReal-me? ExamplesofReal-meBigDataApps
GoogleAnalycs,KissMetrics,mixpanel,Woopra WhatdoweneedtocreateaReal-meBigDataApp?
CanaReal-meBigDataAppruninmydatacenter?
WhattypeoftoolswouldIneedtocreate(#)
-
7/30/2019 How to Create a Real-Time Bag Data Application
4/11
WhatdoIneed?
(#)
WebTierPython, Ruby, Node.js, .Net, Java
DatabaseTier
NoSQL (MongoDB, Cassandra, DynamoDB, etc..)
OffsetAnalysis
Real-meAnalysis
Hadoop, Hive, Mahout, AWS-EMR
Caching (Memcached, Redis, etc..)
-
7/30/2019 How to Create a Real-Time Bag Data Application
5/11
WebTier
YourfavoritelanguageandFramework CreateAPItoreceivedata CreateAPItodistributedata
Thiswillenableyourapplicaontorequestsupdateswithoutrefreshingthewholescreen
BuildUserinterfacetorequestdataelementsandnotjustpages.
(#)
API
UserInterface
-
7/30/2019 How to Create a Real-Time Bag Data Application
6/11
WebTierRealTimeprocessing
Per-Requestprocessing Pre-Definedmatrix's Pre-Definedmeframes Pre-Definedresulngtables
(#)
API
UserInterface
Real-meProcess
Pre-Definedresulngtables
-
7/30/2019 How to Create a Real-Time Bag Data Application
7/11
DatabaseTier
NoSQLMongoDB,Cassandra,DynamoDB,CouchDBImportanttohavesomethingDistributedStorepredefinedtablesHoldsrawdataforHadoopandoffsetprocessing
(#)
-
7/30/2019 How to Create a Real-Time Bag Data Application
8/11
DatabaseTier
CachingMemcached,RedisImproveretrievalmesLowertheburdenonNoSQL/SQLforfrequently
requesteddata
Holdneeddatabetweenrequests
(#)
-
7/30/2019 How to Create a Real-Time Bag Data Application
9/11
DatabaseTier
SQLYourfavoriteRDBMSSomeusecasesbenefitfromhavinganRDBMSintheStack(complexIndexing)SomeNoSQLDatabasesprovidescomplex
Indexing
(#)
-
7/30/2019 How to Create a Real-Time Bag Data Application
10/11
OffsetAnalysis
Currentprocessforprocessinglargeamountsofdata
Veryusefulforaskingquesonsnotpredefinedinthereal-meprocessing NoneedtolosetheabilitytoprocesslargeamountsofdataeveryXXhours.
(#)
-
7/30/2019 How to Create a Real-Time Bag Data Application
11/11
Contact
SteffanyBoldrini
@steold
(#)
LecoleCole
SkyderaInc.
lecole@skydera
@lecole