abdw17-lightning talks track-real-time insight into a billion emails
TRANSCRIPT
Real-time Insight into a Billion EmailsSteve Verlander | CTO
APEX BIG DATA WORLD 2017
The Challenge…Typical marketing campaign of 10 to 20 million messagesBetween 50 to 100 campaigns a month for over 1 Billion emails
Campaign segmentation required using historical dataA/B Testing for subject lines and content effectivenessUp to date reporting and analyticsHigh performance and throughputScalability on demandTrack live feedback including opens, clicks, and bounces
The Old Technology…
Oracle RAC cluster databasesMySQL Databases clustered and shardedDistributed SQL Server databasesNoSQL Solutions like MongoDBMap/Reduce using Hadoop
The Limitations…Relational database systems couldn’t scale to handle the high volume of events
Expensive to make fault tolerant, often requiring hardware that never gets used
Difficult and costly to maintain
Scalable solutions like Map/Reduce only provided delayed results
The Solution…Apache Apex and DataTorrent RTS
Can handle thousands of transactions per second
Very low latency and built in fault tolerance
Easy to deploy on premise or to the cloud
Scales effortlessly
The Demo…Simulation of thousands of Email Open Events per second
Real-time analysis of campaign effectiveness
Easy to use visualization tools to see your data in real-time
