bigdoor business intelligence in 15 minutes
DESCRIPTION
Eva Monsen, BigDoor's own business intelligence developer, gives a real-world example of "big data" and what it means for us at BigDoor. Eva walks us through the BigDoor data pipeline, which is the technology path that data takes to get from its raw form to reports and visualizations.TRANSCRIPT
What is BigDoor?
Online marketing through loyalty programs
Partner: Enterprise brands with online presence
Goals: Registration, engagement, loyalty
Product:
Users earn virtual currency for actions
Users exchange virtual currency for rewards
PacSun.com + BigDoor
PacSun.com + BigDoor
BigDoor Data Goal
Prove that we are meeting Partner goals
Registration: Are people registering?
Registration rate of control and exposed groups
Engagement: Are participants more engaged?
Actions per user in control and exposed groups
Loyalty: Do participants return?
Daily unique users v. monthly unique users
Data Challenges
Peak: ~800 requests per second
Business data ->Transactional SQL DB
Optimized for write speed and flexibility
Unregistered user requests -> Apache logs
Flat text files
Need all data in one place
Fast queries
Easy to slice and dice
BigDoor Architecture
Load Balancer
App Host
App Host
App Host
SQL DB
Log Processing
ETL
Aggregation
Data Warehouse
Drop us a line any time! Contact: [email protected]