how to scale to 100k users using windows azure
TRANSCRIPT
Ionut Antiu
www.get-jukebox.com
@johnnyantiu
1. What are we talking about
2. What is Style Jukebox
3. What were the challenges
4. Wrap up
Style Jukebox is a High Fidelity cloud-music streaming service that helps you keep all your music – be it bought, self-produced,
or imported from CDs or other media, always backed up and synced across all your devices.
TRACTION
125,000+ registered users
52,000+ MAU
Peak of 13,000 users DAU
4,5+ million songs stored in the Cloud
50,000 – 100,000 songs listened daily
Low latency
Scalability
High Availability
OBJECTIVES
What is the basic Cloud architecture?
“LE SERVER”
Database SQL Storage
API
LE CLIENT APP LE CLIENT APP LE CLIENT APP
“LE SERVER”
API
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
Scale Up Scale Out
Database, API, Content bandwidth, Website
Scale Up
One operation that cannot be divided requires more compute power
(Audio/Video encoding, Your own DB, etc.)
Basic A11 Core AMD CPU1,75 GB RAMEUR 41/month*
G5, 32 coresIntel Xeon448 GB RAM6 TB EUR 4865/month*
Scale Out
Used for API, Website, etc.
Ex: Each machine can have a limited number of connected users
Basic A11 Core AMD CPU1,75 GB RAMEUR 41/month*
4xBasic A11 Core AMD CPU1,75 GB RAMEUR 41/month*
SYNC SERVICE
API (GetSongURL, Login)
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
BLOCK SERVICE
API (DownloadSong, UploadSong)
SYNC SERVICE
API (GetSongURL, Login)
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
BLOCK SERVICE
API (DownloadSong, UploadSong)
MEMCACHE
Redis
Memcache Worker
The Memcache Worker predicts and prepares the results
Memcache Hit Ratio 86-91%
Reduced time of response with 80% to an average of 250ms
SYNC SERVICE
API (GetSongURL, Login)
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
BLOCK SERVICE
API (DownloadSong, UploadSong)
MEMCACHE
Redis
Memcache Worker
CDN
Windows Azure CDN
CDN Worker
SYNC SERVICE
API (GetSongURL, Login)
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
BLOCK SERVICE
API (DownloadSong, UploadSong)
MEMCACHE
Redis
Memcache Worker
CDN
Windows Azure CDN
CDN Worker
STORAGE
Upload Buffer
Upload Processing
SYNC SERVICE
API (GetSongURL, Login)
LOAD BALANCER
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
BLOCK SERVICE
API (DownloadSong, UploadSong)
LOAD BALANCER
MEMCACHE
Redis
Memcache Worker
CDN
Windows Azure CDN
CDN Worker
STORAGE
Upload Buffer
Upload Processing
Download Processing
Load Balancer
Integrated in the Azure Cloud Services
Makes sure all instances have the same loadAuto-Scale creates new instances or removes automatically
SYNC SERVICE
API (GetSongURL, Login)
LOAD BALANCER
LE CLIENT APP LE CLIENT APP LE CLIENT APP
STORAGE
Windows Azure Storage Blobs
DATABASE
SQL Azure
BLOCK SERVICE
API (DownloadSong, UploadSong)
LOAD BALANCER
MEMCACHE
Redis
Memcache Worker
CDN
Windows Azure CDN
CDN Worker
STORAGE
Upload Buffer
Upload Processing
Download Processing
TRAFFIC MANAGER
SYNC SERVICE – WEST US
API (GetSongURL, Login)
DATABASE - AMSTERDAM
SQL Azure
SYNC SERVICE – WEST US
API (GetSongURL, Login)
DATABASE - AMSTERDAM
SQL Azure
~ 9,000 km * 3 calls = ~ 30,000 km
30,000 km : 300,000km/s = 100 ms
LE CLIENT APP LE CLIENT APP LE CLIENT APP
TRAFFIC MANAGER
SYNC – WEST US
API x VMs
LOAD BALANCER
BLOCK – WEST US
API x VMs
LOAD BALANCER
SYNC – WEST EU
API x VMs
LOAD BALANCER
BLOCK – WEST EU
API x VMs
LOAD BALANCER
CDN
Windows Azure CDN
STORAGE
Windows Azure Storage Blobs
MAIN DATABASE – WEST EU
SQL Azure
MEMCACHE – WEST EU
Redis
SLAVE DATABASE – WEST US
SQL Azure
MEMCACHE 2 – WEST US
Redis
STORAGE
Upload Buffer
Traffic Manager
stylejukebox.trafficmanager.net
4 VM machinesin West Europe
4 VM machinesin West US
Closest location to the userLoad balancing
Fail-overWorks for Websites, Cloud Services
Azure Datacenters
How to test?
Test 1
1,000 concurrent users1 core machine
Test 2
1,000 concurrent users4 core machine
$ 0,4 more per hour
Autoscale applies to Cloud Services and Websites
1 core machine can handle 200 connections simultaneous
4 core machine can handle 1,000 connections simultaneous
Monitoring
NewRelic
Always prepare for things to go wrong
Ex: Routers on the Internet
Low latency
Scalability
High Availability
OBJECTIVES
Ionut Antiu
www.get-jukebox.com
@johnnyantiu