deep dive into amazon elasticache architecture and design patterns (dat307) | aws re:invent 2013
DESCRIPTION
Peek behind the scenes to learn about Amazon ElastiCache's design and architecture. See common design patterns of our Memcached and Redis offerings and how customers have used them for in-memory operations and achieved improved latency and throughput for applications. During this session, we review best practices, design patterns, and anti-patterns related to Amazon ElastiCache. We also include a demo where we enable Amazon ElastiCache for a web application and show the resulting performance improvements.TRANSCRIPT
![Page 1: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/1.jpg)
© 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
DAT307 - Deep Dive into Amazon ElastiCache
Architecture and Design Patterns
Nate Wiger, Principal Solutions Architect
November 14, 2013
![Page 2: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/2.jpg)
Contents
• Caching: What’s all this then?
• Amazon ElastiCache
• Laziness, impatience, and hubris
• From one to a dozen nodes
• Memcached vs. Redis showdown
![Page 3: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/3.jpg)
![Page 4: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/4.jpg)
Device Fragmentation
• Phones, tablets, PCs, toasters
• HTML, apps, JSON APIs
• Presentation differs
• Data is the same
• CDN for static images, videos
• Doesn’t help “Welcome Back, Kotter!”
![Page 5: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/5.jpg)
Death By 1000 Queries
• Login, session
• New messages, recent posts
• Calls to Facebook, Twitter APIs
• Your friends love the new Coldplay album!!!
• Sudden viral traffic spikes
![Page 6: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/6.jpg)
![Page 7: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/7.jpg)
cache (noun)
a group of things that have been stored in a secret
place because they are illegal or have been stolen
![Page 8: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/8.jpg)
Typical Web 2.0 App
ELB App
External APIs
![Page 9: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/9.jpg)
Amazon ElastiCache
• Managed cache service
• Memcached or Redis
• Launch cluster of nodes
• Scale up / down
• Monitoring + alerts
![Page 10: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/10.jpg)
Memcached
• In-memory
• Slab allocator
• Multithreaded
• No persistence
• Gold standard
![Page 11: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/11.jpg)
Fire It Up
![Page 12: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/12.jpg)
Fire It Up
![Page 13: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/13.jpg)
Fire It Up
![Page 14: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/14.jpg)
Wire It Up
![Page 15: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/15.jpg)
Wire It Up
![Page 16: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/16.jpg)
Wire It Up # Ruby
require ‘dalli’
cache = Dalli::Client([
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
])
cache.set("some_key", "Some value")
value = cache.get("some_key")
cache.set("another_key", 3)
cache.delete("another_key”)
![Page 17: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/17.jpg)
Multiple Cache Nodes
ELB App
External APIs
![Page 18: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/18.jpg)
Sharding Across Nodes server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]
![Page 19: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/19.jpg)
Sharding Across Nodes server_list = [
’mycache.z2vq55.0001.usw2.cache.amazonaws.com:11211’,
’mycache.z2vq55.0002.usw2.cache.amazonaws.com:11211’
]
server_index = hash(key) % server_list.length
server = server_list[server_index]
BAD
![Page 20: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/20.jpg)
Consistent Hashing
![Page 21: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/21.jpg)
It’s All Been Done Before
• Ruby – Dalli
• Python – HashRing / MemcacheRing
• Node.js – node-memcached
• PHP – libketama or ElastiCache Client
• Java – SpyMemcached or ElastiCache Client
![Page 22: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/22.jpg)
So Far
• Launched a cache cluster
• Got the node names
• Connected our client
• Figured out sharding
![Page 23: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/23.jpg)
![Page 24: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/24.jpg)
What To Cache?
• Everything!
• Database records
• Full HTML pages
• Page fragments
• Remote API calls
![Page 25: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/25.jpg)
How To Cache It?
• Lazy population
• Write-through
• Timed refresh
![Page 26: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/26.jpg)
Laziness is a Virtue # Python
def get_user(user_id):
record = cache.get(user_id)
if record is None:
# Run a DB query
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record)
return record
# App code
user = get_user(17)
![Page 27: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/27.jpg)
Ship It
• Most data is never accessed
• Ensures cache is filled
• Caches fail and scale
• But cache miss penalty
• Best approach for most data
![Page 28: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/28.jpg)
Foresight is 20-20 # Python
def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record)
return record
# App code
user = save_user(17, {"name": "Nate Dogg"})
![Page 29: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/29.jpg)
Laziness vs. Impatience
• Ensures cache is always current
• Write penalty vs. read penalty
• But missing data on scale up
• Plus excess data / cache churn
• Still need lazy fetch too
![Page 30: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/30.jpg)
Combo Move! def save_user(user_id, values):
record = db.query("update users ... where id = ?", user_id, values)
cache.set(user_id, record, 300) # ttl
return record
def get_user(user_id):
record = cache.get(user_id)
if record is None:
record = db.query("select * from users where id = ?", user_id)
cache.set(user_id, record, 300) # ttl
return record
# App code
save_user(17, {"name": "Nate Diddy"})
user = get_user(17)
![Page 31: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/31.jpg)
Timed Refresh
• Run job to periodically update cache
• Good for Top-N lists
• Time-intensive rankings
• Trending items
![Page 32: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/32.jpg)
![Page 33: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/33.jpg)
Monitoring
+ Alerts
![Page 34: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/34.jpg)
Monitoring
• Integration with CloudWatch metrics
• Setup alarms to send via email
• Memory usage
• Evictions
• Which ElastiCache metrics should I monitor?
![Page 35: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/35.jpg)
![Page 36: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/36.jpg)
![Page 37: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/37.jpg)
![Page 38: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/38.jpg)
![Page 39: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/39.jpg)
![Page 40: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/40.jpg)
Node Discovery
• Setup an Amazon SNS topic for ElastiCache
• Have app listen for events – ElastiCache:AddCacheNodeComplete
– ElastiCache:RemoveCacheNodeComplete
• Reconfigure connections
• See Event Notifications and Amazon SNS
![Page 41: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/41.jpg)
Programmable Scaling
ELB App
External APIs SNS
Add Node
![Page 42: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/42.jpg)
Node Auto-Discovery # PHP
$server_endpoint = "mycache.z2vq55.cfg.usw2.cache.amazonaws.com";
$server_port = 11211;
$cache = new Memcached();
$cache->setOption(
Memcached::OPT_CLIENT_MODE, Memcached::DYNAMIC_CLIENT_MODE);
# Set config endpoint as only server
$cache->addServer($server_endpoint, $server_port);
# Lib auto-locates nodes
$cache->set("key", "value");
![Page 43: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/43.jpg)
Redis
• Also in-memory
• Advanced data types
• Atomic operations
• Single-threaded
• Persistence
• Read replicas
![Page 44: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/44.jpg)
Leaderboard with Sorted Sets ZADD leaderboard 556 "Andy"
ZADD leaderboard 819 "Barry"
ZADD leaderboard 105 "Carl"
ZADD leaderboard 1312 "Derek"
ZREVRANGE leaderboard 0 -1
1) "Derek"
2) "Barry"
3) "Andy"
4) "Carl"
ZREVRANK "Barry"
2
![Page 45: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/45.jpg)
Follow the Leader def save_score(user, score):
record = db.query("update users ... where id = ?", user_id, score)
redis.zadd("leaderboard", score, user)
def get_rank(user)
return redis.zrevrank(user) + 1
# App code
save_score("Andy", 556)
save_score("Barry", 819)
save_score("Carl", 105)
save_score("Derek", 1312)
get_rank("Barry") # 2
![Page 46: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/46.jpg)
Redis Replicas
ELB App
External APIs
Replication Group
Reads Writes
![Page 47: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/47.jpg)
Redis Sharding
• Same concept as Memcached
• BUT
• Can't shard – Lists
– Sets / sorted sets
– Hashes
• Require single in-memory structure
![Page 48: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/48.jpg)
Anti-Pattern: Dedicated Nodes
• Spawn multiple nodes
• Use for different features – Leaderboard
– Counters
• Can still shard key-value ops
![Page 49: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/49.jpg)
Dedicated Redis Nodes
ELB App
External APIs
Counters Leaderboard
![Page 50: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/50.jpg)
Summary
• Caching is good
• Good caching is hard
• ElastiCache eases deployment
• Memcached or Redis
• More to come
![Page 51: Deep Dive into Amazon ElastiCache Architecture and Design Patterns (DAT307) | AWS re:Invent 2013](https://reader034.vdocuments.site/reader034/viewer/2022052618/554a2013b4c90507558b5ac1/html5/thumbnails/51.jpg)
Please give us your feedback on this
presentation
As a thank you, we will select prize
winners daily for completed surveys!
DAT307 - Nate Wiger