webcamp: developer day: the big, the small and the redis - Андрей Савченко
DESCRIPTION
The Big, the Small and the Redis Андрей Савченко Доклад посвящен Redis: одной из самых недооценённых СУБД. Имея, на первый взгляд, не очень большой функционал, при пристальном рассмотрении Redis может дать фору большинству более жирных конкурентов. О том, как правильно его “готовить”, где применять на практике и как вовремя остановиться и будет идти речь в докладе.TRANSCRIPT
The bigThe small
and The Redis
Andrey Savchenko
Github Twitter
LinkedIn}@ptico
CTO@aejis
The world of DBMS
Big
• PostgreSQL • Cassandra • MongoDB • and counting…
Small
• SQLite • LevelDB • Kyoto cabinet • etc…
Redis
Redis
• Key-value DB • High-performant • Async persistence • Replication • Transactions • Extensions • Lua-scripting • Pub/Sub
Clients
ActionScript, C, C#, C++, Clojure, Common Lisp, D, Dart, Emacs Lisp, Erlang, Fancy, Prolog, Go, Haskell, haXe, Io, Java, Lua, Node.js,
Objective-C, Perl, PHP, Python, Ruby, Rust, Scala, Scheme, Smalltalk, Tcl
CommandsAPPEND, AUTH, BGREWRITEAOF, BGSAVE, BITCOUNT, BITOP, BITPOS, BLPOP, BRPOP,
BRPOPLPUSH, CLIENT KILL, CLIENT LIST, CLIENT GETNAME, CLIENT PAUSE, CLIENT SETNAME, CONFIG GET, CONFIG REWRITE, CONFIG SET, CONFIG RESETSTAT, DBSIZE, DEBUG
OBJECT, DEBUG SEGFAULT, DECR, DECRBY, DEL, DISCARD, DUMP, ECHO, EVAL, EVALSHA, EXEC, EXISTS, EXPIRE, EXPIREAT, FLUSHALL, FLUSHDB, GET, GETBIT, GETRANGE, GETSET,
HDEL, HEXISTS, HGET, HGETALL, HINCRBY, HINCRBYFLOAT, HKEYS, HLEN, HMGET, HMSET, HSET, HSETNX, HVALS, INCR, INCRBY, INCRBYFLOAT, INFO, KEYS, LASTSAVE, LINDEX, LINSERT, LLEN, LPOP, LPUSH, LPUSHX, LRANGE, LREM, LSET, LTRIM, MGET, MIGRATE,
MONITOR, MOVE, MSET, MSETNX, MULTI, OBJECT, PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFCOUNT, PFMERGE, PING, PSETEX, PSUBSCRIBE, PUBSUB, PTTL, PUBLISH, PUNSUBSCRIBE,
QUIT, RANDOMKEY, RENAME, RENAMENX, RESTORE, ROLE, RPOP, RPOPLPUSH, RPUSH, RPUSHX, SADD, SAVE, SCARD, SCRIPT EXISTS, SCRIPT FLUSH, SCRIPT KILL, SCRIPT LOAD,
SDIFF, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SHUTDOWN, SINTER, SINTERSTORE, SISMEMBER, SLAVEOF, SLOWLOG, SMEMBERS, SMOVE, SORT, SPOP,
SRANDMEMBER, SREM, STRLEN, SUBSCRIBE, SUNION, SUNIONSTORE, SYNC, TIME, TTL, TYPE, UNSUBSCRIBE, UNWATCH, WATCH, ZADD, ZCARD, ZCOUNT, ZINCRBY, ZINTERSTORE,
ZLEXCOUNT, ZRANGE, ZRANGEBYLEX, ZRANGEBYSCORE, ZRANK, ZREM, ZREMRANGEBYLEX, ZREMRANGEBYRANK, ZREMRANGEBYSCORE, ZREVRANGE,
ZREVRANGEBYSCORE, ZREVRANK, ZSCORE, ZUNIONSTORE, SCAN, SSCAN, HSCAN, ZSCAN,
Documentation
Documentation
DataTypes
• String • Hash • List • Set • Sorted set • HyperLogLog
String
SET testkey "Hello" GET testkey #=> "Hello" MSET key1 "Hello" key2 "world" !SET counter 1 INCR counter GET counter #=> "2"
Hash
HSET test key1 "Hello" HSET test key2 "World" HGET test key2 #=> "World" HGETALL test #=> "key1" "Hello" "key2" "World"
List
RPUSH test "zero" RPUSH test "one" RPUSH test "two" LRANGE test 1 2 #=> "one" "two"
Set
SADD users "andrey" SADD users "dima" SCARD users #=> 2 SISMEMBER users "andrey" #=> 1 SISMEMBER users "gleb" #=> 0
Performance(MacBook retina 13")
• SET: 67294.75 rps • GET: 71787.51 rps • LPUSH: 72202.16 rps • SADD: 70621.47 rps
Persistence and memory
# save <seconds> <changes> save 900 1 save 300 10 save 60 10000
Persistence and memory
# maxmemory <bytes> maxmemory 1gb
Persistence and memory
appendonly yes # appendfsync always appendfsync everysec # appendfsync no
Persistence and memory
http://redis.io/topics/persistence http://antirez.com/post/redis-persistence-demystified.html
Use cases
Caches, sessions
Caches, sessions
SET a18f045 "cache content" EXPIRE a18f045 100
Invalidation by time
Caches, sessions
SET a18b045 "cache content" SET d98f0c9 "cache content" SADD tag1 a18b045 SADD tag1 d98f0c9 DEL {{SMEMBERS tag1}}
Invalidation by tags
Tags
SADD programming 1 2 3 4 5 SADD ruby 1 4 SADD python 2 3 SADD tdd 1 3 # Posts with both "programming" AND "ruby" tags SINTER programming ruby # => "1" "4" !# Posts with "ruby" OR "python" tags SUNION ruby python #=> "1" "2" "3" "4" !# Posts posts with "ruby" tag but without "tdd" tag SDIFF ruby tdd #=> "4" !# How many posts with "ruby" tag SCARD ruby #=> 2
Job queue
LPUSH worker1 "Job 1" LPUSH worker1 "Job 2" LPUSH worker1 "Job 3" RPOP worker1 #=> "Job 1" RPOP worker1 #=> "Job 2"
Data storage
HSET settings:{id} receive_emails 1 HSET settings:{id} records_per_page 20 HSET settings:{id} preferred_syntax "markdown" HGETALL settings:{id}
Statistics
# Increment counter INCR posts:{id} !# How many views/votes? GET posts:{id}
Statistics
# Increment counter ZINCRBY posts 1 {id} !# How many views/votes? ZSCORE posts {id} !# TOP-10 posts ZREVRANGE posts 0 9 !# TOP-10 with views/votes quantity ZREVRANGE posts 0 9 WITHSCORES
Simple graphs
# User Joe adds user Mila to friends SADD users:{joe}:friends {mila} SADD users:{mila}:friend_of {joe} !# Get Mila's friends SMEMBERS users:{mila}:friends !# Get user Mila's mutual friends SCARD users:{mila}:friends users:{mila}:friend_of !# Get common friends of Mila and Joe SCARD users:{mila}:friends users:{joe}:friends !# Joe is a friend of user Mila? SISMEMBER users:{mila}:friends {joe}
Message bus
# Web-application: SUBSCRIBE messages !# Async jobs server: SUBSCRIBE jobs !# Web-application: PUBLISH jobs "videos:12:mpeg:ogg" !# Async jobs server: PUBLISH messages "Video encoding #12 finished"
Extensions
Redis Geo
GEOADD kyiv 30.437672 50.440884 "Aejis office" !GEORADIUS kyiv 30.437672 50.440884 2 km WITHDISTANCE #=> "Shulyavska station" "1.61" "Politech institute" "1.74" "Aviation university" "0.76" !GEORADIUSBYMEMBER kyiv "Aejis office" 2 km #=> "Shulyavska station" "Politech institute" "Aviation university"
https://matt.sh/redis-geo
Redis JSON
HGETALLJSON settings:{id} #=> {“receive_emails":1,"records_per_page":20, "preferred_syntax":"markdown"} !JSONWRAP ZREVRANGE posts 0 3 WITHSCORES #=> {"42":820,"115":760,"110":522} !JSONDOCSET users '{"id":42, "name":"John"}' !JSONDOCKEYS users !JSONDFIELDGET users name
https://matt.sh/redis-json
Ian Barbour https://flic.kr/p/anCQwx (CC-BY-NC-SA 2.0)Attribution:
Thanks!
Andrey Savchenko @ptico