webcamp: developer day: the big, the small and the redis - Андрей Савченко

The bigThe small

and The Redis

Andrey Savchenko

Github Twitter

LinkedIn}@ptico

CTO@aejis

The world of DBMS

Big

• PostgreSQL • Cassandra • MongoDB • and counting…

Small

• SQLite • LevelDB • Kyoto cabinet • etc…

Redis

Redis

• Key-value DB • High-performant • Async persistence • Replication • Transactions • Extensions • Lua-scripting • Pub/Sub

Clients

ActionScript, C, C#, C++, Clojure, Common Lisp, D, Dart, Emacs Lisp, Erlang, Fancy, Prolog, Go, Haskell, haXe, Io, Java, Lua, Node.js,

Objective-C, Perl, PHP, Python, Ruby, Rust, Scala, Scheme, Smalltalk, Tcl

CommandsAPPEND, AUTH, BGREWRITEAOF, BGSAVE, BITCOUNT, BITOP, BITPOS, BLPOP, BRPOP,

BRPOPLPUSH, CLIENT KILL, CLIENT LIST, CLIENT GETNAME, CLIENT PAUSE, CLIENT SETNAME, CONFIG GET, CONFIG REWRITE, CONFIG SET, CONFIG RESETSTAT, DBSIZE, DEBUG

OBJECT, DEBUG SEGFAULT, DECR, DECRBY, DEL, DISCARD, DUMP, ECHO, EVAL, EVALSHA, EXEC, EXISTS, EXPIRE, EXPIREAT, FLUSHALL, FLUSHDB, GET, GETBIT, GETRANGE, GETSET,

HDEL, HEXISTS, HGET, HGETALL, HINCRBY, HINCRBYFLOAT, HKEYS, HLEN, HMGET, HMSET, HSET, HSETNX, HVALS, INCR, INCRBY, INCRBYFLOAT, INFO, KEYS, LASTSAVE, LINDEX, LINSERT, LLEN, LPOP, LPUSH, LPUSHX, LRANGE, LREM, LSET, LTRIM, MGET, MIGRATE,

MONITOR, MOVE, MSET, MSETNX, MULTI, OBJECT, PERSIST, PEXPIRE, PEXPIREAT, PFADD, PFCOUNT, PFMERGE, PING, PSETEX, PSUBSCRIBE, PUBSUB, PTTL, PUBLISH, PUNSUBSCRIBE,

QUIT, RANDOMKEY, RENAME, RENAMENX, RESTORE, ROLE, RPOP, RPOPLPUSH, RPUSH, RPUSHX, SADD, SAVE, SCARD, SCRIPT EXISTS, SCRIPT FLUSH, SCRIPT KILL, SCRIPT LOAD,

SDIFF, SDIFFSTORE, SELECT, SET, SETBIT, SETEX, SETNX, SETRANGE, SHUTDOWN, SINTER, SINTERSTORE, SISMEMBER, SLAVEOF, SLOWLOG, SMEMBERS, SMOVE, SORT, SPOP,

SRANDMEMBER, SREM, STRLEN, SUBSCRIBE, SUNION, SUNIONSTORE, SYNC, TIME, TTL, TYPE, UNSUBSCRIBE, UNWATCH, WATCH, ZADD, ZCARD, ZCOUNT, ZINCRBY, ZINTERSTORE,

ZLEXCOUNT, ZRANGE, ZRANGEBYLEX, ZRANGEBYSCORE, ZRANK, ZREM, ZREMRANGEBYLEX, ZREMRANGEBYRANK, ZREMRANGEBYSCORE, ZREVRANGE,

ZREVRANGEBYSCORE, ZREVRANK, ZSCORE, ZUNIONSTORE, SCAN, SSCAN, HSCAN, ZSCAN,

Documentation

Documentation

DataTypes

• String • Hash • List • Set • Sorted set • HyperLogLog

String

SET testkey "Hello" GET testkey #=> "Hello" MSET key1 "Hello" key2 "world" !SET counter 1 INCR counter GET counter #=> "2"

Hash

HSET test key1 "Hello" HSET test key2 "World" HGET test key2 #=> "World" HGETALL test #=> "key1" "Hello" "key2" "World"

List

RPUSH test "zero" RPUSH test "one" RPUSH test "two" LRANGE test 1 2 #=> "one" "two"

Set

SADD users "andrey" SADD users "dima" SCARD users #=> 2 SISMEMBER users "andrey" #=> 1 SISMEMBER users "gleb" #=> 0

Performance(MacBook retina 13")

• SET: 67294.75 rps • GET: 71787.51 rps • LPUSH: 72202.16 rps • SADD: 70621.47 rps

Persistence and memory

# save <seconds> <changes> save 900 1 save 300 10 save 60 10000

Persistence and memory

# maxmemory <bytes> maxmemory 1gb

Persistence and memory

appendonly yes # appendfsync always appendfsync everysec # appendfsync no

Persistence and memory

http://redis.io/topics/persistence http://antirez.com/post/redis-persistence-demystified.html

Use cases

Caches, sessions

Caches, sessions

SET a18f045 "cache content" EXPIRE a18f045 100

Invalidation by time

Caches, sessions

SET a18b045 "cache content" SET d98f0c9 "cache content" SADD tag1 a18b045 SADD tag1 d98f0c9 DEL {{SMEMBERS tag1}}

Invalidation by tags

Tags

SADD programming 1 2 3 4 5 SADD ruby 1 4 SADD python 2 3 SADD tdd 1 3 # Posts with both "programming" AND "ruby" tags SINTER programming ruby # => "1" "4" !# Posts with "ruby" OR "python" tags SUNION ruby python #=> "1" "2" "3" "4" !# Posts posts with "ruby" tag but without "tdd" tag SDIFF ruby tdd #=> "4" !# How many posts with "ruby" tag SCARD ruby #=> 2

Job queue

LPUSH worker1 "Job 1" LPUSH worker1 "Job 2" LPUSH worker1 "Job 3" RPOP worker1 #=> "Job 1" RPOP worker1 #=> "Job 2"

Data storage

HSET settings:{id} receive_emails 1 HSET settings:{id} records_per_page 20 HSET settings:{id} preferred_syntax "markdown" HGETALL settings:{id}

Statistics

# Increment counter INCR posts:{id} !# How many views/votes? GET posts:{id}

Statistics

# Increment counter ZINCRBY posts 1 {id} !# How many views/votes? ZSCORE posts {id} !# TOP-10 posts ZREVRANGE posts 0 9 !# TOP-10 with views/votes quantity ZREVRANGE posts 0 9 WITHSCORES

Simple graphs

# User Joe adds user Mila to friends SADD users:{joe}:friends {mila} SADD users:{mila}:friend_of {joe} !# Get Mila's friends SMEMBERS users:{mila}:friends !# Get user Mila's mutual friends SCARD users:{mila}:friends users:{mila}:friend_of !# Get common friends of Mila and Joe SCARD users:{mila}:friends users:{joe}:friends !# Joe is a friend of user Mila? SISMEMBER users:{mila}:friends {joe}

Message bus

# Web-application: SUBSCRIBE messages !# Async jobs server: SUBSCRIBE jobs !# Web-application: PUBLISH jobs "videos:12:mpeg:ogg" !# Async jobs server: PUBLISH messages "Video encoding #12 finished"

Extensions

Redis Geo

GEOADD kyiv 30.437672 50.440884 "Aejis office" !GEORADIUS kyiv 30.437672 50.440884 2 km WITHDISTANCE #=> "Shulyavska station" "1.61" "Politech institute" "1.74" "Aviation university" "0.76" !GEORADIUSBYMEMBER kyiv "Aejis office" 2 km #=> "Shulyavska station" "Politech institute" "Aviation university"

https://matt.sh/redis-geo

Redis JSON

HGETALLJSON settings:{id} #=> {“receive_emails":1,"records_per_page":20, "preferred_syntax":"markdown"} !JSONWRAP ZREVRANGE posts 0 3 WITHSCORES #=> {"42":820,"115":760,"110":522} !JSONDOCSET users '{"id":42, "name":"John"}' !JSONDOCKEYS users !JSONDFIELDGET users name

https://matt.sh/redis-json

Ian Barbour https://flic.kr/p/anCQwx (CC-BY-NC-SA 2.0)Attribution:

Thanks!

Andrey Savchenko @ptico