Redis: The Universal NoSQL-Tool

Eberhard Wolff Architecture and Technology Manager

adesso AG

NoSQL is all about Big Data

NoSQL is all about Big Data

Redis •  Remote Dictionary Server •  Advanced Key Value Store •  Actually also a cache and a messaging

server •  Open Source – free to change the code •  Sponsored by VMware •  No commercial agenda •  Implemented in C (actually pretty small) •  In memory (fast)

Redis •  Data should fit in memory •  Data written asynchronously to disk or in

an append only file – Won't slow down operations – Configurable delay

•  Bindings for many languages •  Very easy replication •  i.e. like an in-memory cache with

persistence option •  http://redis.io/

How to Use Redis •  Very simple protocol •  Can telnet to it

•  Libraries for lots of languages

•  Java: –  Jedis –  JRedis –  RJC

•  Spring Data Redis as abstraction –  Uniform concepts (e.g. templates) across all NoSQL and SQL

stores –  Even JPA –  Exception translation –  Still exposes unique features

Use Case: High Scores for Players

•  Online game •  Store high scores

•  High data volume •  Frequently changing data

Use Case: High Scores for Players

•  Note: Other data (name, email, etc) can still be stored somewhere else

•  Key: Name of player •  Value: Score •  Encoded as bytes

•  Operations: GET, SET, DEL

Demo: RedisWriter / RedisMultiWriter / RedisClean /

redis-cli / telnet / redis-benchmark

Points to Note •  Extremely fast •  See http://redis.io/topics/benchmarks •  Multi Set: Can issue multiple set operations in

one round trip for very high performance •  Data in database does not impact performance •  Data written asynchronously •  Not shown:

–  APPEND : append data to a key –  KEYS : find all keys for a pattern –  RENAME : Change key

Redis as a Cache •  Data kept in memory •  …and on disk •  Data survives restarts •  No need for the cache to warm up •  But: What about evicting data from the cache? •  Solution: Data can have time-to-live •  Can set global memory policy to avoid

excessive memory consumption •  Let assume certain texts (e.g. HTML) should be

cached for our application…

Demo: RedisCache

Create Unique Player ID •  Each player should have a unique ID •  Needs to be unique, even in a cluster •  Solution: atomic incr operation •  i.e. works correctly even if multiple

threads access the counter at the same time

•  Quick – in memory

Demo: RedisCreateID

Similar operations •  GETSET : Atomically get old value and

set to new value •  Global locks using SETNX / DEL •  SETNX returns true only if the client

successfully set the value •  i.e. lock was successfully acquired

Replication •  Redis uses master / slave replication •  Slaves can replicate to other slaves •  Replication does not block the master •  Useable for scalability of reads or data

redundancy •  Write would go to the master and be

propagated •  Make backups quite easy •  Just add

slaveof <ip> <port> to redis.conf

Scaling •  Slaves can be used for (eventually

consistent) reads •  Storage to disk can be offloaded to

slaves •  No more breaks during disk writes •  But: Only one master can write •  Still: All data should fit in RAM

Scaling •  Can use Virtual Memory •  Redis has its own Virtual Memory

implementation (to be deprecated) •  Partially implemented: Redis Cluster with

Master / Slave replication •  Sharding i.e. data distributed across many

nodes •  Could implement your own sharding on the

client

Redis Data Types •  So far: Simple types: byte[]

–  String and numbers encoded as byte[] –  Can cluster multiple get/set into one atomic action

•  Atomic increment / decrement for integers •  Lists

–  Can add / remove atomically •  Sets

–  Like lists but no order •  Sorted sets

–  Each element has a score to sort by

Highscores •  Range the high scores •  Solutions: Use a sorted set

•  Result: scores automatically sorted

•  Trivial to get the top high scores

Demo: RedisHighScore

Sending EMails •  EMails should be sent by a server

•  Need to queue emails

•  Solution: Use a list + push / pop

Demo: RedisEMailReaderList / RedisEMailWriterList

Next step: Messaging •  Lists are not really designed for communication •  Message driven middleware is an established

concept •  Redis has some parts •  Topics

–  Have a name –  Multiple receivers can listen to it

•  Subscribe to –  A Topic –  Or a patterns of topic names

Demo: RedisEMailReaderMessaging / RedisEMailWriterMessaging

Transactions •  Multiple actions can be put into a transaction •  Command MULTI to start transaction •  EXEC to execute •  DISCARD to discard

•  Guarantees: –  Executed in the defined order –  Atomic (all or nothing) –  No isolation –  Can use optimistic locking

Key-Value Stores: Store Complex Data •  Storing data as serialized blobs / JSON / XML

–  ”player:wolff" è ”all the data" •  Storing data as multiple keys

–  "player:wolff:score" è 42 –  "player:wolff:email" è ”eberhard.wolff@adesso.de"

•  Requires multi get/set to be efficient •  Can find all keys using patterns •  Can create your custom Serializer in Spring Data •  Key / values stores are no really useful for

complex data

Conclusion – Redis is a Swiss Knife •  Very fast - In Memory •  Cache + persistence •  Messaging •  Centralized locking and counting •  Much more than just persistence •  Not shown: Lua scripting •  Not useful for

– Big Data – Seamless scaling – Ad hoc queries

•  Will not be the only data store you use

Links •  http://redis.io/ •  Try it yourself: http://try.redis-db.com/ •  Spring Data Redis

http://www.springsource.org/spring-data/redis •  Demos: https://github.com/ewolff/spring-redis-demo •  Who is using it?

http://redis.io/topics/whos-using-redis •  Books

–  Tiago Macedo, Fred Oliveira: Redis Cookbook –  Pollack, Gierke, Risberg, Brisbin, Hunger: Spring Data