redis beyond
DESCRIPTION
TRANSCRIPT
![Page 1: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/1.jpg)
+
Redis {beyond the basics}
Presented by
Steph ☺
![Page 2: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/2.jpg)
+Outline
■ Revisiting Redis
■ Keeping data safe
■ Persistence
■ Replication
■ Replacing Failed Master
■ Transaction
■ Reducing memory use
■ Ziplist
■ Intlist
■ Sharding
■ Scaling
![Page 3: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/3.jpg)
+Redis
■ In-memory remote database
■ Advanced key-value store
■ Data-structure server
■ Offers
■ High Performance
■ Replication
■ Unique data model
![Page 4: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/4.jpg)
+Snapshotting
■ Data is taken as it exists and is written to the disk
■ Point-in-time copy of in-memory data
■ Backup & transfer to other server
■ Written to file in “dbfilename” stored in “dir”
■ Until next snapshot is taken, last snapshot can be lost if redis crashes
{Persistence}
![Page 5: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/5.jpg)
+Snapshotting
Initiating Snapshots■ bgsave
■ SAVE
■ Save lines save 60 10000
■ SHUTDOWN / TERM
■ SYNC
{Persistence}
![Page 6: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/6.jpg)
+Snapshotting
■ How often to perform an automatic snapshot
■ Accept writes on failure
■ Snapshot compression
■ What to name the snapshot on the disk
![Page 7: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/7.jpg)
+Append Only File (AOF)
■ Copies incoming write commands as it happens
■ Records data changes at the end of the backup file
■ Data set could be recovered with replaying AOF
■ “append only yes”
■ “appendfsyc always”
■ Limited by disk performance
{Persistence}
![Page 8: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/8.jpg)
+Append Only File (AOF)
■ Option to use AOF
■ Occurrence of sync writes to disk
■ Option to sync during AOF compaction
■ Occurrence of AOF compaction
![Page 9: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/9.jpg)
+Replication
■ Method where other servers receive an updated copy of the data as its being written
■ Replicas can service read queries
■ Single master database sends writes out to multiple slave databases
■ Set operations can take seconds to finish
![Page 10: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/10.jpg)
+Replication
■ Configuring for replication■ On master, ensure that the path and filename are writable
by redis process
■ Enable slaving : slaveof host port
■ In a running system, redis can be stopped slaving or connect to a different master
■ New / Transfer connection: slaveof host port
■ Stop data update: SLAVEOF no one
![Page 11: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/11.jpg)
+Replication {SLAVE to MASTER Connection}
![Page 12: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/12.jpg)
+Replication{Redis Master-Slave Replica Tree}
![Page 13: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/13.jpg)
+Replacing Failed Master
■ What will we do in case of system failure?
■ Scenario■ Machine A – Redis Master, Machine B – Redis Slave
■ Machine A loses network connectivity
■ Machine C has Redis, but no copy of data
■ Solution A■ Make a fresh snapshot using Machine B using SAVE
■ Copy snapshot to Machine C
■ Start Redis on Machine C
■ Tell Machine B to be a slave of Machine C
{Scenario and Solution}
![Page 14: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/14.jpg)
+Replacing Failed Master{Sequence of Commands}
![Page 15: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/15.jpg)
+Replacing Failed Master
■ What will we do in case of system failure?
■ Solution B■ Use Machine B (Slave) as Master
■ Create a new Slave (maybe Machine C)
■ Update client configuration to read/write to proper servers
■ (optional) update server configuration if restart is needed
{Scenario and Solution}
![Page 16: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/16.jpg)
+Transactions
■ Begin transaction with MULTI
■ Execute commands with EXEC
■ Delayed execution with multi/exec can improve performance■ Holds off sending commands until all of them are known
■ When all of the commands are known, MULTI is sent by client
![Page 17: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/17.jpg)
+Transactions
■ Pipelining■ Send multiple commands at once
■ Wait for all replies
■ Reduces number of network roundtrips that the client waits for
![Page 18: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/18.jpg)
+Reducing Memory Use
■ Method of reducing memory use
■ Ziplist – compact storage and unstructured representation of LISTs HASHes and ZSETs
■ Intset – compact representation of SET
■ As structures grow beyond limits, they are converted back to their original data structure type
■ Manipulating compact versions can become slow as they grow
{Short Structures}
![Page 19: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/19.jpg)
+Ziplist
■ Basic configuration for the 3 data types are similar
■ *-max-ziplist-value – max number of items to be encoded as ziplist
■ If limits are exceeded, redis will convert the list/hash/zset into non-ziplist structure
![Page 20: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/20.jpg)
+ZIPLIST - LIST
![Page 21: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/21.jpg)
+Intset
![Page 22: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/22.jpg)
+Sharded Structures
■ Sharding – takes data, partitions it to smaller pieces and sends data to different locations depending on which partition the data is assigned to
■ Sharding LISTs – uses LUA scripting
■ Sharding ZSETs – zset operations on shards violate how quickly zsets perform, sharding is not useful on zsets
![Page 23: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/23.jpg)
+Sharded Structures
■ Sharding HASHes■ Method of partitioning data must be chosen
■ Hash’s keys can be used as source of info for sharding
■ To partition keys:
■ Calculate hash function on the key
■ Calculate number of shards needed depending on number of keys we want to fit in one shard and the total number of keys
■ Resulting number of shards along with hash value will be used to find out which shard we’ll use
![Page 24: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/24.jpg)
+Scaling
■ In using small structures, make sure max ziplist is not too large
■ Use structures that offer good performance for the types of queries we want to perform
■ Compress large data sent to redis for caching to reduce network reads and writes
■ Use pipelining and connection pooling
{read capacity}
![Page 25: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/25.jpg)
+Scaling
■ Increase total read throughput using read only slave servers■ Always remember to WRITE TO THE MASTER
■ Writing on SLAVE will cause an error
■ Redis Sentinel■ Mode where redis server binary doesn’t act like the typical one
■ Watches behavior and health of master(s) and slave(s)
■ Intended to offer automated failover
{read capacity}
![Page 26: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/26.jpg)
+Scaling
■ Make sure to check all methods to reduce read data volume
■ Make sure larger pieces of unrelated functionality are moved to different servers
■ Aggregate writes in local memory before writing to redis
■ Consider using locks or LUA when limitations such as watch/multi/exec are encountered
■ When using AOF, keep in mind that the disk needs to keep up with the volume we’re writing
{memory capacity}
![Page 27: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/27.jpg)
+Scaling
■ Presharding for growth■ Run multiple redis servers on your machine (listen on diff. ports)
■ Use multiple redis database on your database server
{write capability}
![Page 28: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/28.jpg)
+Scaling
■ Scenario : machines have enough memory to hold index, but we need to execute more queries that server can handle
■ Use : SUNIONSTORE, SINTERSTORE, SDIFFSTORE, ZINTERSTORE, and/or ZUNIONSTORE
■ Since we “read” from slave, set : slave-read-only no
{complex queries}
![Page 29: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/29.jpg)
+Reference
■ Carlson, Josiah. (2013) Redis in Action. Shelter Island, NY: Manning Publications
![Page 30: Redis Beyond](https://reader033.vdocuments.site/reader033/viewer/2022051314/54b6bb3e4a7959fa048b4569/html5/thumbnails/30.jpg)
+
Thank You ☺