data focused docker clustering
DESCRIPTION
See also the more detailed blog post at https://clusterhq.com/blog/data-focused-docker-clustering/ In this talk from the July Docker Edinburgh Meetup Luke Marsden, from ClusterHQ, explains how a data driven approach to clustering can improve robustness and and scalability, while reducing complexity.TRANSCRIPT
Data focused Docker clustering
Luke MarsdenFounder, ClusterHQ
clusterhq.comgithub.com/clusterhq/flocker
Background:
Container clusters where volumes are a first class citizen
Motivations:
robustnessscalability
eliminate SPOFsreduce complexity
Modern applications
are complex distributed systems
Containers and PaaS today
capture the easy, data-free bits
We should aim to capture the entire app
enabling larger class of apps to move to containers
Goals
Multi-node/clustered
Ease of operational manageability
Homogeneous serversNo single point of failure
Multi-regionData services the cluster depends on must
be protected by/within the system itself
Containers should be portable, even when they have state
(corollary: the system should expose portable, resilient volumes)
Dependent services or applications should not need to be reconfigured when a service
moves around
Stateful things scale vertically, stateless
things scale horizontally
Proposed implementation
Management layer
Strongly consistent group messaging semantics
Distributed consensus protocolsNo single master
Storage
ZFS: storage analogue to containers for compute
Independently snapshotted and replicatedSharding container volumes across a cluster
Redundancy invariantAutomatic slave allocation
Networking
AwesomeProxyany request to any node
Routes vs. Links
Live migration
Pausing in flight requestsQuiesce filesystem
Replicate last snapshotUnleash requests
Thank you - questions?
Check it out!github.com/clusterhq/flocker
clusterhq.com