zmq driver (mitaka summit)
TRANSCRIPT
Copyright © 2015 Mirantis, Inc. All rights reserved
www.mirantis.com
ZMQ driver
Current state and further developmentOleksii Zamiatin
Software [email protected]
IRC, launchpad: ozamiatin
Copyright © 2015 Mirantis, Inc. All rights reserved
Well known HLD
Copyright © 2015 Mirantis, Inc. All rights reserved
Old Driver Message Path
request
reply
Copyright © 2015 Mirantis, Inc. All rights reserved
New Driver Message Path
request
reply
Copyright © 2015 Mirantis, Inc. All rights reserved
What Have Changed
Reduced Proxy-receiver. Dynamic port binding instead of static + proxy. (Thanks Alec Hothan for the idea! :))
Changed PUSH/PULL+PUB/SUB to REQ/REP BP:zmq-req-rep
Internal architecture become more flexible - easy to extend BP:zmq-patterns-usage, BP:zmq-driver-folder
Not depends on eventlet directly BP:zmq-work-without-eventlet
More clear topics management and message serialization
Copyright © 2015 Mirantis, Inc. All rights reserved
Client proxy to avoid blocking
requestreply
cfg.BoolOpt('zmq_use_broker', default=True) - controlled by the option
Copyright © 2015 Mirantis, Inc. All rights reserved
Why we still need a proxy?
Eventletlibzmq - c-libraryBLOCKING call to DEALER by specification
DEALER socket blocks when no listeners present
eventlet.green.zmq - wouldn’t handle the situation…
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
So-called pipeline. Driver consists of publisher-consumer pairs, each implements some OM patterns in terms of ZMQ.
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
Basic publishers/consumers compatibility
Based on ZeroMQ socket types compatibility
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
Each pipeline serves for some subset of OM patterns. Overlapping takes place.
Based on ZeroMQ socket types compatibility
Patterns:
R - CALLC - CASTCf - CAST+FanoutN - NotifyNf - Notify+Fanout
<-,C,Cf,N,Nf>
<R, - , - , - , - >
< - , - ,Cf, - ,Nf>
< - ,C, - ,N, - >
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
How can the old driver’s pattern fit into current architecture
Patterns:
R - CALLC - CASTCf - CAST+FanoutN - NotifyNf - Notify+Fanout
<R, C, Cf, N, Nf>
Hybrid publishers/consumers are possible with current approach
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
ZmqClient combines publishers in order to make a complete <R,C,Cf,N,Nf>.
Patterns:
R - CALLC - CASTCf - CAST+FanoutN - NotifyNf - Notify+Fanout
<-,C,Cf,N,Nf>
<R, - , - , - , - >
< - , - ,Cf, - ,Nf>
< - ,C, - ,N, - >
ZmqServer should match the client with corresponding consumers combination.
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
Patterns:
R - CALLC - CASTCf - CAST+FanoutN - NotifyNf - Notify+Fanout
<-,C,Cf,N,Nf>
<R, - , - , - , - >
< - , - ,Cf, - ,Nf>
< - ,C, - ,N, - >
Just as an example
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture Client
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture Server
Copyright © 2015 Mirantis, Inc. All rights reserved
Driver’s architecture from the inside
What we have now - the basic “reliable” pipeline to start from
More reliable but slower
Unreliable but faster
Blocking, so makes sense only with blocking executor
<-,C,Cf,N,Nf>
<R, - , - , - , - >
Copyright © 2015 Mirantis, Inc. All rights reserved
What ZMQ doesn’t cover
Comparatively to RabbitMQ, ZMQ doesn’t provide:Nodes discovery and managementUsers management and security (just
experimental encryption functionality)Persistence of messaging queues
Supporting of this functionality makes zmq-driver more complicated.
Copyright © 2015 Mirantis, Inc. All rights reserved
What should be done for production-readiness
Heartbeats, keeping connections, reconnectsAcks based retries (in progress)Persistence for queues (at least research
possibilities)Performance optimizations (optimize calls to Redis
implement ttl-based cache)Testing HA, scaling.Fix all bugs in the list