nsq meetup-messaging
TRANSCRIPT
![Page 1: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/1.jpg)
NSQ decentralized messaging
Guillaume Arnaud @guillarnaud
voyages-sncf.com
1
![Page 2: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/2.jpg)
2
P publisher
C consumer
A Message-oriented Middleware (MoM)just a queue ?
![Page 3: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/3.jpg)
3
ActiveMQ RabbitMQ Nats NSQ ZeroMQ Kafka AmazonSQS Disque Aeron Mosquitto
Beanstalk Apollo Nanomsg …
JMS Stomp MQTT AMQP XMPP …
Flume Redis Logstash Syslog …
Brokers Protocol/API Almost
https://github.com/mfornos/awesome-microservices#messaginghttp://queues.io/
![Page 4: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/4.jpg)
4
ActiveMQ RabbitMQ Nats NSQZeroMQ Kafka AmazonSQS Disque Aeron Mosquitto
Beanstalk Apollo Nanomsg …
JMS Stomp MQTT AMQP XMPP …
Flume Redis Logstash Syslog …
Brokers Protocol/API Almost
https://github.com/mfornos/awesome-microservices#messaginghttp://queues.io/
![Page 5: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/5.jpg)
MoM.diversity
5
![Page 6: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/6.jpg)
NSQ.id
6
parents: nsq.io age: v0.3.8 (mar. 2016) license: open source native language: go fluently spoken: go, python, javascript, http(!) zodiacal sign: syslog++
![Page 7: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/7.jpg)
7
P
topic: server_start_eventnsqd
message A
![Page 8: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/8.jpg)
8
P
topic: server_start_eventnsqd
topic: server_start_eventnsqd
P
message Bmessage A
![Page 9: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/9.jpg)
9
P
C
topic: server_start_eventnsqd
channel: server_subscribe
topic: server_start_eventnsqd
channel: server_subscribe
P
consumer: server_subscribe
tcptcp
message A message B
![Page 10: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/10.jpg)
10
P
C
topic: server_start_eventnsqd
channel: server_subscribe
topic: server_start_eventnsqd
channel: server_subscribe
P
consumer: server_subscribe
tcptcp
message A message B
![Page 11: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/11.jpg)
11
P
C
topic: server_start_eventnsqd
topic: server_start_eventnsqd
P
consumer: server_subscribe
C
message A message B
![Page 12: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/12.jpg)
12
P
C
topic: server_start_eventnsqd
topic: server_start_eventnsqd
P
consumer: server_subscribe
C
message A message Bmessage A message B
![Page 13: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/13.jpg)
13
P
C
topic: server_start_eventnsqd
topic: server_start_eventnsqd
P
consumer: server_subscribe
Cmessage A
message B
message A message B
![Page 14: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/14.jpg)
14
C
topic: server_start_eventns
qdnsqlookupd
topic: server_start_eventns
qd topic: server_start_eventns
qd
server 1 server 2 server 3
subscribessubscribessubscribes
server 1 server 2 server 3declare server:
![Page 15: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/15.jpg)
15
server
nsqd
server
nsqd
server
nsqd
server
nsqd
push
pull
![Page 16: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/16.jpg)
NSQ.service discovery
16
1. application (container, tomcat, nodejs…) starts 2. application sends its information about who is it to admin 3. admin process receives and validates new configuration 4. admin fills a new haproxy configuration from a template 5. admin sends to haproxy its new configuration 6. haproxy sends confirmation of the reload to the admin
![Page 17: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/17.jpg)
17
nsqd
nsqd
nsqd
haproxydaemon
admin
tomcat
new_ip_port1
nsqd
tomcat
![Page 18: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/18.jpg)
18
nsqd
nsqd
nsqd
haproxydaemon
admin
tomcat
new_ip_port1
2
nsqd
tomcat
![Page 19: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/19.jpg)
19
nsqd
nsqd
nsqd
haproxydaemon
admin
tomcat
new_ip_port1
2
3
nsqd
tomcat
![Page 20: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/20.jpg)
20
nsqd
nsqd
nsqd
haproxydaemon
admin
tomcat
new_ip_port1
2
3
4
nsqd
tomcat
![Page 21: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/21.jpg)
21
nsqd
nsqd
nsqd
haproxydaemon
reload
admin
tomcat
new_ip_port1
2
3
4
nsqd
tomcat
5
![Page 22: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/22.jpg)
22
nsqd
nsqd
nsqd
haproxydaemon
reload
admin
tomcat
new_ip_port1
2
3
5
6
nsqd
tomcat
5
![Page 23: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/23.jpg)
23
nsqd
nsqd
nsqd
haproxydaemon
reload
admin
tomcat
new_ip_port1
2
3
5
6
7
nsqd
tomcat
5
![Page 24: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/24.jpg)
24
nsqd
nsqd
nsqd
haproxydaemon
admin
tomcat
nsqd
tomcat
my.really.hype.application.com
192.128.0.12:5123
![Page 25: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/25.jpg)
NSQ.ops
25
$ tree bin/ bin/ ├── nsq_pubsub ├── nsq_stat ├── nsq_tail ├── nsq_to_file ├── nsq_to_http ├── nsq_to_nsq ├── nsqadmin ├── nsqd ├── nsqlookupd └── to_nsq
![Page 26: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/26.jpg)
NSQ.ops
26
$ nsqd —help -e2e-processing-latency-percentile value … -max-bytes-per-file int -max-heartbeat-interval duration -max-msg-size int -max-msg-timeout duration -max-output-buffer-size int -max-output-buffer-timeout duration -max-rdy-count int -max-req-timeout duration -mem-queue-size int -msg-timeout string -snappy …
![Page 28: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/28.jpg)
your MoM is a part of your application
28
![Page 29: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/29.jpg)
your application is a part of your distributed system
29
![Page 30: Nsq meetup-messaging](https://reader034.vdocuments.site/reader034/viewer/2022042908/58ece51e1a28abf27a8b4617/html5/thumbnails/30.jpg)
Questions ?
30