real world enterprise reactive programming using vert.x
DESCRIPTION
Shows best practices and lessons learnedTRANSCRIPT
Mariam Hakobyan,
Sascha Möllering,
Björn Stahl
Real World Enterprise Reactive
Programming using Vert.x
| zanox AG
TABLE OF CONTENTS
1.Introduction
2.The Beginning
3.How to start?
4.Best Practices
5.Architecture
6.Deployment
7.Vert.x module system
8.Integration with messaging system
INTRODUCTION ZANOX
Europe‘s leading performance advertising network
THE BEGINNING
Java Magazin 04.14:
Vert.x im Unternehmenseinsatz
Entwicklung und Betrieb von
asynchronen Applikationen mit Vert.x in
der Praxis
THE BEGINNING
● New core system for zanox
● Requirements are pretty high (not negotiable):
● Low latency
● High throughput
● Reactive
● Fast
THE BEGINNING
https://github.com/Mr-Steel/vertx_fatjar
“Vert.x is a lightweight, high performance application
platform for the JVM that's designed for modern
mobile, web, and enterprise applications.”
Vert.x
WHAT IS VERT.X?
WHAT IS VERT.X?
Polyglot
WHAT IS A VERTICLE?
● Classes with an own Classloader
● operates Single Threaded
● executed by an Event Loop
Event Loop Pool
Event Bus
V V V W W W
TAKE A LOOK INSIDE
HOW TO START?
● mvn archetype:generate -
Dfilter=io.vertx: (do not forget the
colon!)
● generates structure for all languages (JS,
Ruby, Groovy, Python)
● maven pom is already set with relevant data
HOW TO START?
Lessons learned - Profiling
● Verticles and WorkerVerticles appear in different
subtrees
● Be careful with blaming, a lot of stuff is just waiting
● If you think a verticle is a bottleneck: start more
instances of this verticle
● Measure every change & compare results (yeah yeah -
but really - do it!)
BEST PRACTICES
Starter Verticle
BEST PRACTICES
Logging
BEST PRACTICES
Logger Verticle
● Disadvantages:
● log entries are not in order
● loss of log levels
● Solutions:
●use IDs to follow log message flow
●use one LoggerVerticle per log level
BEST PRACTICES - On the fly deployment
BEST PRACTICES - On the fly deployment
● Do not block the loop
● put blocking code or extensive computation into
worker verticles
● Keep the application responsive
● stress test as often as possible
● Encapsulate common code in modules (more
on this later)
BEST PRACTICES
In General
fat-jar: one package to rule the deployment
BEST PRACTICES
Deployment
BEST PRACTICES
Deployment
zip-File:
● have a separate folder for scripts
● bind packaging to verify phase to have it all
with one command “mvn verify”
● complete example is on GitHub
Internet
Auto Scaling group
Auto Scaling group
Availability Zone
Availability Zone
Architecture
Architecture
● Main framework Vert.x
● mod-kinesis (Kinesis-Adapter for Vert.x)
● AWS Java SDK
● IAM roles for Amazon EC2 instances
● Coda Hale metrics and CloudWatch reporter
● Jedis (Redis client for Java)
● …
Deployment
WorkstationGitHub
Jenkins
Python/Fabric
Nexus
Datacenter
Artefakt
Deployment
Deployment
EC2 instance with Docker
registry (port 5000)
S3 bucket to store
Docker images
Deployment
Amazon Cloud
WorkstationGitHub
Jenkins
Python/Boto
Docker Image
Docker Registry S3 bucket with
Docker Images
Deployment
Deployment
Deployment
Deployment
python start_docker_instance.py \
-r <myregistry>:5000 \
-i kinesis-producer \
-t 95 \
-q 2 \
-s quality \
-d '-d -p 8080:8080'
https://github.com/SaschaMoellering/aws-docker-scripts
Deployment
python start_elb.py \
-r <myregistry>:5000 \
-i kinesis-producer \
-t 95 \
-s quality \
-d '-d -p 8080:8080'
https://github.com/SaschaMoellering/aws-docker-scripts
Deployment
Deployment
Vert.x Module System
Vert.x Module System
Benefits:
● Re-usable components as a zip file
● Maven and Bintray integration
● Vert.x module registry
● Vert.x downloads and installs modules for you
Vert.x Module Registry
Vert.x Event Bus
● Address is a simple string
● Types - Pub/Sub & P2P
Pub/Sub
P2P
Sender
Receiver
Integration with Messaging System
KAFKA MESSAGING SYSTEM
● Apache Kafka is a distributed publish-subscribe
messaging system
● Fast
● Scalable
● Durable
● Distributed
Integration with Messaging System
Main Terminology:
● Topic
● Producer
● Consumer
● Broker
Kafka Module in Module Registry
Open source Kafka module in Vert.x’ module registry
● http://modulereg.vertx.io/
Using Vert.x Module
// programmatically
// command line
Using Kafka Module
Configuration is a JSON object:
StatsD Support
Config
Metrics of Vert.x Project
On 4 Cores virtual machine we had the following
results:
●~28 K requests per second without Kafka,
with lookup from Redis
●~18 K requests per second with Kafka and
lookup from Redis
Metrics of Vert.x Project
Test Traffic ~1000-3000 requests per minute:
● mean response time: 1-2ms
● 90% of responses 0.86 ms
●fully cached: 0.65 ms
● CPU Load: 1-3%
● Memory: 90MB (300MB reserved)
GET INVOLVED
Kafka Module - https://github.com/zanox/mod-kafka
Kinesis Module - https://github.com/zanox/mod-kinesis
https://github.com/Mr-Steel/vertx_fatjar