Elixir: An Agent for Supporting Elasticity inDocker Swarm

Michail S. Alexiou, Euripides G.M. Petrakis

Abstract Elixir is an autonomous agent that extends the capabilities ofDocker Swarm in certain ways: (a) it supports monitoring of resources inreal-time, (b) introduces a scheduler for balancing the workload amongswarm nodes and, most importantly, (c) supports automatic scaling-up (ordown) of worker nodes at run-time with no downtime (i.e. without stop-ping the system). The proposed scaling strategy is reactive and works byresponding to static (pre-defined) thresholds applying on resource metrics(e.g. RAM, CPU). The experimental results demonstrate that Docker Swarmwith Elixir responds to the increasing (or decreasing) resource demands ofeach application leading to significantly faster response times compared toits non-auto scaled counterpart, while maintaining optimal allocation of re-sources per application.

1 Introduction

Scalability [3] is the ability of a system to increase (or decrease) its compu-tational resources in order to continue to function well, in response to in-creasing (or decreasing) workloads. A system is scalable if its performance(e.g. response time) does not change drastically with the size of the work-load. A real-time system whose response time increases (e.g. linearly) withthe workload will eventually become non real-time. Adding computationalresources (e.g. CPU or memory) may solve the problem (vertical scaling).If this is not good enough to guarantee good (e.g. uniform) performance,more instances of the application can be spawned in the same or differentphysical or virtual machines (horizontal scaling). Elasticity is a term relatedto scalability and refers to the property of a system to scale automaticallyat run-time while optimizing its use of computational resources. It appearsto be a key problem in cloud computing where the infrastructure must becapable of allocating computational resources on demand. However, mostof the times (the same in this work) these two terms are used the one in theplace of the other.

Michail AlexiouSchool of Electrical and Computer Engineering, Technical University of Crete (TUC), Cha-nia, Greece, e-mail: michalis.alex@yahoo.com

Euripides G.M. PetrakisSchool of Electrical and Computer Engineering, Technical University of Crete (TUC), Cha-nia, Greece, e-mail: petrakis@intelligence.tuc.gr

1

2 M. S. Alexiou, E. G.M. Petrakis

Service provision using VMs proved to be costly for both clients andproviders. Containers can be viewed as processes that are isolated from eachother, provide many of the benefits of virtualization in a lightweight mannerwithout the need of a hypervisor layer. They deploy much faster than VMsand are portable across systems running a different version of the OS but, donot provide the same high level of security and process isolation as VMs. Inthe environment of a cloud provider, there is a need for enhanced coordina-tion of resources allocated to virtualized applications. At the very basic level,Docker1 facilitates the build and management of applications deployed ascontainers. At a larger scale (e.g. at cloud provider level) Kubernetes2 andDocker Swarm3 are prominent platforms for managing and orchestratingmultiple applications using containers. An interesting idea implemented byboth platforms is to deploy containers on top of VMs thus combining the ad-vantages of both (i.e. highly isolated and secure applications which deployand run on VMs).

Scalability solutions may apply at any abstraction level of the virtualizedenvironment. In Docker and Docker Swarm, allocation of resources is static.New nodes can be added manually by issuing commands on Docker Com-mand Line Interface (CLI). ElasticDocker [1] is a solution powering verticalelasticity of Docker containers. Scaling up (or down) the containers them-selves within a node (rather than the node itself) is not a viable solution.Containers have access to the entirety of a node’s resources4 and will useas much of these resources (e.g. CPU, disk) as they need to. This solution isconstrained by the capacity of the node. If workload demands surpass thecapacity of the node, containers are migrated to another host machine withmore resources (vertical scaling). Migrating and restoring the application onthe new host machine has a negative impact on performance.

Elixir builds on top of Docker Swarm and extends its capabilities in cer-tain ways: (a) it supports monitoring of resources in real-time, (b) introducesa reverse proxy and scheduling mechanism for enhancing security and forbalancing the workload among swarm nodes and, (c) supports automaticscaling-up (or down) of worker nodes at run-time with no downtime (i.e.without stopping the system). The experimental results demonstrate thatElixir responds to the increasing (or decreasing) resource demands of eachapplication leading to significantly faster response times compared to a non-auto scaled implementation of Docker Swarm.

An introduction to Docker Swarm and the problem being solved are dis-cussed in Sec. 2. Elixir solution including architecture and workflows aredescribed in Sec. 3. Sec. 4 demonstrates the experimental results along withan in-depth analysis of the performance of Elixir using a realistic CPU in-

1 https://www.docker.com2 https://kubernetes.io3 https://docs.docker.com/engine/swarm/4 https://docs.docker.com/config/containers/resource_constraints/

Elixir 3

tensive application, followed by conclusions, system extensions and issuesfor future research in Sec. 5.

2 Background

Docker Swarm utilizes Docker CLI to create a cluster of Docker engines (theswarms). A swarm is a collection of nodes joined into a single cluster. Typ-ically, applications are installed in containers inside nodes. Many copies ofan application can run in different nodes within each swarm. Each swarmcomprises of a set of nodes which, in turn, can be either physical or vir-tual machines. Swarm nodes are distinguished into managers and workers.Fig. 1 illustrates a typical Docker Swarm deployment with N swarms. Eachswarm comprises of one manager and two worker nodes.

Fig. 1: Docker Swarm deployment with N nodes.

The managers handle swarm management tasks using Docker CLI com-mands such as, maintaining cluster state (i.e. they keep track of nodes be-longing to a swarm), scheduling services (i.e. they assign application con-tainers to nodes, they distribute containers among the swarm nodes) andscaling-up (or down) containers and swarm nodes (i.e. new nodes can beadded in a swarm). Deploying an odd number of managers is recommendedin order to guarantee fault tolerance (i.e. the system recovers without down-time when a manager node fails5). However, it is still possible to run aswarm with a single manager. Manager nodes are also responsible for de-ploying application containers, for balancing the workload within a swarm,and for authorizing other machines to join a swarm as workers. They canalso act as workers themselves thus sharing the workload with workers.Worker nodes provide computational resources to the system (i.e. for exe-cuting the tasks assigned to them by the managers) and have no other au-thority (e.g. to start a service on their own). Every worker or manager nodehas a Docker engine installed. Docker Swarm provides the system with an“Ingress Network”. The Ingress Network is an overlay network which han-

5 https://docs.docker.com/v17.09/engine/swarm/how-swarm-mode-works/nodes/

4 M. S. Alexiou, E. G.M. Petrakis

dles control and data traffic related to swarm services. It is the default net-work assigned to swarms and supports communication between nodes ofdifferent swarms.

If the system experiences a heavy workload and a derating of perfor-mance is observed, the solution is to increase the number of nodes (hori-zontal scaling). New nodes must be allocated and join an existing swarmmanually by issuing commands on Docker CLI6. As of yet, there is no mech-anism to scale up (or down) the number of nodes automatically.

3 Elixir Solution

Elixir, is an autonomous agent written in Python 2.7. It runs together withDocker Swarm and is capable of managing multiple applications at thesame time. Swarm (manager and worker) nodes are deployed as VirtualMachines (VMs). This not only results in better utilization of computationalresources (i.e. CPU, memory and network are defined virtually) but also,in enhanced software security by isolating applications from each other.Fig. 2 illustrates Elixir’s architecture. Compared to Docker Swarm deploy-ment of Fig.1, Elixir installs an additional Swarm (referred to as “ElasticitySwarm Cluster”) whose purpose is to monitor the swarms of an infrastruc-ture provider and balance the workload among nodes running the sameapplication (i.e. nodes that belong to the same swarm).

In order to support decisions on whether to scale-up (or down) a swarm,a resource monitoring mechanism is installed in the manager node of eachswarm. Scaling decisions are taken by Autoscaler which runs on top of Elixirand is controlled by the infrastructure provider. Autoscaler takes scaling de-cisions based on actual resource monitoring information it receives by Elas-tic Manager node. In turn, Elastic Manager periodically probes the swarmsand collects utilization metrics. For each application (i.e. swarm) a decisionto scale (or not) is based on rules checking whether the computational re-sources of the swarm are within predefined limits.

Elixir does not create new swarm clusters, it only takes over manage-ment of existing swarms, monitors the swarms and takes decisions to scalethe swarms up (or down). In response, new worker nodes can be added toa swarm (or deleted). There is no need for scaling the number of managernodes of a swarm. Docker swarm recommends an odd number (i.e. max-imum seven) of manager nodes7 in order to guarantee fault tolerance anduninterrupted operation.

3.1 Architecture

The architecture of Elixir is described below focusing on new componentsand changes introduced to Docker swarm components. Manager and workernodes are deployed as Docker engines.

6 https://docs.docker.com/engine/swarm/swarm-tutorial/add-nodes/7 https://docs.docker.com/v17.09/engine/swarm/how-swarm-mode-works/nodes/

Elixir 5

Fig. 2: Docker Swarm deployment with Elixir agent.

3.1.1 Elasticity Swarm Cluster - Elasticity Manager node

This new swarm deploys one manager node (but no worker nodes) referredto as Elasticity Manager node that works as an orchestrator and takes controlof the entire Docker Swarm system. It is responsible for, managing multi-ple swarms (applications), balancing the workload among the nodes, takingscalability decisions and (in response) for adding or removing worker nodesin each swarm. Similarly to other Docker swarms, an odd number of Elas-ticity Manager nodes can be deployed as well (to ensure fault tolerance).Elasticity manager node runs the following services in containers: One con-tainer running Zabbix server8, one container with a MySQL database, andN containers running NGINX9 server with N being the number of swarms.Fig. 2 illustrates services running in this node.

Each NGINX server (container) looks after one swarm and is entitled toaccess the Ingress Network. Its role is twofold: First, it acts as a reverse proxystanding between the client and the swarm nodes (notably the applications).It forwards client requests to the applications while hiding their IPs from theclients. Second, it acts as a scheduler which is responsible for distributing theworkload (i.e. system requests) to the worker nodes according to a schedul-ing algorithm (i.e. round robin in this implementation). Each NGINX con-tainer has a configuration file which lists the IP addresses of the swarmnodes in the “upstream backend" section. Listing 1 is an example configura-tion file with three nodes. Every time a new node is added (or deleted), theconfiguration file is updated on the fly (i.e. in the case of scaling-up, the IP ofthe new node is appended in the upstream backend section) so that, restart-ing NGINX server is not necessary. NGINX will resume operation with theupdated configuration in place and no downtime is observed.

Listing 1: NGINX configuration file for 3 swarms.http {

server {listen 80;

8 https://www.zabbix.com9 https://docs.nginx.com/nginx/admin-guide/web-server/reverse-proxy/

6 M. S. Alexiou, E. G.M. Petrakis

location / {proxy_pass http://backend;

}}upstream backend {

server 192.168.111.201:80;server 192.168.111.202:80;server 192.168.111.203:80;

}}

Besides working as a reverse proxy server and scheduler, Elasticity Serverruns a monitoring service with a Zabbix server installed in a separate con-tainer. A single Zabbix server is capable of keeping track of all swarms. Elixirwill identify the manager nodes running in each swarm and will turn themto monitoring hosts by installing a Zabbix agent10 in each one. Zabbix agentcollects resource usage metrics (e.g. CPU, RAM, disk utilization), computesaverage resource utilization values periodically (i.e. every five minutes) andreports this information to Zabbix server. In turn, Zabbix server collects thisinformation from all agents and checks triggers (i.e. whether their values arewithin the predefined thresholds).

Listing 2: Trigger create example{“jsonrpc": "2.0",“method": “trigger.create",“params": [ {

“description":“Processor load is too low on HOST.NAME",“expression":

“{Template OS Linux:system.cpu.util[,idle].avg(5m)}<20",“dependencies": [ { “triggerid": “17367" } ]

} ]}

Rules for triggering autoscaling are defined in Zabbix server. If the rulesare violated, Zabbix server reports an event to Zabbix Web front-end. Asit is typical for reactive scaling mechanisms, triggers are defined based onlow and high threshold values for utilization metrics. Zabbix supports disk,memory and CPU-based triggers, as well as custom triggers defined as acombination of threshold values. Different triggers can be defined for eachswarm (application). Listing 2 illustrates a trigger. “Description" is the alertmessage that the infrastructure provider will receive, “expression" is themacro expression which checks whether the trigger is satisfied or not, and

10 https://www.zabbix.com/zabbix_agent

Elixir 7

“dependencies” is a parameter added only if a custom trigger requires thevalues of another trigger. In this example, “expression” checks whether av-erage CPU utilization during the last 5 minutes is above 80% (i.e. if the av-erage CPU idle time is less than 20% then, the average CPU utilization isabove 80%).

History data are stored in a MySQL database installed in a separate con-tainer so that, they can be analysed for detecting regularities and hidden as-sociations in monitored data. This is important for modeling system behav-ior (e.g. for finding trends of permanence and resource utilization which isuseful information for implementing predictive elasticity policies for Elixir[2] (left as future work).

3.1.2 Autoscaler

This service is responsible for taking decisions to scale-up (or down) aswarm by adding (or deleting) worker nodes. It checks periodically (e.g.every one minute) Zabbix server Web front-end from where it gets noti-fied whether triggers have been violated and if so, it initiates a scale-up (ordown) operation in response. Autoscaler contacts infrastructure provider(OpenStack in this work) and inquires an additional VM (or to de-allocate aVM). The infrastucture provider creates a new VM, its network is initialized(e.g. IPs are assigned) and the VM joins Ingress Network. It is a responsibil-ity of the Autoscaler to update the NGINX configuration file of the swarmwith the IP of the new node (or to delete the IP of the de-allocated node).Finally, Elixir installs Docker and a new copy of the application in the node.

3.1.3 Swarm Cluster - Manager node

In a typical Docker Swarm deployment, its role is to preserve consistent in-ternal state of the swarm and balance the workload load among its nodes.However, this feature is overridden by the NGINX service managing theswarm. The rationale behind this decision is to allow management of multi-ple swarms (i.e. applications) at the same time in the same place and add se-curity to the entire system. Elixir is entitled to command each manager nodeof a swarm to scale-up (or down) and will turn each manager node into amonitoring host by installing Zabbix agent in it. Zabbix agent is then respon-sible for collecting resource utilization metrics from all worker and managernodes, and for processing this information. Average utilization metrics arecomputed periodically (i.e. every five minutes). Zabbix agent has a config-uration file which declares the IP of the Zabbix server where the agent isreporting.

3.1.4 Swarm Cluster - Worker node

It executes commands it receives from the manager node. New workernodes can be added to the swarm (or deleted from the swarm) if instructedto do so by the manager node (which, in turn are controlled and instructedto do that by Elasticity Manager node). Once a new node is created, Au-

8 M. S. Alexiou, E. G.M. Petrakis

toscaler will update NGINX’s configuration file in Elasticity server with theIP address of the new node.

3.2 Workflows

Elixir works as an agent and controls the entire Docker Swarm system. In or-der to achieve that level of control, Elixir must first find the manager nodesof each swarm. Algorithm 1 summarizes this process. Elixir searches theswarms for manager nodes and groups manager nodes by application. Thisis achieved by issuing “docker node ls" command on Docker CLI which, foreach node, determines, whether this node is a manager or not and to whichswarm it belongs to. For each application, one manager node is selected (atrandom) to be the monitoring host. Elixir will keep track of one managernode for each swarm and controls the swarm using that node. Should amonitoring host fail (e.g. it is not responding), Elixir will select a new moni-toring host by invoking Algorithm 1 again. For a swarm with one managerand worker nodes this process takes less than 30 seconds.

Algorithm 1 Finding all manager nodes.

1: procedure FIND ALL MANAGER NODES

2: Nodes List← All Nodes3: for node ∈Nodes List do4: if node contains ’Manager’ then Add node to manager nodes

Once the managers are identified, for each swarm a monitoring host isdefined in Zabbix server and Elixir starts monitoring the swarms. For eachswarm, a separate NGINX configuration file is created. Zabbix server willperform periodical health checks (i.e. every one minute) on each swarmkeeping track the trigger values. For each swarm (i.e. application) appropri-ate triggers are defined in Zabbix server. Each swarm (i.e. its Zabbix agent)reports average resource utilization metrics every five minutes in order toallow a swarm to adapt after a scaling operation and to reduce the impactof values received before scaling. Algorithm 2 summarizes this process. Ifthe trigger values for a swarm are surpassed, Zabbix server will report theevent to Zabbix Web front-end, Autoscaler (which monitors Zabbix server)will notice the event and will initiate a scaling-up (or down) process.

3.2.1 Scale-up

Algorithm 3 outlines the scaling-up process. Autoscaler will first contact in-frastructure provider (OpenStack in this work) and request authorizationto access its virtualization and networking services (Nova and Neutron re-spectively). The service responsible for granting access to Nova and Neutronis Keystone. If authorization is granted, Autoscaler will order OpenStack’sNova service to create a new VM with the flavor and OS specified (e.g. smallflavor with 1 CPU 2,800MHz, 2,048MB RAM, 20GB hard drive capacity run-

Elixir 9

Algorithm 2 Elixir’s basic workflow

1: procedure ELIXIR(thresholds)2: Initialize Zabbix Clients3: initialize Zabbix Server← CreateTriggers4: Initialize NGINX5: Find All Manager Nodes6: loop7: Select Monitroing Hosts8: for node ∈MonitoringHosts do9: Health Check swarm

10: if ‘Scale−up′ then Add Worker11: else if ‘Scale−down′ then Delete Worker12: wait 1 minute

ning Ubuntu 18.04). The new VM and its network are initialized (e.g. IPs areassigned) and becomes part of the Ingress Network of Docker Swarm auto-matically thanks to the OpenStack’s Neutron Software Defined Networking(SDN) capabilities. The new VM will join the swarm as a worker node. Therest of the scale-up process is carried out by Elixir which orders the man-ager of the swarm to create a name and a join token for the new workernode. Elixir passes this token to the worker node so it can join the swarm.Elixir will then add the IP of the worker node to the monitoring host (i.e. us-ing “host-create” command of Zabbix API) and to NGINX in the ElasticityServer (i.e. the IP is added to its configuration file).

Algorithm 3 Add Worker node.

1: procedure ADDWORKER

2: Get a name for the new node3: OpenStack Nova creates new VM with OS and Docker4: The VM gets a join token from swarm manager5: The VM becomes worker node and application is installed6: The VM joins Ingress Network7: Zabbix Server is updated8: NGINX server is updated9: Swarm manager is instructed to scale-up

3.2.2 Scale-down

If the decision is to scale-down, Autoscaler will free a worker node (oneworker node at a time). Algorithm 4 outlines this process. Elixir creates alist with the worker nodes of the swarm. The last one is poped-up fromthe list and put in“Drain mode” (i.e. this prevents a node from receivingrequests from the swarm manager). The node (i.e. its IP) is deleted fromNGINX configuration file. Finally, infrastructure provider deletes the VM.An alternative would to keep the VM in reserve so that it can be re-used bya scaling-up operation).

10 M. S. Alexiou, E. G.M. Petrakis

Algorithm 4 Delete Worker node.

1: procedure DELETEWORKER

2: list[]← Create list of worker nodes3: if list is NotEmpty then return4: worker← pop-up list5: Put worker in Drain mode6: Delete worker from NGINX server7: Delete worker from the swarm8: Swarm manager is instructed to scale-down9: Delete worker from the infrastructure

4 Performance Evaluation

Elixir is deployed in OpenStack (Kilo release) as a cluster with one man-ager, no workers but with Elixir in place. The manager acts also as worker.The swarm node is deployed in a small flavor VM with one processor2,800MHz, 2,048MB RAM, 20GB hard drive capacity, runs Ubuntu 18.04 andhas Apache HTTP server installed (that handles HTTP requests). Autoscalerruns on a second small flavor VM.

The application finds the shortest path on a map with 100 cities. Eachrequest executes Dijktra’s algorithm on a graph with 100 nodes. The appli-cation runs in one container and is stressed with many thousands requestsout of which, 10 are executed in concurrently. The purpose of the experimentis to demonstrate that Elixir scales-up gratefully keeping response time loweven when the requests reach 20,000 (the crash limit of the no-autoscale so-lution). A scale-up operation is triggered when average CPU utilization ex-ceeds 95%.

Fig. 3: Response time of Elixir as a function of the number of requests whilescaling-up.

Fig. 3 illustrates average response times of Elixir as a function of the num-ber of requests. The performance of the naive Docker Swarm solution (with-out scaling) is also shown for comparison. The system crashes for more than

Elixir 11

20,000 requests. A solution to this problem would be to engage automati-cally as many worker nodes as necessary in order to maintain response timeat the desired level.

CPU utilization exceeds the threshold after 1,071 requests and this ini-tiates a scaling-up operation and a request to OpenStack to create a newworker node (i.e. a new VM). The new node will join the network as soonas NGINX configuration file is updated with the IP of the new worker node.This is a rather slow process that takes up to five minutes to complete. Mostof this time accounts for the time to create a new VM in OpenStack (lat-est versions of OpenStack reduced the VM creation time to a few seconds).An alternative would be to allow spawning VMs in advance so that theyare available once a scaling operation is initiated. Scaling-up is triggeredagain after 2,321 requests. The third worker node joins the system and startssharing the load. Scaling-up is triggered for third time after 6,439 requestsand for last (fourth) time after 10,000 requests. Finally, the system comprisesof six VMs out of which, four are the new worker nodes. During the ex-periment, CPU utilization is approximately 100% (close to threshold) whilememory utilization is always below 20% on the average. After each scaling-up operation, CPU utilization drops to 94% (below the threshold) and thenstarts rising again as the number of requests increases.

Fig. 3 illustrates the transition between trigger upscale and system expan-sion stages as a series of vertical red and green color lines that alternate witheach other. The experimental results demonstrate that Elixir clearly outper-forms a typical Docker Swarm implementation several times. The responsetime increases only marginally with the workload accounting mainly for theincreasing demands for managing the swarm. After the addition of the 4th

worker node the system is capable of handling up to 8 requests per second.A static system with a single manager node and no elasticity cannot handlemore than 1.7 requests per second.

The purpose of the next experiment is to study the behavior of Elixir dur-ing down-scaling. A scale-down operation is triggered when average CPUutilization becomes less than 30%. Elixir starts with three nodes (one man-ager and two workers) running the shortest path application on a map with50 cities. Elixir soon observes the over-utilization of resources and triggersa scale-down operation although the number of requests is increasing. Thefirst scale-down operation is triggered after 700 requests and the operation iscompleted after 1,000 requests. The time for de-allocating one node (VM) isless than 1 minute. A second scale-down operation is initiated after 1,500 re-quests and de-allocation of the second worker node is completed after 1.930requests. Fig. 4 summarizes these results. Trigger down-scale and node dele-tion stages are shown as a sequence of vertical red and green color lines (i.e.as a series of events that alternate with each other). Finally, the system runswith a single node and is still capable of dealing with the load (i.e. no up-scaling is necessary). Only after 10,000 requests, system response time willrise again (the same as CPU utilization) indicating that scaling-up will even-

12 M. S. Alexiou, E. G.M. Petrakis

tually be necessary. It is important to observe the small footprint of Elixircausing a small (i.e. marginally detectable) overhead in performance com-pared to Docker Swarm.

Fig. 4: Response time of Elixir as a function of the number of requests whilescaling-down.

5 Future Work

Elixir’s current version works on OpenStack. Extending it to work withmore virtualized platforms is a natural extension of this work. Incorporatingan elaborate decision-making mechanism for triggering scaling and turningElixir from reactive to proactive is an important direction for future research.The main idea for this advance is to benefit from machine learning and dataanalytics and design a custom scaling mechanism which better fits the pecu-liarities of different applications. The result would be a scaling mechanismwhich foresees and adapts to variations of system behavior and to the uti-lization of resources of each application, and takes optimal scaling decisionswell in advance. This would potentially power an additional service for ap-plying container-level operations such as altering the resources allocated toapplications by taking decisions for optimal placement (migration) of work-load by moving nodes across the swarms.

References

1. Y. Al-Dhuraibi, F. Paraiso, N. Djarallah, and P. Merle. Autonomic Vertical Elasticityof Docker Containers with ELASTICDOCKER. In Intern. Conf. on Cloud Computing(CLOUD’17), pages 472–479, Honolulu, USA, 6 2017.

2. P. Bodík, R. Griffith, C.A. Sutton, A. Fox, A. Fox, M.I. Jordan, and D. Paterson. Statis-tical machine learning makes automatic control practical for internet datacenters. InProc. of the 2009 conference on Hot topics in cloud computing (HotTopics 2009), Berkeley,CA, USA, 6 2009. USENIX Association.

3. P. Sharma, L. Chaufournier, P. Shenoy, and Y. C. Tay. Containers and Virtual Machinesat Scale: A Comparative Study. In 17th Intern. Middleware Conf., pages 1:1–1:13, NY,USA, 12 2016.