dynamically scaling web applications in amazon ec2

8/6/2019 Dynamically Scaling Web Applications in Amazon EC2

http://slidepdf.com/reader/full/dynamically-scaling-web-applications-in-amazon-ec2 1/19

Dynamically Scaling Web Applications inAmazon EC2

Ramesh Rajamani

Blog : www.techmasala.com Email : [email protected]

Twitter : http://twitter.com/rramesh

http://www.techmasala.com/

mailto:[email protected]


http://twitter.com/rramesh


http://www.techmasala.com/





Disclaimer The approach and the scripts that are provided along with this whitepaper is “AS IS” without any warranties of any kind. The scripts have not been thoroughly tested under all conditions. I, therefore, cannot guarantee or imply reliability, serviceability, or

function of these programs.



Table of Contents

Disclaimer ................................................................................................... 2Introduction ................................................................................................ 4

Assumptions ............................................................................................... 4The Problem ................................................................................................ 4The Setup .................................................................................................... 5

Elastic IP .......................................................................................................... 6The Approach ............................................................................................. 7Nginx Configuration ................................................................................... 9Nagios Configuration ............................................................................... 11The Shell Script Web Server ................................................................... 14

The Approach ................................................................................................ 14Examples ....................................................................................................... 14The Script and Configuration bindings .......................................................... 15The Scripts ..................................................................................................... 15Adding to Xinetd service ............................................................................... 15

Client Side Script ...................................................................................... 16Finally ........................................................................................................ 18What Next? ................................................................................................ 18Appendix A – References ....................................................................... 19



IntroductionThis whitepaper attempts at a solution to scale your web applications running on EC2instances dynamically using simple shell scripts. The whitepaper uses Linux instances,with Nginx acting as the load balancer and Nagios for monitoring the instances. Theapplication server could be anything but for this whitepaper, am assuming a Railsapplication running on Mongrel and using the mongrel cluster. While these technologiesare specific to this whitepaper to provide an easy understanding, what is important here isthe approach and if one understands it, s/he could apply it using any technology that fitsthe respective roles.

AssumptionsThis document assumes that you have fair amount of knowledge working with Linuxsystems, and you know and have worked with Nginx and Nagios and their setup andconfiguration. Also this whitepaper assumes that you know how to work with AmazonEC2 instances such as to bring up and bring down instances and bundling and uploadingthe AMI’s to S3.

The ProblemEven though the Amazon cloud is a dynamic computing environment that enables one to

bring in or run as many instances at any given point of time, the reality of all theseinstances working together as a cluster with load balancing and scaling up or down asneeded is far from true. Let us take an example to see why high scalability is a possibilitythan something that is available off the shelves.

A highly scalable environment means that the resources that are setup can individually or

together be scaled up or scaled down based on the demand and the amount of utilizationof these resources. When a program or an application that is run on an environment is asindependent from any program or application or an environment that lives physicallyseparate on a different environment, then that program or application can be highlyscalable. This also includes state information that needs to be shared. Think of acalculator program that accepts an input, processes it and gives back a result. Now this

program is only dependent on the environment where it runs. One can bring in as manysimilar environments as needed and route a user to any of these instances to be servedwithout having to worry about partially served transactions and thereby inconsistency. Onthe other hand take a web application as an example. Typically a web application might

be dependent on a database or a service from third party application or vendor that could

be residing outside of the environment. Also the application might be dependent onmaintaining the state of a user for continuity and session validity for performing anaction. In such a case the difficulty in scaling up or scaling down the environment

becomes tight as there are dependencies that need to be considered.

Given these dependencies when a web application is configured to run on environment itis dependent on information that will enable it to communicate with its dependencies. For example a database configuration that helps the web application knows where the



Application Server 1

Application Server 2

Application Server N

Database Server Load Balancer

Figure 1

As per the above setup we will have three AMI’s as described below.

Load Balancer – This AMI will contain the Nginx web server that will help us do theload balancing of the application in the application server. This instance will also containthe Nagios monitoring software. We will see how we can add the app server instancescan be added to the monitoring service for health check up.

Application Server – This AMI will contain your application. As mentioned earlier wewill be assuming this is a Rails application running on Mongrel and Mongrel Cluster

plugin. But in fact this could be any web application running on any technology. All weare going to do is write some script that will update our load balancer configuration of thenew instance and the application.

Database Server – This AMI contains the RDBMS required by the web application.

Elastic IPTo make our lives easy we will associate an Elastic IP for our load balancer instance and

the database instance. Since these two instances are single and will not change we willuse the Elastic IP and ensure that the Elastic IP address is used wherever we refer to theseinstances.The following information will be used for reference within this document and in thescripts. This is just dummy data and you should use the actual information pertaining toyour environment before making use of the scripts.

Load Balancer Instance



Elastic IP Address – 111.222.111.1Public DNS - ec2-111-222-111-1.compute-1.amazonaws.com

Database Server InstanceElastic IP Address – 111.222.111.3Public DNS - ec2-111-222-111-3.compute-1.amazonaws.com

Application Server InstanceDynamic IP address – Gets assigned when starting an instanceApplication Mongrel cluster running on ports 8500, 8501, 8502

Notice that since we are using mongrel cluster, now this application is kind of verticallyload balanced through mongrel and then horizontally load balanced through NGINX. Inyour case it could be just one instance of the application running on an application server.Accordingly you might have to update the scripts.

The ApproachIn order for dynamically scaling the application server we need to make sure that we can

meet the flow depicted in Figure 2. As you can see the key thing is this whole thing canwork if we are able to call a program on the load balancer server when an instance islaunched and when it is taken down. This program should assist us in adding or removingan instance to the load balancer configuration and the monitoring applicationconfiguration.

The server application could be written in any programming language that can listen on aspecific port for requests and then it can call some native commands to update theconfiguration files and restart the load balancer and the monitoring application. Sinceafter receiving the request all the tasks performed by this server are commands to beexecuted in the shell, why not this server itself is created using shell script? My

inspiration in making it came from sites that I stumbled upon that had articles on creatinga web server using shell script 1 and using Bourne shell 2.

Using the above references I created a web server in shell script that accepts REST URLtype requests that is parsed and based on the request changes the load balancer andmonitoring configuration and restarts.

Before we get into the details of how this server works, let us see how we can setup the Nginx configuration and Nagios configuration to take in dynamically instances that needto be load balanced and monitored respectively.

1 A Web Server in a Shell Script - http://www.debian-administration.org/articles/3712 Writing a web server in Bourne Shell - http://sprocket.io/blog/2008/03/writing-a-web-server-in-bourne-shell/

http://www.debian-administration.org/articles/371


http://sprocket.io/blog/2008/03/writing-a-web-server-in-bourne-shell/







Start an application server instance

Get this

instance’s IPaddress

Call service on loadbalancer, pass this IP and

application ports to beadded to NGINX

configuration

Call service on loadbalancer, pass this IP andask it to add this instance

for monitoring

Instance Bootand ServiceInitializations

InstanceReady, serve

requests

ShutdownRequest


ask it to remove thisinstance from monitoring


application ports to beremoved from NGINX

configuration

ShutdownProcess

Instance Terminate

Requests proxied from NGINX

Figure 2 - The scaling flow



Nginx ConfigurationTypically the Nginx configuration is in the file nginx.conf under the conf directory of

Nginx home directory. As a standard practice it would be easy to maintain and separate,application specific configurations in a separate configuration file under a separate folder and include the files in the main configuration file using the include directive. I prefer tocreate a separate directory include.conf under Nginx home directory and have separateconfiguration file for every application I want to be served through Nginx. Below is asample section of the main configuration file that includes the other configuration filesunder the include.conf folder.

……

include ../include.conf/*.conf;……

Notice that you don’t have to include the configuration files individually but use wildcardto include all the files that have the suffix .conf.

Next we want to configure Nginx for a virtual host configuration for a specific domainand we want Nginx to proxy those requests to the application on the application server.We not only want to proxy the request but also want Nginx to load balance and pass therequests to one of the instances of the application.

Here is an example configuration application.conf which needs to be placed under theinclude.conf folder.

# application.cluster will hold the available clusters# at any given point of timeinclude ../include.conf/application.cluster;

server {listen 80;server_name www.example.com;access_log logs/example.access.log main;error_log logs/example.error.log;

location / { proxy_pass http://clustapp;

proxy_set_header X-Real-IP $remote_addr;error_page 500 502 503 504 /50x.html;

# Other parameters}

}

What are important in the above example is the first line that includes another file and theline that has proxy_pass keyword. The proxy_pass basically says proxy all the requestscoming to this server (defined by www.example.com listening on default HTTP port 80)



to http://clustapp. Where is this clustapp and what it is? Well it is a way of defining theroute where the request has to be reflected. Let us look at a sample of whatapplication.cluster file contains.

upstream clustapp { ip_hash;

server 100.110.120.130:8500;server 100.110.120.130:8501;server 100.110.120.130:8502;

}

The first line is the upstream directive that helps in configuring the load balancer for anapplication. As you can see the name clustapp is what we used as http://clustapp in the

proxy_pass directive. Nginx picks up an instance out of the list and passes the request.Refer to the Nginx manual for more detail on how Nginx load balancing works using theabove syntax.

One of the problems we discussed was session sharing and the difficulties in sharing auser’s state in a clustered environment. As also said that one of the easiest approaches isto ensure that a client’s request is always sent to the same instance from the moment theclient hits the server. This is also called as session stickiness. This is easily achievedthrough Nginx through the ip_hash directive.

As you can see in the above example there are three server entries, which means Nginxcan use those servers for load balancing. The IP address of all the three servers are thesame but the ports are different. In our case as we said our application is a Railsapplication using mongrel cluster those are nothing but the three instances of the sameapplication run by mongrel on the same server on ports 8500, 8501 and 8502

respectively.

But since we said that the application servers are going to be dynamically addedwhenever the instances are run, by default before starting our instances this file would beempty as below.

upstream clustapp {ip_hash;

}

We will use our shell script web server to add and remove the servers with ports as andwhen an instance is added and removed.

When the configuration is changed it is not picked up by Nginx automatically. If we stopand start the service it is possible that some of the client requests are not met or some of the clients might be routed to different instance in the cluster which may not be aware of the user’s session and the user might be taken to a login page all of a sudden. But Nginx

picks up the configuration when it is reloaded. To reload the nginx configuration one hasto kill the nginx process with the HUP signal. This ensures that nginx reloads itself



thereby loading the latest configuration and at the same time preserving clientinformation and the routing.

Nginx doesn’t come with any script that allows starting, stopping and reloading theserver. When you download the scripts along with this white paper it contains a script

nginx.server that will help doing the starting, stopping and reloading Nginx server. Thisscript is used in the cluster.modify script to reload Nginx after modifying the cluster details.

Nagios Configuration Nagios is a monitoring software that can help in checking if things are going right in aserver through certain commands passing in some parameters and evaluating the resultversus the expected. If things are wrong then it sends out notification emails so thatsomeone can take care of it. Nagios can not only monitor the server on which it resides

but can also monitor any remote server through its plugin called NRPE. NRPE can run asa daemon on a remote server, execute the commands that can check the status of various

things on the server and send back a status to the main Nagios server.

The servers that need to be monitored and the services that need to be monitored on eachof these servers are configured using configuration files on the Nagios server. For our application server we need to have a configuration file and the services defined. For the

purpose of simplifying things we will show a simple configuration file with only oneservice to check the CPU load of the server. You could always add more services thatyou want to be monitored. Refer the Nagios documentation for more details on how toconfigure a server and services to be monitored.

The below is an example Nagios configuration file for our application server.

define host{name appserver ; Name of this templateuse generic-host ; Inherit default valuescheck_period 24x7check_interval 5retry_interval 1max_check_attempts 10check_command check-host-alivenotifications_enabled 1notification_period 24x7notification_interval 30notification_options d,r

contact_groups adminsregister 0 ; DONT REGISTER THIS - ITS A TEMPLATE}

define host{use appserver ; Inherit default values from a templatehost_name appserver-1 ; Name of this serveralias Application Server - 1 ; A longer nameaddress 111.222.111.1 ; IP address of the server

}



http://host:port/cluster/scale/application/up/10.10.10.1

To add monitoring to an EC2 instance with the private address 10.10.10.1, the client hasto call the server with the following URL.

http://host:port/cluster/monitor/application/10.10.10.1/add

To remove monitoring of an EC2 instance with the private address 10.10.10.1, the clienthas to call the server with the following URL.

http://host:port/cluster/monitor/application/10.10.10.1/remove

The Script and Configuration bindingsThe shell script web server is dependent on certain configuration requirements for it towork properly. If you have noticed in the above examples, there is a parameter “application”. The script uses this to look for some files in the load balancer configuration and as well as in the Nagios configuration.

Nginx – The file application.cluster is being looked for in the <nginx home>/include.conf directory where the server:port is added or removed whenever a scale up or down iscalled.

Nagios – The file application.tpl is being looked for in the <nagios home>/templatedirectory and is used to replace the label @boxname and the replaced file is placed under <nagios home>/etc/cluster/application-ip-address.cfg file.

The ScriptsThere are two scripts that are used to achieve the whole configuration setup and restart of

Nginx and Nagios services.

scale.server – This is the core script that receives all the requests coming from the client.It parses the requests, gets the parameters, does a bit of validation and then calls theappropriate function to scale or to monitor.

cluster.modify – This script is called by the scale.server when the request is to add aninstance to be added to the nginx load balancer configuration. This script is kept separatefor the sake of better readability, maintainability and to use it separate or with other scripts if need be.

The above scripts are available along with this white paper in the zip file. The script hasnecessary comments to understand where what happens.

Adding to Xinetd serviceWe want the scale.server to be listening on a specific port for processing the requests andalso we want this to be a service that gets started when load balancer server starts. So we



This can be achieved by writing a couple of simple shell scripts that can be run during thestartup and shutdown. In both the scripts we need to be able to find the IP address of thisinstance so that we can pass that to our script server. For this I am thankful to the scriptthat I found on the nixCraft website 3. The script is saved as an executable shell script,findip in the /usr/bin folder for easy access.

Below is the script readyToServe.sh that does the job of calling the script server to addthis instance to the cluster and to be monitored.

#!/bin/bash# Find the IP address of this instancemyIP=`/usr/bin/findip`

# The public DNS name of the host that runs the server scriptlbHost=ec2-111-222-111-1.compute-1.amazonaws.com

# The port on which the server script is listeningserverPort=11511

# Where do we want to log the output of the execution of this scriptlogFile=/mnt/logs/readyToServe.log

# Start our Rails application mongrel cluster/etc/init.d/mongrel_cluster start

# Add Rails application mongrel clustercurl -s http://$lbHost:$serverPort/scale/application/up/$myIP/8500 >> $logFilecurl -s http://$lbHost:$serverPort/scale/application/up/$myIP/8501 >> $logFilecurl -s http://$lbHost:$serverPort/scale/application/up/$myIP/8502 >> $logFile

# Add this instance to be monitored by nagioscurl -s http://$lbHost:$serverPort/monitor/appserver/$myIP/add >> $logFile

Similarly when we shutdown the instance we can create a script takeMeOff.sh that willremove this instance from the load balance cluster and also remove to be monitored.

#!/bin/bash# Find the IP address of this instancemyIP=`/usr/bin/findip`

# The public DNS name of the host that runs the server scriptlbHost=ec2-111-222-111-1.compute-1.amazonaws.com

# The port on which the server script is listeningserverPort=11511

# Where do we want to log the output of the execution of this script

logFile=/mnt/logs/readyToServe.log

# Remove the Rails application mongrel cluster, pass only the IP# our server will remove all references to this IPcurl -s http://$lbHost:$serverPort/scale/application/down/$myIP >> $logFile

# Remove this instance from being monitored by nagioscurl -s http://$lbHost:$serverPort/monitor/appserver/$myIP/remove >> $logFile

3 Shell Script to Read IP Address - http://bash.cyberciti.biz/misc-shell/read-local-ip-address/

http://bash.cyberciti.biz/misc-shell/read-local-ip-address/




# Stop our Rails application mongrel cluster/etc/init.d/mongrel_cluster stop

Now that we have the scripts we need to be able to add these to be run when the instanceis started and when the instance is shutdown respectively. This can be done in severalways. The easiest approach would be to call the readyToServe.sh script in the /etc/rc.localfile and add the line in the stop function in /etc/init.d/xinetd script. These two scripts getexecuted during the instance start and shutdown and hence our script will be calledduring the boot and shutdown of the instance.

But a standard professional approach would be to add them as services to be executedduring boot and shutdown. There are several websites and manuals that explain aboutthis. As one reference you can read the article “Using chkconfig and /usr/sbin/service torun startup scripts in Redhat” 4. Depending on your linux distribution the steps,configurations and files could be different.

FinallyFinally after you are done with setting up everything on the instances don’t forget to

bundle up the instances and upload them to S3 and register them as AMI’s.

What Next?The approach and the scripts presented in this whitepaper are just first step to makingyour application in Amazon environment highly scalable. As you might have realized thissetup only focuses on scaling up the application running on a server. There are few

bottlenecks in this setup. The load balancer server and database server are single point of failures in this setup. Database clustering and load balancing is a separate topic of its own

and comes with its own complexity and problems. There are several techniques to solvethe problem of load balancer being single point of failure. One of them being that everyinstance in the cluster could also act as a load balancer and a dynamic DNS with roundrobin algorithm can be used to make any instance front end the requests and then load

balance the requests to other instances in the cluster. These are beyond the scope of thisdocument and are left to the user to explore further and also modify the setup provided inthis whitepaper accordingly.

4 Using chkconfig and /usr/sbin/service to run startup scripts in Redhat -http://www.netadmintools.com/art94.html

http://www.netadmintools.com/art94.html





Appendix A – References

Amazon EC2 Tools http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88

AWS SecurityWhitepaper

http://s3.amazonaws.com/aws_blog/AWS_Security_Whitepaper_2008_09.pdf

EC2 Tools Developer documentationWhitepaper

http://awsdocs.s3.amazonaws.com/EC2/latest/ec2-dg.pdf

Extend Elastic IPaddress limit

http://aws.amazon.com/contact-us/eip_limit_request/

NGINX Wiki http: //wiki.codemongers.com/Main Nagios Guide http://nagios.sourceforge.net/docs/nagios-3.pdf NRPE Guide http://nagios.sourceforge.net/docs/nrpe/NRPE.pdf Using chkconfig and/usr/sbin/service to runstartup scripts in Redhat


Shell Script to Read IPAddress


A Web Server in a ShellScript


Writing a web server inBourne Shell

http://sprocket.io/blog/2008/03/writing-a-web-server-in- bourne-shell/

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88%20






http://wiki.codemongers.com/Main



http://nagios.sourceforge.net/docs/nagios-3.pdf


http://nagios.sourceforge.net/docs/nrpe/NRPE.pdf





















dynamically scaling web applications in amazon ec2

Documents