Auto-scaling Axis2 Web Services on

Amazon EC2

ByAfkham Azeez

2

Overview

• The Problem

• A Solution

• Some Concepts

• Design & Implementation Details

3

The Problem

• Fault tolerance, high availability & scalability are essential prerequisites for any enterprise application deployment

• One of the major concerns is avoiding single points of failure

• There is a high cost associated with achieving high availability & scalability.

4

Solutions

• Traditional solution

– Buying safety-net capacity

• Better solution

– Scale-up the system when the load increases

– Scale-down the system when the load decreases

– Should not have idling nodes

– Virtualization

– Pay only for the actual computing power & bandwidth utilized

5

Project Objective

• Building a framework which will auto-scale the number of Axis2 nodes on Amazon EC2, depending on the load

• This is a PoC Project

6

Amazon EC2

Amazon EC2 provides resizable compute capacity in the cloud

Provides a simple Web service interface & tooling that allows one to obtain and configure capacity with ease & to build failure resilient applications

7

Apache Axis2

Axis2 is a middleware platform which enables hosting of Web service applications and supports some of the major Web services standards

Can host Web services written in Java as well as various scripting languages

Can be deployed in a clustered configuration Uses Apache Tribes for clustering Axis2 clustering can be adopted to work on EC2

8

Apache Synapse

Apache Synapse is designed to be a simple, lightweight and high performance ESB

Supports load balancing with or without failover Supports static & dynamic load balancing Uses Apache Axis2

9

Apache Tribes

A messaging framework with group communication abilities

Allows you to send and receive messages over a network, it also allows for dynamic discovery of other nodes in the network.

Used by Apache Tomcat & Apache Axis2

10

Deploying a Service on the Cloud

AMI Instances

11

Deploying a Service on the Cloud

12

Deploying a Service on the Cloud

13

Deploying a Service on the Cloud

Service is available now

14

Auto-scaling

15

Auto-scaling

Load Increases

16

Auto-scaling

Startup new instances

17

Auto-scaling

New instances join group

18

Auto-scaling

Load Decreases

19

Auto-scaling

Terminate instances

20

Auto-scaling

Terminate instances

21

Deployment Architecture

22

Membership Aware Dynamic Load Balancing (1/2)

23

Membership Aware Dynamic Load Balancing (2/2)

24

Membership Schemes

Static Dynamic Hybrid (WKA based)

25

WKA Based Membership (1/3)

Application member joins. The load balancer is also a well-known member

26

WKA Based Membership (2/3)

A non-WK load balancer joins

27

WKA Based Membership (3/3)

A well-known load balancer rejoins after crashing

28

Membership Channel Architecture

29

Initialization Channel Architecture

30

Synapse Configuration - axis2.xml

31

Synapse Configuration - synapse.xml

32

Synapse Configuration

• AutoscaleInMediator

• AutoscaleOutMediator

• Autoscale Task

33

Synapse Configuration

AutoscaleInMediator

AutoscaleOutMediator

MessageID List LoadAnalyzerTask

SendMediator DynamicLoadbalanceEndpoint

SendMediator

EC2 Client

SynapseSynapse

From client

To client

To Axis2

From Axis2

To EC2 WS

Get length Invoke

IN

OUT

Normal Flow

34

Synapse Configuration

AutoscaleInMediator

AutoscaleOutMediator

MessageID ListLoadAnalyzer

Task

SendMediator DynamicLoadbalanceEndpoint

MakeFaultMediator

EC2 Client

SynapseSynapse

From client

Toclient

To Axis2

From Axis2

To EC2 WS

Get length Invoke

IN

ErrorHandlerSendMediator

Fault Flow

35

Synapse Autoscale Task

36

Axis2 Configuration - axis2.xml

37

Synapse Autoscale Task

• Sanity Check

• Autoscaling– Scale up– Scale down

38

Implementation Details• Single AMI – autoscalews

• Start– ec2-run-instances ami-a03fdbc9 -k autoscale -f payload.zip -g

autoscale-lb

• Payload

– Extract params from payload

– Env variables

• Axis2 & Synapse Configuration files & Repositories

– Maintained on S3

• Fault Tolerance

– Monitoring cron job

– Java Service Wrapper daemons

– Future: Use monit

– Future: Axis2/Synapse agent to check process status

39

Implementation Details

Start

Start

Start

StartInitial Instance

LB Group Axis2 App Group

synapse.xml

S3 Bucket

Load configuration

40

Implementation Details

Repo

Conf

Axis2 Instance

S3 Bucket

Load repo

Load configuration

41

TODO

RightScale comparison

Google App Engine comparison

Eucalyptus AppScale

Perf figures

42

Questions

Suggestions

Improvements

43

Thank You