the impact of soft resource allocation on n-tier application scalability

The Impact of Soft Resource Allocation on n-tier Application

Scalability

Qingyang Wang, Simon Malkowski, Yasuhiko Kanemasa, Deepal Jayasinghe, Pengcheng Xiong,

Motoyuki Kawaba, Lilian Harada, Calton Pu

2225th IEEE International Parallel & Distributed Processing Symposium

19 May 2011

Outline

Background & Motivation Background Motivational experiment

Performance Impact of Soft Resource Allocation Over-allocation of soft resources Under-allocation of soft resources Special case of under-allocation

Solution A practical algorithm for good soft resource allocation

Conclusion & Future Works


19 May 2011

Cloud Computing EnvironmentGood performance + Cost efficiency

Scaling applications on demand

BottleneckBottleneck

High throughput + low response time

High resource utilization


Soft resources in n-tier systems Threads, database connections, TCP

connections, locks, etc.

Soft Resource Allocation

19 May 2011

Bottleneck

Thread pool

Thread pool

Thread pool

Connection pool

Is it okay to duplicate the same configuration of soft resource allocation?

Thread pool


19 May 2011

Outline






RUBBoS benchmark Bulletin board system like

Slashdot (www.slashdot.org) Typical 3-tier or 4-tier architecture Two types of workload

Browsing only Read/Write mix

24 web interactions

C-JDBC Middleware for database scale-out

Read: act as a load-balancer Write: send a request to all

databases to keep consistency

Experimental Environment (1)

19 May 2011

http://www.slashdot.org/


Emulab (http://www.emulab.net) Relatively modest testbed originally for

network research Virtual network & physical machines (not VM)

Experimental Environment (2)

19 May 2011

Hardware

Specifications

Server type

PC3000 in Emulab

Processor Xeon 3GHz 64bitMemory 2GBNetwork 1GbpsDisk 2 x 146GB

10,000rpm


Software setupsExperimental Environment (3)

19 May 2011

Function SoftwareWeb server Apache 2.0.54Application server Apache Tomcat 5.5.17DB clustering middleware

C-JDBC 2.0.2

Database server MySQL 5.0.51aJava Sun jdk1.6.0_14Operating system Redhat FC4System Monitor Sysstat 7.0.2


NotationExperimental Environment (4)

19 May 2011

400-6-2001 / 3 / 1 / 2 configuration


19 May 2011

Result of Motivational Experiment

5000

5200

5400

5600

5800

6000

6200

6400

6600

6800

7000

7200

7400

7600

7800

600650700750800850900950

10001050

Workload [# Users]

Thro

ughp

ut [R

eqs/

s]

400-150-60

400-150-60

400-6-6

400-6-6

1/2/1/2

1/4/1/4

Scale out


Hardware configuration

2

19 May 2011

Challenge

Hardware configuration

Soft resource allocation

Hardware configuratio

n 1

Soft resource allocation 1Thread pool,

DB connection pool

Thread pool, DB connection pool

Soft resource allocation 2

Good performance

Bad performance

Good performance

Scale out

How to choose a reasonable soft resource allocation to match the hardware configuration?


Evaluate two important soft resources

Threads Database connections

Show their performance impact on n-tier applications

Over-allocation & under-allocation cases

Special case of under-allocation Introduce a practical way to choose a

reasonable allocation of soft resources

Focus of This Paper

19 May 2011


19 May 2011

Outline






19 May 2011

Performance Loss due to Soft Resource Over-allocation (1)

400-200-6

Sensitivity analysis: change DB Connection pool size in Tomcat


19 May 2011

50

6

100200

Performanc

e degradation

Goodput 1/4/1/4 CPU utilization in CJDBC

6

200

Throughput with response time boundary

Performance Loss due to Soft Resource Over-allocation (2)

High allocation of DB connections in Tomcat degrades the system performance


19 May 2011

JVM Garbage Collection Costs

Over-allocation of soft resources causes waste of critical hardware resources

200

6

8% more time used for GC over experimental time

JVM Garbage Collection in CJDBC


19 May 2011

Outline






19 May 2011

Performance Loss due to Soft Resource Under-allocation (1)

400-6-200

Sensitivity analysis: change thread pool size in Tomcat


6

200

19 May 2011

Performance Loss due to Soft Resource Under-allocation (2)

Goodput 1/2/1/2

20

10

CPU utilization of Tomcat

20

6

Under-allocation of soft resources causes inefficient utilization of hardware resources


19 May 2011

Outline






19 May 2011

Special Case of Soft Resource Under-allocation

30-6-100

Sensitivity analysis: change thread pool size in Apache


30-6-100

400-6-100

50-6-100100-6-100

19 May 2011

Performance Loss due to Under-allocation of Apache Threads

Goodput 1/4/1/4 CPU utilization in CJDBC

30-6-100

400-6-100

Low allocation of Apache threads degradesthe system performance


19 May 2011

Non-Trivial Correlation between Apache and Tomcat Threads (1)

30-6-100

30 > 6 * 4Why are 30 threads in Apache not enough?


19 May 2011


1: HTTP request

2

34: HTTP response

5: FIN reply Waiting for TCP FIN reply from the client


19 May 2011


1: HTTP request

2

34: HTTP response

5: FIN reply

Apache thread active period

Communicating with Tomcat

The long waiting time for FIN reply from clients is unpredictable and frequently happens under high workload

Waiting for TCP FIN reply from client


Concurrency for 30-6-100

Concurrency for 400-6-100

Large number of soft resources in front tier acts as a buffer providing stable workload for lower tiers

19 May 2011

Concurrency of Apache Threads

Connecting to Tomcat


19 May 2011

Summary of Experiments

1. Over-allocation of soft resources causes waste of critical hardware resources

2. Under-allocation of soft resources causes inefficient utilization of hardware resources

3. Large number of soft resources in front tier acts as a buffer providing stable workload for downstream tiers


19 May 2011

Outline






19 May 2011

Soft Resource Allocation Algorithm Key idea: allocating soft resources

globally to utilize the critical hardware resource as efficiently as possible


1. Identifying the critical hardware resource first

Soft Resource Allocation Algorithm (1)

19 May 2011

5000

5600

6200

6800

7400

600

750

900

1050

WorkloadTh

roug

hput

Throughput vs. Work-load(1)

Bottleneck server


5000

5600

6200

6800

7400

600

750

900

1050

WorkloadTh

roug

hput

Throughput vs. Work-load

1. Identifying the critical hardware resource first2. Allocating proper soft resources for the bottleneck

server


19 May 2011

Throughput knee

Minimum Saturation workload(2) Ave.

number of active threads

(1) Bottleneck server


(3) Dependency between current tier and bottleneck tier

1. Identifying the critical hardware resource first2. Allocating proper soft resources for the bottleneck

server3. Allocating proper amount soft resources in other

tiers


19 May 2011

(2) Ave. number of active threads

(1) Bottleneck server

RTTTPL *

)Re/(* ratioratiocjdbcapache qRTTLL

ratiocjdbcapache qTPTP Re/


19 May 2011

Outline





3434

Achieving good performance by scaling n-tier applications in Cloud requires a unified exploration of both hardware and software

Contributions: We showed allocation of soft resources has a big impact

on the total performance of an N-tier system We showed to decide a proper soft-resources allocation is

a complex problem, especially in cloud environments which requires dynamic scaling-out/in

We gave a practical algorithm for proper soft resource allocation


19 May 2011

Conclusion


19 May 2011

Future Work

Explore more soft resources such as locks, buffer/queue

Explore more efficient ways to find proper soft resource allocation

Explore the impact of soft resource allocation in virtualized environment

Thank You. Any Questions?

Qingyang [email protected]

the impact of soft resource allocation on n-tier application scalability

Documents