challenges in service-oriented networking · service-oriented networking literature review:...

Application-Aware NetworkingService-Oriented Networking

Literature Review: Internet Server DesignPerformance Analysis of Different Concurrency Mechanisms

Conclusions and Future Work

Challenges in Service-Oriented Networking

Bob Callaway

North Carolina State UniversityDepartment of Electrical and Computer Engineering

Ph.D Qualifying Examination

April 14, 2006

Advisory Committee: Dr. Michael Devetsikiotis, Dr. Yannis Viniotis,Dr. Harry Perros, Dr. Andy Rindos, Dr. Adolfo Rodriguez, Dr. Mihail Sichitiu

Bob Callaway Challenges in Service-Oriented Networking 1 of 38




Presentation Outline

1 Application-Aware NetworkingActive NetworksOverlay NetworksXML: A Standard for Data Interoperability

2 Service-Oriented NetworkingService-Oriented ArchitecturesNetwork Service Intermediaries

3 Literature Review: Internet Server DesignAdmission ControlConcurrency Mechanisms

4 Performance Analysis of Different Concurrency MechanismsExperimental DesignResults of Experiments

5 Conclusions and Future Work





Active NetworksOverlay NetworksLacking an Open Standard for Data InteroperabilityXML: A Standard for Data Interoperability

Section Outline











What is Application-Aware Networking?

Definition

Application-aware networking is an emerging technology thatpromises to provide increased end-to-end system performance fornext-generation applications and networks by providing differentialtreatment of traffic dependent on application data

Why do we need it?

Application performance is limiting factor in improvingend-to-end performance

Integration of heterogeneous distributed systems is difficultand costly

Complex algorithms are needed to secure enterpriseapplications






Active Networks

Overview

Attempted to add application layer functionality by executinguser-supplied bytecode in “smart” packets in specific activenodes (programmable routers, switches)

Suffers from issues of security, resource allocation,performance, and cost of deployment






Overlay Networks

Overview

Consist of peer nodes that self-organize into a distributed data structurebased on application criteria

Goals are to alleviate the effects of slow or sporadic deployment of newservices in the Internet, and to directly provide application-levelfunctionality that is out-of-scope for the underlying network






Lacking an Open Standard for Data Interoperability

Why is this an issue?

Active and overlay networks operate under the presumptionthat the underlying protocol is proprietary and specific to theapplication

Difficult to build a scalable and robust optimization strategywithin active or overlay networks for numerous proprietaryprotocols

Management of updates to these protocols is time consuming






The Extensible Markup Language (XML)

What is XML?

XML is an open standard for representing self-describingapplication data in a textual format, enabling heterogeneoussystems to easily operate on the data

2002 2003 2004 2005 2006 2007 20080

5

10

15

20

25

30

35

40

45

Year

Per

cent

age

of N

etw

ork

Tra

ffic

Estimated Percentage of XML in Network Traffic

<xml version="1.0">

<book>

<title>Green Eggs and Ham</title>

<author>Dr. Seuss</author>

</book>






XML as a Catalyst for Adding Intelligence in the Network

Why should we try to make the network fabric application-awareagain?

The underlying assumptions of active and overlay networkshave changed

Advances in hardware, software, networking technologiesThe network fabric can now support application-awareness

An open standard for application-to-applicationcommunication now exists and is widely adopted

XML enables services and SOA, and much more





Service-Oriented ArchitecturesNetwork Service Intermediaries

Section Outline











Service-Oriented Architectures

Issues in IT Architectures

Rapid Pace of Change

Moore’s Law exacerbates this

Integration of Heterogeneous Systems

Loosely Coupled Services

Represent a reusable business function

Removes dependencies on implementationspecifics through standardized interfaces

Standardized interfaces enable theflexibility of SOA






Web Services

What are Web Services?

XML-based standard forapplication communicationover the Internet

Based upon afind/publish/use approach

Many standards already existfor WS (e.g. WS-Security)






Enterprise Service Bus (ESB)

What is an ESB?

An ESB virtualizes the enterprise resources, allowing thebusiness logic of the enterprise to be developed and managedindependently of the infrastructure, network, and provision ofthose business services

The main functions of an ESB are to convert underlyingtransport protocols, transform message formats, andintelligently route requests made to services within an SOA






Service-Oriented Networking (SON)

What is SON?

SON enables application-awareness in the network,overcoming the previous limitations and constraints imposedby active and overlay networks, to improve performance andease the integration of heterogeneous systems withinservice-oriented environments

What benefits does SON provide?

Service Virtualization

Locality Exploitation

Increased Manageability






Network Service Intermediaries (NSI)

What is an NSI?

Network service intermediaries(NSI) are XML-enabledapplication-aware networkappliances which are configuredwith accelerated hardware foroptimized security and XMLoperations

Examples of NSI

DataPower XI50

Cisco AON Blade






The NSI as a Platform for ESB Functionality

ServiceProvider

Encrypted &Signed

SOAP/XML

Decrypted & AuthenticatedSOAP/XML

WS-Security:Decryption & Authentication

ServiceOriginator

SON Appliance

Functional Offloading

ServiceProviders

Unclassified Requests

XPathRouting

SON Appliance

Intelligent Routing

WidgetsRUSServiceProvider

Purchase Order in Widgets, Inc.XML Schema

Purchase Order in WidgetsRUSXML Schema

XSLTransformationWidgets, Inc.

ServiceOriginator

SON Appliance

Service Integration

What would types of functionswould an NSI perform?

Functional Offloading

Service Integration

Intelligent Routing






NSI - Architectural Design Goals

Architectural Design Goals

If the execution architecture is poorly implemented orarchitected, the NSI will become a bottleneck for theinfrastructure and will actually degrade the performance ofthe overall system

We define three goals for the NSI architecture: scalability,robustness, and adaptivity





Admission ControlConcurrency Mechanisms

Section Outline











Keys to Scalability, Robustness, and Adaptivity

Important Architectural Components

Admission Control Algorithms

Explicitly enumerate the process of admitting (and denying)requests according to specified criteria in order to ensure thatthe protected resource is never overloaded

Concurrency Mechanisms

Operating system structures designed to provide thevirtualization that multiple programs can be executedsimultaneously on a single processorManage how multiple programs can compete for the sharedprocessor






Admission Control

History of Admission Control

The initial surge in research on admission control techniquesbegan with the popularity of connection-oriented networkssuch as ATM networks

Admission control algorithms for the application layer beganto appear, due to the increasing popularity of mission-criticalapplications such as e-commerce are deployed on servers thatare subject to a dynamic and complex workload






Web Server Admission Control

Algorithms Utilizing Network Bandwidth

K. Li and S. Jamin: A measurement-based admission-controlled web server. in Proceedings of IEEE

INFOCOM 2000

Based upon concept of user-perceived bandwidth =�

bytes transferredresponse time

�

T. Voigt, et. al: Kernel mechanisms for service differentiation in overloaded web servers, in Proceedings of

the USENIX Annual Technical Conference, 2001.

Combines TCP-SYN policing and HTTP header analysis

Algorithms Utilizing Processor Utilization

L. Cherkasova and P. Phaal, Session-based admission control: A mechanism for peak load management of

commercial web sites, IEEE Transactions on Computers, 2002

Created static and adaptive MBAC algorithm based upon session length






Web Server Admission Control (continued)

Algorithms Utilizing Response Times

V. Kanodia and E. Knightly, Multi-class latency-bounded web services, in Proceedings of the 8th

International Workshop on Quality of Service, 2000.

Requests are classified into a service class based on latency target

M. Welsh and D. Culler, Adaptive overload control for busy internet servers, in Proceedings of the 4th

USENIX Conference on Internet Technologies and Systems, 2003.

Adapt token-bucket rate relative to 90th percentile of response times

Algorithms Utilizing Other Metrics

A. Robertsson, et. al, Admission control web server systems - design and experimental evaluation, in

Proceedings of 43rd IEEE Conference on Decision and Control, 2004.

Create non-linear admission control model for a G/G/1 queueing system

A. Verma and S. Ghosal, On admission control for profit maximization of networked service providers, in

WWW, 2003

Supports various objective functions in profit maximization






Concurrency Mechanisms

Threads vs. Events

Lauer/Needham argued in a 1978 paper that threads andevent-driven systems are duals of one another

Stated that the correct concurrency mechanism depends onthe problem and implementation constraints

Thread Per Request Event Driven






Thread-based Concurrency Mechanisms

Thread-Based LiteratureR. von Behren, et. al, Why events are a bad idea (for high-concurrency servers), in 9th Workshop on Hot

Topics in Operating Systems (HotOS IX), 2003.

Argued that poor performance of threads was due to poor implementation

R. von Behren, et. al, Capriccio: Scalable threads for internet services, in SOSP 2003.

Described a threading library which out performs event-driven systems

D. Xu and B. Bode, Performance study and dynamic optimization design for thread pool systems, in

Proceedings of CCCT, 2004.

Developed a dynamic optimization method to determine the optimal sizeof the thread pool

H. Jamjoom, et. al, The impact of concurrency gains on the analysis and control of multi-threaded

internet services, in Proceedings of IEEE Infocom 2004.

Developed a Markov-Chain based model of a multithreaded system






Event-Driven Concurrency Mechanisms

Event-Driven LiteratureJ. C. Hu, I. Pyarali, and D. C. Schmidt, Measuring the impact of event

dispatching and concurrency models on web server performance over high-speed

networks, in Proceedings of Global Internet Mini-Conference, IEEE GLOBECOM

1997.

Investigated the use of event-dispatching as an alternative tothreading-based concurrency methods

V. S. Pai, P. Druschel, and W. Zwaenepoel, Flash: An efficient and portable

Web server, in Proceedings of the USENIX 1999 Annual Technical Conference.

Compares the implementation of a multithreaded web serverwith an event-driven implementation; makes first attempt at ahybrid concurrency environment






Hybrid Concurrency Mechanisms

Hybrid Concurrency Literature

M. Welsh, et. al, SEDA: An architecture for well-conditioned, scalable internet services, in SOSP 2001

An architecture that separates functions within applications into stages,which each have their own thread pool

J. Suzuki and Y. Yamamoto, Openwebserver: an adaptive web server using software patterns, IEEE

Communications Magazine, 1999

Present an architecture that supports adaptively reconfiguring theconcurrency policy

M. Leclercq, et. al, DREAM: a component framework for the construction of resource-aware,

reconfigurable MOMs, in ACM Workshop on Adaptive and Reflective Middleware, 2004

Proposed that stages should be able to be adaptively combined or split,which would dynamically change the concurrency model

S. Michiels, et. al, Self-adapting concurrency: The DMONA architecture, in ACM WOSS 2002

Argues that the control should be an orthogonal extension to theframework, rather than tightly integrated





Experimental DesignResults of Experiments

Section Outline











Experimental Motivation

Goals

Create a prototype NSI and implement components thatwould likely be a part of an actual NSI

Gain insight on how the choice of a concurrency mechanismeffects overall system performance

Experiment Details

Comparing Thread-Per-Request, Bounded Thread Pool,Event-Driven, and SEDA Concurrency Mechanisms

Implemented in JCyclone, an open-source Javaimplementation of SEDA

Performance Metrics: Average Response Time, AverageThroughput, CPU Utilization






Experimental Design

Nonblocking Experiment

Blocking Experiment






Results - Nonblocking Experiments

0 1000 2000 3000 4000 5000 6000 7000 80000

0.5

1

1.5

2

2.5

3

3.5x 10

4 Average Response Time − Nonblocking Implementation

Number of Concurrent Users

Ave

rage

Res

pons

e T

ime

(ms)

TPRBTPEDSEDA

Average Response Time







0 1000 2000 3000 4000 5000 6000 7000 8000200

250

300

350Average Throughput − Nonblocking Implementation


Ave

rage

Thr

ough

put (

requ

ests

/sec

)

TPRBTPEDSEDA

Average Throughput







0 1000 2000 3000 4000 5000 6000 7000 800091

92

93

94

95

96

97

98Average CPU Utilization − Nonblocking Implementation


Ave

rage

CP

U U

tiliz

atio

n (%

bus

y)

TPRBTPEDSEDA

Average CPU Utilization






Results - Blocking Experiments

0 1000 2000 3000 4000 5000 6000 7000 80000

0.5

1

1.5

2

2.5x 10

5


Ave

rage

Res

pons

e T

ime

(ms)

Average Response Time − Blocking Implementation

TPRBTPEDSEDA−evenSEDA−weighted

Average Response Time







0 1000 2000 3000 4000 5000 6000 7000 80000

50

100

150

200

250

300

350


Ave

rage

Thr

ough

put (

requ

ests

/sec

)Average Throughput − Blocking Implementation


Average Throughput







0 1000 2000 3000 4000 5000 6000 7000 80000

10

20

30

40

50

60

70

80

90

100


Ave

rage

CP

U U

tiliz

atio

n (%

bus

y)Average CPU Utilization − Blocking Implementation


Average CPU Utilization





Section Outline










Conclusions

Conclusions

The adoption of XML and acceptance of Web Services and SOAhave enabled network components to make intelligent decisionsbased on application data

Our review of the Internet server design literature provides insightinto two large components of a scalable, robust, and adaptive NSI:admission control and concurrency mechanisms

Our experiments validated our intuition and emphasized theimportance of concurrency mechanisms in the overall performanceof an NSI

We believe that SON provides exciting new multidisciplinaryresearch opportunities in service-oriented computing, hardware,software, and networking which could have dramatic effects on thedevelopment of emerging network services





Future Work and Publications

Future Work

An adaptive and hybrid concurrency mechanism that usesonline measurements to determine the correct concurrencymechanism for each component and for the system as a whole

Admission control that balances the complex tradeoff betweensystem utilization and performance

Investigating how SON affects other architectures, such asgrid computing or P2P systems

PublicationsR. D. Callaway, A. Rodriguez, M. Devetsikiotis, and G. Cuomo.”Challenges in Service-Oriented Networking”, Submitted to 49th Annual IEEEGlobal Telecommunications Conference (GLOBECOM 2006).


challenges in service-oriented networking · service-oriented networking literature review:...

Documents