sun n-tire

8/9/2019 Sun N-tire

1/56

Sun Microsystems, Inc.4150 Network Circle

Santa Clara, CA 95045 U.S.A.650 960-1300

http://www.sun.com/blueprints

Network Design Patterns:N-Tier Data Centers

Deepak Kakadia, Staff Engineer, Network Architect,Sun Microsystems

Richard Croucher, Chief Architect, Professional

Services, Sun MicrosystemsSun BluePrints™ OnLine—October 2003

Part No. 817-3997-10

Revision 1.0, 10/9/03Edition: October 2003

8/9/2019 Sun N-tire

2/56

Please

Recycle

Copyright2003 SunMicrosystems, Inc. 4150Network Circle, Santa Clara, California 95045 U.S.A.All rightsreserved.

SunMicrosystems, Inc. hasintellectual propertyrights relatingto technology embodied in theproduct that is describedin this document. Inparticular, andwithout limitation, these intellectual propertyrights mayinclude oneor more of theU.S. patents listedat http://www.sun.com/patents andone or more additionalpatents or pending patentapplicationsin theU.S. andin other countries.

This product or document is protectedby copyrightand distributed under licenses restricting itsuse, copying, distribution, anddecompilation.No part of this product or document maybe reproduced in anyform by anymeans without prior written authorization of Sunand itslicensors,if any. Third-party software, including font technology, is copyrighted and licensed fromSun suppliers.

Parts of theproduct maybe derived from Berkeley BSDsystems,licensedfrom theUniversity of California. UNIX is a registered trademark inthe United States and othercountries,exclusivelylicensed through X/OpenCompany, Ltd.

Sun, SunMicrosystems, theSun logo, SunBluePrints, SunONE, SunStorEdge,Sun Fire, SunDocs, Jump Start,iPlanet,Enterprise Java Beans, Java Database Connectivity, and Solaris are trademarks or registered trademarks of Sun Microsystems,Inc. in the United States and othercountries. AllSPARCtrademarks areused under license andaretrademarks or registered trademarks of SPARCInternational, Inc. in theUSandother countries. Products bearing SPARCtrademarks arebased upon an architecture developedby SunMicrosystems, Inc.

TheOPEN LOOKand Sun™ GraphicalUser Interface wasdeveloped by SunMicrosystems, Inc. forits users andlicensees. Sunacknowledgesthepioneering efforts of Xerox in researchingand developing theconcept of visualor graphicaluser interfaces forthe computer industry. Sunholds a non-exclusive license from Xerox to theXeroxGraphical User Interface, which license also coversSun’slicensees whoimplement OPENLOOKGUIs andotherwise complywith Sun’s written license agreements.

U.S. Government Rights—Commercial use.Governmentusersare subject to the Sun Microsystems, Inc. standard license agreementandapplicable provisions of theFar anditssupplements.

DOCUMENTATION IS PROVIDED "AS IS" AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE OR NON-INFRINGEMENT,ARE DISCLAIMED, EXCEPT TO THE EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.

Copyright 2003 Sun Microsystems, Inc.,4150 Network Circle, SantaClara,Californie 95045 Etats-Unis. Tous droits réservés.

SunMicrosystems, Inc. a lesdroits de propriété intellectuels relatants à la technologie incorporée dans le produit quiestdécrit dans cedocument. En particulier, et sans la limitation, cesdroits de propriété intellectuelspeuvent inclure un ou plus des brevets américains énumérésà http://www.sun.com/patents et un ou lesbrevets plus supplémentairesou lesapplicationsde brevet en attente dans les Etats-Unis et dansles autrespays.

Ce produit ou document estprotégé par un copyrightet distribuéavec deslicences quien restreignent l’utilisation, la copie, la distribution, et ladécompilation. Aucunepartiede ce produit ou document ne peut être reproduite sous aucuneforme, parquelque moyen quece soit, sans

l’autorisation préalable et écrite de Sunet de sesbailleursde licence, s’il y en a. Le logiciel détenupar des tiers,et quicomprend la technologierelative aux polices de caractères, estprotégé par un copyrightet licencié par desfournisseurs de Sun.

Desparties de ce produit pourrontêtre dérivées dessystèmesBerkeley BSDlicenciés parl’Université de Californie. UNIX estune marqueenregistree auxEtats-Unis et dans d’autres pays et licenciée exclusivement par X/OpenCompany Ltd.

Sun, SunMicrosystems, le logoSun, SunBluePrints, SunONE, SunStorEdge, SunFire,SunDocs,Jump Start,iPlanet, Enterprise Java Beans, Java Database Connectivity, et Solaris sont des marques de fabrique ou des marques déposées, ou marques de service, de Sun Microsystems,Inc. auxEtats-Unis et dans d’autres pays. Toutes lesmarques SPARCsont utiliséessous licence et sont des marques de fabrique ou des marquesdéposées de SPARCInternational, Inc. auxEtats-Unis et dans d’autres pays. Lesproduitsportant lesmarques SPARCsont basés surunearchitecture développée par Sun Microsystems, Inc.

L’interface d’utilisation graphiqueOPEN LOOK et Sun™ a étédéveloppée par SunMicrosystems,Inc. pour sesutilisateurset licenciés. Sunreconnaîtles efforts de pionniersde Xerox pour la recherche et le développement du concept des interfaces d’utilisation visuelle ou graphique

pour l’industrie de l’informatique.Sun détient unelicence nonexclusive de Xerox surl’interface d’utilisation graphique Xerox, cette licencecouvrant égalementles licenciésde Sunqui mettent en place l’interface d’utilisation graphique OPEN LOOKet quien outre se conforment auxlicences écrites de Sun.

LA DOCUMENTATION EST FOURNIE "EN L’ÉTAT" ET TOUTES AUTRES CONDITIONS, DECLARATIONS ET GARANTIES EXPRESSESOU TACITESSONT FORMELLEMENTEXCLUES, DANS LA MESUREAUTORISEE PAR LA LOIAPPLICABLE, Y COMPRIS NOTAMMENTTOUTE GARANTIE IMPLICITE RELATIVE A LA QUALITE MARCHANDE, A L’APTITUDE A UNE UTILISATION PARTICULIERE OU AL’ABSENCE DE CONTREFAÇON.

8/9/2019 Sun N-tire

3/56

1

Network Design Patterns: N-Tier

Data Centers

This article describes recommendations and best practices for designing andimplementing N-Tier architectures for web-based service delivery infrastructures. Itdocuments a set of fully tested configurations, comparing solutions using twodifferent network equipment providers.

The design and implementation of optimal N-Tier architectures requires a thoroughanalysis of functional and non-functional requirements, including performance,availability, security, scalability and flexibility. The degree to which non-functionalrequirements are met and exceeded often contributes to the quality factor of anarchitecture. In this paper, we present some proven design principles and concepts,which we hope are of value towards the successful delivery of N-Tier architecturesolutions.

For the purposes of this paper, we distinguish network “architecture” from “design”as follows:

Architecture is a high-level description of how the major components of the systeminterconnect with each other from a logical and physical perspective.

Design is more of a process that specifies , in sufficient detail for implementation,how to construct a network of interconnected nodes that meets or exceeds functionaland non-functional requirements (performance, availability, scalability, and so on).

There currently exists a sufficient level of available public information describing theconstruction of N-Tier data center architectures from a software standpoint.However, from a systems and network architecture perspective, the best way to goabout constructing these architectures might not be as clear. This article attempts tofill that void, by describing:

The recommended network design patterns and reasons behind these designs

Actual implementations that have been tested, tuned and proven in actualdeployments

8/9/2019 Sun N-tire

4/56

2 Network Design Patterns: N-Tier Data Centers • October 2003

This article also describes two popular approaches to network architectures, eachwith its own merits and limitations.

This article addresses the following topics:

“Defining Architectures” on page 2 “Implementation Overview” on page 10 “Designing the Network Configuration” on page 11 “Configuring the Physical Network” on page 35 “Network Security” on page 48

The focus of this article is limited to data flow. Systems and network management isa topic in itself and is not discussed in this paper. Readers should have sufficiently

detailed information to extend and modify this discussion to their particular needs.

Defining Architectures

This section breaks architecture considerations into two sections: “Logical Architectures” on page 2 “Physical Architectures” on page 6

Logical Architectures

A typical high-level functional overview of an N-Tier data center architecture isshown in FIGURE 1. The tiers are logically segregated by functionality as follows:

1. Web service tier—The set of load-balanced, front-end web servers that handles theinitial ingress client HTTP web requests. Each web server typically includes aservelet engine or JSP engine, and the capability to execute locally hosted cgi/binscripts which execute native code.

2. Application service tier—The set of application server instances that might span

one large multi-CPU server or several smaller multi-CPU servers. This tier providesgreater functionality and sophicated processing capabilities and services, such as EJBcontainers, an ORB, or connector modules that integrate with legacy servers. Theapplication server usually interfaces with the web server. In recent years, the webserver has been included in the application server text image for increasedperformance.

3. Naming service tier—The set of servers, possibly load balanced, that providessecondary infrastructure support services for managing and maintaining naming

information for hosts addressing data, configuration data, and so on. This caninclude services such as DNS and LDAP.

8/9/2019 Sun N-tire

5/56

Defining Architectures 3

4. Data service tier—The set of highly available servers that provides persistent datastorage services for critical corporate data. Due to the the sensitive nature of thedata, this tier is usually highly protected in terms of security and availability. We

briefly describe new virtual tiered firewalling techniques that provide increasedsecurity without sacrificing performance, using hardware-based firewalls.

FIGURE 1 Logical High-Level Overview of N-Tier Architecture

Note – Client access represents all the client systems that can access the referencearchitecture (client systems, web browsers, and so forth). This access is not

considered part of the N-Tier architecture

The actual reference implementation described leveraged Sun™ ONE technology.The Sun ONE software stack provides the software needed to deploy applicationsusing this reference implementation. The tiers of the N-Tier data center architecture,shown in FIGURE 1, are mapped to the Sun ONE software as follows:

Web service tier—Supported by Sun ONE Web Server, Enterprise Edition software

Naming service tier—Supported by Sun ONE Directory Server software Application service tier—Supported by Sun ONE Application Server software

Applica

tion ser

vice tier

Naming ser

vice tier

Web se

rvice tie

r

Client acc

ess

Refer

ence

archit

ecture

servi

ce tiers

LDAP

DNS

Data se

rvice tie

r

SAP

Oracle

8/9/2019 Sun N-tire

6/56


Data service tier—Provided by Oracle 9i database software, which is not part of the Sun ONE software stack

FIGURE 2 illustrates in greater detail how the logical N-Tier architecture is mapped tothe software systems architecture, and how Sun ONE and other softwarecomponents fit to create a complete system.

8/9/2019 Sun N-tire

7/56


FIGURE 2 Logical N-Tier Software Systems Architecture

F i r ew al l

F i r ew al l

F i r ew al l

JDBC

Connec

tions

Naming

Messag

ing

Mail

Security

JPDA

Administratio

n

and mo

nitoring

Servlet

contain

er

EJB

contain

er

HTTP

server

ORB

JDBC

Connec

tions

Naming

Messag

ing

Mail

Security

JPDA

Adminis

tration

and mo

nitoring

Servl

et

container

EJB

contain

er

HTTP

server

ORB

Web

server

Router

Internetor

Intranet

Oracle

Legacy

Sun ONE app

lication

server

instanc

es

Data

service

tier

Applica

tion

servi

ce

tier

Web

service

tier

LDAP

LDAP

DNS

DNS

Naming

service

tier

Identity

and policy

Platform

Service

contain

er

Web servi

cesApp

lication

s core

web ser

vices

Servic

e

delivery

Service

integrat

ionWe

b

server

Router Ser

vice cre

ation, a

ssembl

y and de

ployment

8/9/2019 Sun N-tire

8/56


Physical ArchitecturesThis logical architecture is then mapped to a physical realization as shown in

FIGURE 3, which describes the physical architecture. This mapping is influencedheavily at this point by non-functional requirements, such as performance andavailability. For example, redundant servers and redundant network connectivity area result of availability requirements and possibly performance requirements.Wirespeed multilayer switches might satisfy performance requirements, providedthat the switch performs packet processing within timing constraints. Most Layer 2and Layer 3 switches are implemented using ASICs, which are commonly available.Higher layer packet service implementations differentiate vendors from each other.

Some vendors, such as Foundry in the ServerIron product line, implement Layer 7using general purpose CPUs. This is much slower that an ASIC or an FPGAimplementation.

8/9/2019 Sun N-tire

9/56


FIGURE 3 Physical N-Tier Architecture Showing Hardware Components

Client 1 Client 2

L2-L3

Edge switch

Web

service

tier

Edge

network

Sun Fire

6800

Sun Fire

6800

Sun Fire

6800

Sun Fire

6800

Clientaccess

Applicationservice

tier

Data

service

tier

Naming

servicetier

Core switches

Sun Fire 280R Sun Fire 280R Sun Fire 280R Sun Fire 280R


T3

T3

8/9/2019 Sun N-tire

10/56


TABLE 1 through TABLE 3 describe the actual tested hardware components andconfigurations. This architecture is tested and proven. However, it is easilyextensible for adapting to specific customer requirements while keeping the

fundamental pattern intact. For example, increasing or decreasing the number of redundant servers of a particular tier directly impacts the degree of availability andperformance (if configured as Active Active), but does not violate the fundamentalarchitecture.

TABLE 1 Tested Systems Configuration: Sun Servers and Storage

Service Tier Function Equipment Features

Memory

(RAM)

Storage

Capacity

Operating

Enviro nmen t Software

Web Web servers (4) Sun Fire™ 280Rservers

Two 900-MHz CPUs

2 GB 36 GB Solaris 9 10/02with patches

Sun ONEWeb Server6.0, ServicePack 5,EnterpriseEdition

Naming DNS servers (2) Sun Fire 280R

servers

Two 900-

MHz CPUs

2 GB 36 GB Solaris 9 10/02

with patches

n/a

Directoryserver

(2) Sun Fire 280Rservers

Two 900-MHz CPUs


Sun ONEDirectoryServer 5.1

Application Applicationservers

(2) Sun Fire 6800servers

24 1-GHzCPUs


Sun ONEApplicationServer 7.0

Data Data servers (2) Sun Fire 6800servers

24 1-GHzCPUs


VERITASVolumeManager 3.5software,Oracle 9iRelease 2software

Databasestorage

(2) Sun StorEdge™T3 Arrays

n/a n/a 200 GB n/a n/a

8/9/2019 Sun N-tire

11/56


TABLE 2 Tested Network Configuration 1 - Extreme Networks Equipment

Function Equipment Features Operating Environment

Edge switch (1) Extreme NetworksSummit7i switch

24 1-Gb ports Extreme OS 6.2.2 build 18

Core switch (2) Extreme NetworksBlackDiamond 6800switches

Chassis switch with 7 blades. Each blade has 81-Gb ports (56 portstotal)

Extreme OS 6.2.2 build 18

TABLE 3 Tested Network Configuration 2 - Foundry Networks Equipment

Function Equipment Features Operating Environment

Edge switch (1) Extreme NetworksSummit7i switch

24 1-Gb ports Extreme OS 6.2.2 build 18

Core switch (2) Foundry BigIron4000 Layer 2/3

Server load- balancingswitch

(2) FoundryServerIron XL load-

balancing switches

8/9/2019 Sun N-tire

12/56


Implementation OverviewOnce the design and component selection is complete, follow these implementationsteps to set up your service tiers and their software:

To Implement Systems as Service Tiers

1. Install computer systems and storage hardware.

2. Establish network connectivity.

3. Configure partitions and domains on the Sun Fire 6800 servers in the applicationand data service tiers.

4. Install and configure the Solaris™ Operating System on all systems.

5. Configure the Sun StorEdge T3 arrays on the Sun Fire 6800 servers in the dataservice tier.

6. Install Oracle 9i Release 2 database software on the Sun Fire 6800 servers in thedata service tier.

7. Install the Sun ONE software on the following systems:

a. Web service tier—Install the Sun ONE Web Server software on the four SunFire 280R servers in this service tier.

b. Naming service tier—Configure DNS on two of the Sun Fire 280R servers.

c. Naming service tier—Install and configure the Sun ONE Directory Server 5.1software on the other two Sun Fire 280R servers.

Refer to the Sun ONE Directory Server 5.1 documentation collection (see “RelatedResources” on page 54 for more information).

d. Application service tier—Install and configure the Sun ONE Application Serversoftware on two Sun Fire 6800 servers.

Note – See http://docs.sun.com for installation and hardware guides.

8/9/2019 Sun N-tire

13/56

Designing the Network Configuration 11

Designing the Network ConfigurationOne can argue that the heart of any architecture composed of multiple nodes is thenetwork architecture. The network architecture is vital to assembling components or

building blocks in such a way that the system delivery is high performance, flexible,highly available, scalable, and secure. We describe a recommended methodology forcreating these architectures. We also describe actual implementations which showthe contrasts between two different network vendors, and how to optimally leverageeach of them. This section discusses the following topics:

“Logical Network Architecture” on page 11 “IP Services” on page 11 “Designing Networks for Availability” on page 19 “Networks and VLANs” on page 23 “Inter-Tier Traffic Patterns” on page 32

Logical Network ArchitectureThe logical network design is composed of segregated networks, implementedphysically using virtual local area networks (VLANs) defined by the networkswitches. The internal network uses private IP address spaces (for example,10.0.0.0) as specified in RFC 1918 for security and portability advantages. Lookahead to FIGURE 10 for a high-level overview of the logical network architecture.

The management network provides centralized data collection and management of all devices. Each device has a separate interface to the management network toavoid contaminating the production network in terms of security and performance.The management network is also used for automating the installation of thesoftware using Solaris JumpStart™ technology.

Although several networks physically reside on a single active core switch, networktraffic is segregated and secured using static routes, access control lists, and VLANs.From a practical perspective, this is as secure as separate individual switches.

IP ServicesThe following subsections provide a description of some of the more importantemerging IP services that are often important components of a complete networkdesign for a Sun ONE deployment. These IP services are divided into two categories:

8/9/2019 Sun N-tire

14/56

8/9/2019 Sun N-tire

15/56


FIGURE 4 IP Services Switch Functions on Incoming Packets

FIGURE 4 shows that a typical client request is destined for an external VIP, with IPaddress a.b.c.d and port 123. The SLB algorithm eventually intercepts the packet,and rewrites the destination IP address corresponding to the real server, which waschosen by a particular algorithm. The packet is then returned as indicated by thesource IP address.

Application Redirection - Class 2 Stateless

The Application Redirection function intercepts a client’s HTTP request andredirects the request to another destination, which is usually a group of cacheservers. Application Redirection rewrites the IP destination field. This is differentfrom proxy switching, where the socket connection is terminated and a newconnection to the server is created to fetch the requested web page.

Application Redirection serves the following purposes:

Reduces the load on one set of web servers and redirects it to another set, usuallycache servers for specific content

Intercepts client requests and redirects them to another destination for control of certain types of traffic based on filtered criteria

FIGURE 5 illustrates the functional model of Application Redirection, which onlyrewrites the IP header.

VIP1 = a.b.c.d:123 VIP2 = e.f.g.h:456

Real Dest. IP[1,2,3,...n]

srcIP:e.f.g.h srcPort: 456 dstIP:RealIP http://www.a.com/index.html

URLStringMatch

HTTPHeader Cookie Cache

SSLSession

ID

srcIP:a.b.c.d srcPort: 123 dstIP:VIP1 http://www.a.com/index.htmlPacket A

Packet A1 Packet A2

SLB function, finds the best server, and rewrites thedstIP of the target real server - Real IP

First Function rewrote srcIP and port, so that the real serverwill reply to this switch, which is at srcIP e.f.g.h and port 456.Dest is set to the SLB function

Client

srcIP:e.f.g.h srcPort: 456 dstIP:VIP2 http://www.a.com/index.html

SLB

CustomAlgorithm

RoundRobin

LeastConnections

8/9/2019 Sun N-tire

16/56


FIGURE 5 Application Redirection Functional Model

Content Switching - Class 1 Stateful

Content Switching is also known as Layer 7 processing, proxy switching, or URLswitching. This function accepts a client’s incoming HTTP request, terminates thesocket connection, and creates another socket connection to the target web server,which is chosen based on a user-defined rule. The Content Switching function thenfetches the requested web page and returns it to the client.

FIGURE 6 shows an overview of the functional Content Switching mode:

Application Redirection

Filter has a defined destination IP

Client request meets filter criteria,request is intercepted, IP dest isrewritten to new destination IP addrDEST = B

servergroup 1DEST = A

srcIP:a.b.c.d srcPort: 123 dstIP:VIP1 http://www.a.com/index.html

client http requestDEST = A

Client

srcIP:a.b.c.d srcPort: 123 dstIP:VIPA http://www.a.com/index.html

servergroup 2

DEST = BsrcIP:a.b.c.d srcPort: 123 dstIP:VIPB http://www.a.com/index.html

8/9/2019 Sun N-tire

17/56


FIGURE 6 Content Switching Functional Model

Content Switching with full Network Address Translation (NAT) serves thefollowing purposes:

Isolates internal IP addresses from being exposed to the public Internet.

Allows reuse of a single IP address. For example, clients can send their webrequests to www.a.com or www.b.com, where DNS maps both a.com and b.comdomains to a single IP address. The proxy switch receives this request, with thepacket containing an HTTP header in the payload that contains the target domain(a.com or b.com), and redirects this request to the appropriate group of servers.

Allows parallel fetching of different parts of web pages from servers optimizedand tuned for that type of data. For example, a complex web page may need GIFs,dynamic content, and cached content. With Content Switching, one set of web

servers can hold the GIFs, and another can hold the dynamic content. The proxyswitch can make parallel fetches and retrieve the entire page at a faster rate thanwould otherwise be possible.

Ensures requests with cookies or Secure Sockets Layer (SSL) session IDs areredirected to the same server to take advantage of persistence.

FIGURE 6 shows that the client’s socket connection is terminated by the proxyfunction. The proxy retrieves as much of the URL as is needed to make a decision

based on the retrieved URL. In this example, various URLs map to various server

client httprequest

Layer 7 Switching Function

VIP

http://www.a.com/SMA/stata/index.html servergroup1

http://www.a.com/SMA/dnsa/index.html servergroup2

http://www.a.com/SMB/statb/index.html servergroup3

http://www.a.com/SMB/CACHEB/index.html servergroup4

http://www.a.com/SMB/DYNA/index.html servergroup1

servergroup 1

stata

servergroup 2dnsa

servergroup 3statb

servergroup 4cacheb

servergroup 5dynab

- terminate socket connection

- get URL

- check against rules

- forward to servergroup/slb function- or get valid cookie, with server ID and forward it to the same server

8/9/2019 Sun N-tire

18/56


groups, which are virtual IP addresses. At this stage, the request may be forwardedto the server, or passed off to the SLB function that is waiting for traffic destined forthe server group.

The proxy is configured with a virtual IP, so the switch forwards all client requestsdestined for this virtual IP to the proxy function. The proxy function rewrites the IPheader, particularly the source IP and port, so that the server sends back therequested data to the proxy instead of directly to the client.

Network Address Translation - Class 1 Stateful

Network Address Translation (NAT) is a critical component for security and propertraffic direction. There are two basic types of NAT: half and full. Half NAT rewritesthe destination IP address and MAC address to a redirected location such as a webcache, which returns the packet directly to the client because the source IP address isunchanged. Full NAT is where the socket connection is terminated by a proxy, so thesource IP and MAC are changed to that of the proxy server.

NAT serves the following purposes:

Security—Prevents exposing internal private IP addresses to the public. IP Address Conservation—Requires only one valid exposed IP address to fetch

Internet traffic from internal networks with invalid IP addresses.

Redirection—Intercepts traffic destined for one set of servers and redirects it toanother by rewriting the destination IP and MAC addresses. The redirectedservers are able to send back the request directly to the clients with half NATtranslated traffic because the original source IP address has not been rewritten.

NAT is configured with a set of filters, usually a quintuple Layer 3 rule. If theincoming traffic matches a certain filter rule, the packet IP header is rewritten oranother socket connection is initiated to the target server, which itself can bechanged, depending on the rule.

Secure Sockets Layer (SSL) Session ID Persistence - Class 1Stateful

SSL can be implemented in a variety of ways: in software, hardware, or both. SSLcan be terminated at the target server, an intermediate server, an SSL networkappliance, or at an SSL-capable network switch. An SSL appliance, such as Netscaleror Array Networks, tends to be implemented with a PC board and have a PCI-basedcard containing the SSL accelerator ASIC. Hence the SSL acceleration isimplemented in libraries, which offload only the mathematical computations. Therest of the SSL processing is implemented in software, directing selective functions

to the hardware accelerator. Clearly, one immediate limitation is the PCI bus. Other

8/9/2019 Sun N-tire

19/56


newer SSL devices have an SSL accelerator integrated into the datapath of thenetwork switch. These are very advanced products, just starting to emerge fromstartups such as Wincom Systems.

FIGURE 7 shows an example of switch activity. Once a client has made initial contactwith a particular server, which may have been selected based on SLB, the switchensures that subsequent requests are forwarded to the same SSL server based on theSSL ID that the switch has stored during the initial SSL handshake. The switch keepsstate information based on the client’s initial request for HTTPS on port 443, whichcontains a hello message. This first request is then forwarded to the server selected

by the SLB algorithm or by another function. The server responds to the client’shello message with an SSL session ID. The switch then intercepts this SSL session

and stores it in a table. The switch forwards all of the client’s subsequent requests tothe same server, as long as each request contains the SSL session ID in the HTTPheader. There may be several different TCP socket connections that span the sameSSL session. State is maintained by the SSL Session ID in each HTTP request sent bythe same client.

FIGURE 7 Network Switch with Persistence Based on SSL Session ID

An appliance can be added for increased performance in terms of SSL handshakesand bulk encryption throughput. FIGURE 8 shows how an SSL appliance could bedeployed. Client requests first come in for a particular URL using the HTTPSprotocol on port 443. The switch recognizes that this needs to be directed to theappliance, which is configured to provide that SSL service. A typical appliance, suchas Netscaler, can also be configured to provide Content Switching and LoadBalancing in addition to SSL acceleration. The appliance then reads or insertscookies and resubmits the HTTP request to an appropriate server, which maintainsstate based on the cookie in the HTTP header.

SSL Server1Client

SSL tunnel - SSL session IDHTTP Session 1

HTTP Session 2

HTTP Session 3

HTTP Session 4

Network

Switch

8/9/2019 Sun N-tire

20/56


FIGURE 8 Tested SSL Accelerator Configuration

Cookie Persistence - Class 1 StatefulThe HTTP/1.0 protocol was originally designed to provide static pages in onetransaction. As more complex web sites evolved, which required multiple HTTP getson the same server, it was later discovered that performance was severely limited bythe tearing down and opening of TCP socket connections. This problem was solved

by HTTP 1.1, which allowed persistent connections. Immediately after a socketconnection, the client could pipeline multiple requests. However, as more complex

web sites evolved for applications such as the shopping cart, persistence acrossmultiple HTTP 1.1 requests was required. The problem was further complicated byproxies and load balancers that interfered with the traffic being redirected to thesame web server. Another mechanism was required to maintain state across multipleHTTP 1.1 requests. The solution was the introduction of two new headers in theHTTP request, Set-Cookie and Cookies, as defined in RFC 2109. These headerscarried the state information between the client and server. Typically, most load-

balancing switches have enough intelligence to ensure that a particular client’ssession with a particular server is maintained, based on the cookie that is inserted bythe server and maintained by the client.

Client

Internet

SSL http

http

MultilayerSwitch

Session PersistenceBased on SSL Session ID

Session PersistenceBased on Cookie

Sun ONEWeb Servers

SSL Accelerator ApplianceKey Exchange and Bulk Encryption

https

8/9/2019 Sun N-tire

21/56


FIGURE 9 Network Availability Strategies

Designing Networks for Availability

FIGURE 9 shows a cross-sectional view of the tier types and the functions performedat each tier. It also shows the availability strategies for the network and web tier.External tier availability strategies are outside the scope of this article. Ourdiscussion is limited to the services tiers: Web, Application Services, Naming, and soon.

Designing network architectures for optimal availability requires maximizing twoorthogonal components:

Intra Availability—Refers to maximizing the function that considers theestimated failure probability of the components themselves. The components thatcause the failure are only considered by the following equation:

F Availability = MTBF ÷ (MTBF + MTTR)

Where MTBF is Mean Time Between Failures and MTTR is Mean Time toRecovery.

Client Tier External NetworkConnectivity

Network Tier Services Tier

Layer 3 redundancy

session based services

requires session sharing.

Other stateless servicesfailover with no problem.

Sun Servers with IPMP-

dual NICs

8/9/2019 Sun N-tire

22/56


Inter Availability—Refers to minimizing the impact of failures caused by factorsexternal to the system by the surrounding environment, such as single points of failure (SPOFs), power outages, or a technician accidently pulling out a cable.

It is not sufficient to simply maximize the F Availabili ty function. The SPOF andenvironmental factors also need to be considered. The networks designed in thisarticle describe a highly available architecture which conforms to these designprinciples.

FIGURE 10 provides an overview of the logical network architecture. This shows howthe tiers map to the different networks, which are also mapped to segregate VLANs.This segregation allows inter-tier traffic to be controlled by filters on the switch, orpossibly a firewall (which is the only bridge point between VLANs).

8/9/2019 Sun N-tire

23/56


FIGURE 10 Logical Network Architecture Overview

The following list describes each subnetwork shown in FIGURE 10:

External network—The external-facing network that directly connects to theinternet. All IP addresses must be registered, and it is advisable to secure themwith a firewall.

Externalnetwork

192.168.10.0/24

Web servicenetwork

10.10.0.0/24

Namingservices network

10.20.0.0/24

Loadbalancer

Loadbalancer

Managementnetwork

10.100.0.0/24

Accessto all

networksDatabase

service network10.100.0.0/24

Backupnetwork

10.110.0.0/24

Devicenetwork

10.0.0.0/24

Applicationservices network

10.30.0.0/24

Clientnetwork

172.16.0.0/24

Productionnetwork

Externalnetwork

Managementnetwork

8/9/2019 Sun N-tire

24/56


The following networks are assigned non-routable IP addresses, as specified byRFC 1918, which can also be based on the following:

10.0.0.0 - 10.255.255.255 (10/8 prefix)

172.16.0.0 - 172.31.255.255 (172.16/12 prefix) 192.168.0.0 - 192.168.255.255 (192.168/16 prefix)

Web Services network—A dedicated network containing web servers. Typicalconfigurations include a load-balancing switch. This switch can be configured toallow the web server to return the client’s HTTP request directly, or require theload-balancing device to return the request on behalf of the provider’s web server.

Naming Services network—A dedicated network consisting of servers thatprovide LDAP, DNS, NIS, and other naming services. The services are for internal

use only, and should be highly secure. Internal infrastructure support servicesmust be sure that requests originate and are destined for internal servers. Mostrequests tend to be read intensive, hence the potential use of caching strategies forincreased performance.

Management network—A dedicated service network that provides managementand configuration for all servers, including the JumpStart installation option fornew systems.

Backup network—A dedicated service network that provides backup and restoreoperations. This network is pivotal to minimizing disturbances to otherproduction service networks during backup and other network bandwidth-intensive operations.

Device network—A dedicated network that attaches IP devices for storage, aswell as other devices.

Application Services network—A dedicated network, typically consisting of larger multi-CPU servers that host multiple instances of the Sun ONE Application

Server software image. These requests tend to only require low network bandwidth, but can span multiple protocols, including HTTP, CORBA,proprietary TCP, and UDP-based protocols. The network traffic can also besignificant when Sun ONE application server clustering is enabled. Every updateto a stateful session bean triggers a multicast update to all servers on thisdedicated network so that participating cluster nodes update the appropriatestateful session bean. Network utilization increases directly proportional to theintensity of session bean updates.

Database network—A dedicated network, typically consisting of one or twomulti-CPU database servers. Network traffic typically consists of jdbc traffic

between the application server and the web server.

8/9/2019 Sun N-tire

25/56


Networks and VLANsEach service is deployed in a dedicated Class C network where the first three octets

represent the network number. The design represents an innovative approach whereseparate Layer 2 devices are not required because the functionality was collapsedinto the core switch. Decreasing the management and configuration of separatedevices while maintaining the same functionality is a major step toward cuttingcosts and increasing reliability.

FIGURE 11 shows how a traditional configuration requires two Layer 2 switches. Aparticular VLAN spans the six segments, which gives each interface access to theVLAN on failover.

FIGURE 11 Traditional Availability Network Design Using Separate Layer 2 Switches

The design shown in FIGURE 12 results in the same network functionality, buteliminates the need for two Layer 2 devices. This is accomplished using a taggedVLAN interconnect between the two core switches. Collapsing the Layer 2functionality reduces the number of network devices. This provides a reduced riskof unit failure, lower cost, and reduced manageability issues.

Edge switch

Master

switch

(layer 3)

Standby

switch

(layer 3)

Layer 2

switch

Layer 2

switch

Client network

Network

interface 0

Network

interface 1

Sun server

8/9/2019 Sun N-tire

26/56


FIGURE 12 Availability Network Design Using Large Chassis-Based Switches

Solaris IPMP Host Network Interface RedundancyIntroduction

Solaris 8 introduced a novel technology that added network load balancing and path

failover with IP Multi Path (IPMP) built into the base operating system. IPMP isused to provide failover between multiple physical network interfaces. Itsadvantages over NAFO and other network interface redundancy approaches aresupport for outbound load balancing, and a single mechanism to support bothclustered and non-clustered nodes. Failover is typically fast enough to preserveapplication session connectivity. The default failover time is ten seconds. NormalTCP/IP recovery ensures no loss of data. Interfaces are collected into groups. Theminimum number of interfaces assigned to a group is two. Where larger domains

are built, the interface group can scale to the total number of installed networkinterface cards (NICs) that are attached to the same VLAN.

Client network

Edgeswitch

Standby core switch 2Master core switch 1

Networkinterface 0

Network

interface 1

Sun server

8/9/2019 Sun N-tire

27/56


Note – Currently, configuration of Quad Fast Ethernet cards is limited to eightcards, and configuration of Gigabit Ethernet cards is limited to six cards. AdditionalI/0 cards can be added by blacklisting and adding as needed. Please consult a Sun

Systems Engineer for the latest configuration rules.

Solaris IPMP Host Network Interface Redundancy Internals

FIGURE 13 describes the internal components of Solaris IPMP. The key is the GroupIdentifier (GID) associated with each physical MAC. IP forwards packets destined

for a particular GID, as opposed to a physical interface.

8/9/2019 Sun N-tire

28/56


FIGURE 13 IPMP Internals

Further, IP has the ability to loadbalance across all physical interfaces that have acommon GID. As can be seen, each physical interface has a unique MAC and a

unique test IP address (TIP). The Virtual IP address associated with each physicalinterface that belongs to the same GID has two modes of operation:

VIP

TIP

GID

MAC

PHY

10.6.97.

1

TCP

Stream head

10.6.

97.53

ha

8:0:20:f

f:9b:b1

ce0

VIP

TIP

GID

MAC

PHY

10.6.97.

1

10.6.97.

54

ha

8:0:20:f

f:9b:b2

ce1

Appli

cation

in.mpath

d

P o r t 1

P o r t 2

P o r t 3

P o r t 2 3

P o

r t 1

P o r t 2

P o r t 3

P o r t 2 3

L a y e r 3

F D B

N e t w

o r k

P o r t

L a y e r 2

F D B

M A C

A D D

R P o

r t

P o r t 1

P o r t 2

P o r t 3

P o r t 2 3

P o r t 1

P o r t 2

P o r t 3

P o r t 2 3

L a y e r 3

F D B

N e

t w o r k

P o r t

L a y e r 2

F D B

M A C

A D D

R P o

r t

Sun server

Network switch

Network switch

8/9/2019 Sun N-tire

29/56


1. Active Active —The VIP must be specified using the addif command for eachactive interface. Only active interfaces are load balanced for outgoing packets.

2. Active Standby—The VIP is not specified on a particular interface, and the

standby command is issued for the standby interface. In this case, the standbyinterface does not participate in outbound load balancing. It associates with the VIPonly in the event that an active interface fails and remains down.

It is important to configure the network switch carefully when using IPMP. Asshown in FIGURE 13, there are two types of forwarding tables: Layer 3 for IP-basedforwarding, and Layer 2 forwarding. It is important that the MAC addresses aredifferent: the switch must have a unique port associated with each MAC address.Otherwise the switch may get confused, depending on the network forwardingimplementation. Also, if a standby interface has just failed over, the VIP will beassociated with the old, failed MAC and the switch may incorrectly forward to thefailed NIC. It is important to use tested network switches that know, when a linkfails, to remove the appropriate entries from the forwarding table are re-ARPed foran updated entry.

IPMP detects failures using two methods:

Link level—The NIC knows when the link is physically down. The driver issues a

message to IP to perform failure detection operations based on the configurations. IP level—The daemon in.mpathd detects Layer 3 failures with the default router.

Periodically in.mpathd tests the connectivity between the TIP and the defaultrouter, ensuring correct Layer 3 connectivity. In the event of a failure, a message issent to IP to to perform failure detection operations, based on the configurations.

Solaris IPMP Host Network Interface RedundancyAvailability

A typical highly available configuration includes a Sun server that has dual NICcards, which increases the availability of these components by several orders of magnitude. For example, the Gigabit Ethernet card, part number X2069A, by itself has an MTBF of 199,156 hours, and assuming approximately 2 hours MTTR, has anavailability of 0.999989958. With two cards, the MTBF increases so that availability

becomes 9 9’s at .9999999996 availability. This small incremental cost has a bigimpact on the overall availability computation.

FIGURE 14 shows the Sun server redundant NIC model using IPMP in Active Standbymode. The server has two NICs, ge0 and ge1, with fixed IP addresses a.b.c.d ande.f.g.h. The virtual IP address w.x.y.z is the IP address of the service. Clientrequests use this IP address as the destination. This IP address floats between thetwo interfaces: ge0 or ge1. Only one interface can be associated with the virtual IPaddress at any one instant. If the ge0 interface owns the virtual IP address, then data

8/9/2019 Sun N-tire

30/56


traffic follows the P1 path. If the ge0 interface fails, the ge1 interface takes over andassociates the virtual IP address, and then data traffic follows the P2 path. Failurescan be detected within two seconds, depending on the configuration.

.

FIGURE 14 High-Availability Network Interface Cards on Sun Servers in Active Standby

Layer 3—Integrated Virtual Router Redundancy Protocol(VRRP) RFC 2338 and IPMP

By combining the availability technologies of routers and server NICs, we can createa reusable cell that can be reused in any deployment where servers are connected to

routers. This reusable cell is highly available and scalable. In most LANenvironments, there are many servers or client nodes, with one default router. If thatdefault router fails, or the link that connects the hosts to the default router fails, thenconnectivity to the outside network is down. To solve this problem, Virtual RouterRedundancy Protocol (VRRP), as defined in RFC 2338, was introduced. VRRP is

basically a multicast protocol used by switches to monitor one another ’s health.When a backup switch detects a failure of the master switch, it assumes the IPaddress of the master. This is the same IP address used by all servers of a particular

VLAN as the default route. To integrate IPMP and VRRP so that both redundancyprotocols interoperate properly, care must be taken to ensure correct operating andcorrective operations in the event of a failure. FIGURE 15 describes a workingconfiguration. Lines 1 and 2 show the VRRP protocol used by the routers to monitorone another. If one router detects that the other has failed, the surviving routerassumes the role of master and inherits the IP address and MAC address of themaster.

Lines 3 and 5 in FIGURE 15 show how a switch can verify that a particular connection

is up and running. This connection can be port-based, link-based, or based on Layers3, 4, and 7. The router can make synthetic requests to the server and verify that a

HCS HCS

VLAN

ge0:a.b.c.d

w.x.y.z

ge1:e.f.g.h

P1

P2

IPMP-dual NIC

8/9/2019 Sun N-tire

31/56


particular service is up and running. If the router detects that the service has failed,the VRRP can be configured on some switches to consider this failure as part of theelection algorithm and tie the failure to the priority of the VRRP router.Simultaneously, the server is also monitoring links. Currently, IPMP consists of a

daemon, in.mpathd, that constantly pings the default router. As long as the defaultrouter can be pinged, the master interface (ge0) assumes ownership of the IPaddress. If the in.mpathd daemon detects that the default router is not reachable,automatic failover occurs, which brings down the link and floats over the IP addressof the server to the surviving interface (ge1).

In the lab, we were able to tune IPMP and VRRP to achieve failure detection andrecovery within one second. There is a trade-off, however. Because the control

packets are on the same network as the production network, and because VRRP is aCPU-intensive task, false failures are possible if the switches, networks, or servers become overloaded. This is because the device might take longer than the stricttimeout to respond to the peer’s heartbeat.

FIGURE 15 Design Pattern for IPMP and VRRP Integrated Availability Solution

N-Tier Data Center Logical Network Design

The logical network design for the N-Tier data center (FIGURE 16) incorporates serverredundant network interfaces and integrated VRRP and IPMP.

1

4

3 65

2VRRP

VRRP

ge0

ge1

in.mpathd

8/9/2019 Sun N-tire

32/56


FIGURE 16 Logical Network Architecture With Virtual Routers, VLANs, and Networks

172.16.0.1

Clients

Edge switch

192.168.0.1

Master

Servers

Slave

192.168.0.2

10.10.0.110.50.0.1

10.20.0.110.40.0.1

10.30.0.1

192.168.0.2

10.10.0.1 10.50.0.1

10.20.0.1 10.40.0.1

10.30.0.1

8/9/2019 Sun N-tire

33/56


TABLE 4 summarizes the eight separate networks and associated VLANs.

The edge network connects redundantly to the internal reference architecturenetwork. One of the core switches has ownership of the 192.16.0.2 IP address,which means that switch is the master and the other is in slave mode. When theswitch is in slave mode, it does not respond to any traffic, including ARPs. Themaster also assumes ownership of the MAC that floats along with the virtual IPaddress of 192.16.0.2.

Note – If you have multiple NICs, make sure each NIC uses its unique MACaddress.

Each switch is configured with the identical networks and associated VLANS thatare shown in TABLE 4. An interconnect between the switches extends each VLAN, butis tagged to allow multiple VLANs to share a physical link. (This requires a networkinterface, such as cards that use the ce driver, that supports tagged VLANS.) TheSun servers connect to both switches in the appropriate slots, where only one of the

two interfaces will be active.Most switches support Routing Information Protocol (RIP and RIPv2), Open ShortestPath First (OSPF), and Border Gateway Protocol v4 (BGP4). However, static routesprovide a more secure environment. A redundancy protocol based on Virtual RouterRedundancy Protocol (VRRP, RFC 2338) runs between the virtual routers. The MACaddress of the virtual routers floats among the active virtual routers, so that the ARPcaches on the servers do not need any updates when a failover occurs.

TABLE 4 Network and VLAN Design

Name Network Default Router VLAN Purpose

client 172.16.0.0 172.16.0.1 client Client load generation

edge 192.16.0.0 192.16.0.1 edge Connects client network to internalreference architecture network

web 10.10.0.0 10.10.0.1 web Web services

ds 10.20.0.0 10.20.0.1 ds Directory services

db 10.30.0.0 10.30.0.1 db Database services

app 10.40.0.0 10.40.0.1 app Application services

dns 10.50.0.0 10.50.0.1 dns DNS services

mgt 10.100.0.0 10.100.0.1 mgt Management and administration

san 10.0.0.0 10.0.0.1 san Devices network

8/9/2019 Sun N-tire

34/56

8/9/2019 Sun N-tire

35/56


FIGURE 17 Logical Network Describing Application Data Flow

1 12

2 11

3 5

4

8

9

10

7

6

Clients

Switching

services

Load

balancer

Static contentNFS server

Web services

Directory

services

Application

services

Database

services

8/9/2019 Sun N-tire

36/56


TABLE 5 Sequence of Events for FIGURE 17

Item Interface1 Interface2 Protocol Description

1 Client Switch HTTP/HTTPS

Client initiates web request. Clientcommunication can be HTTP or HTTPS(HTTP with secure socket layer). HTTPScan be terminated at the switch or at theweb server.

2 Switch Web server HTTP/

HTTPS

Switch redirects client request to

appropriate web server.3 Web Server Application

serverApplicationserver webconnectorover TCP

The web server redirects the request tothe application server for processing.Communication passes through a webserver plug-in over a proprietary TCP-

based protocol.

4 Applicationserver

Directoryserver

LDAP The Java 2 Enterprise Edition (J2EE)application hosted by the application

server identifies the requested process asrequiring specific authorization. It sendsa request to the directory server to verifythat the user has valid authorization.

5 Directoryserver

Applicationserver

LDAP The directory server successfully verifiesthe authorization through the user’sLDAP role. The validated response isreturned to the application server.Application server then processes

business logic represented in J2EEapplication.

6 Applicationserver

Databaseserver

JDBC The business logic requests data from adatabase as input for processing. Therequests may come from servlets, JavaData Access Objects, or Enterprise

JavaBeans™ (EJBs) that in turn use JavaDataBase Connectivity (JDBC™) to access

the database.7 Database

serverApplicationserver

JDBC The JDBC request can contain any validSQL statement. The database processesthe request natively and returns theappropriate result through JDBC to theapplication server.

TABLE 5 Sequence of Events for FIGURE 17 (Continued)

8/9/2019 Sun N-tire

37/56

Configuring the Physical Network 35

Configuring the Physical NetworkThe next step involves implementing a real network based on the logical network

design. This process involves: “Physical Network Connectivity” on page 38 “Configuring Switches” on page 40

There are several approaches to realize a network that functionally satisfies thelogical architectural requirements.

The Sun ONE N-Tier Data Center is vendor independent, so you can use the

network equipment that best suits your environment. In the lab, we built twodifferent network configurations. One configuration uses Extreme Networks BlackDiamond 6800 Core Switches, (FIGURE 18), and the other uses Foundry Networks BigIron Core Switch and Server Iron XL Load Balancer (FIGURE 19). The ExtremeNetworks switch that we used has built-in load balancing, so there is no need for anexternal load-balancing device. The Foundry Networks products required a separateload-balancing switch.

Similar implementations could be realized using products from Cisco or Nortel.

8 Application

server

Web server Application

server Webconnectorover TCP

The J2EE application completes the

business logic processing, packages thedata for display, usually through a JSPthat renders HTML, and returns theresponse to the web server.

9 Web server Switch HTTP/HTTPS

Switch receives reply from web server.

10 Switch Client HTTP/

HTTPS

Switch rewrites IP header and returnsrequest to client.

TABLE 5 Sequence of Events for FIGURE 17 (Continued)

Item Interface1 Interface2 Protocol Description

8/9/2019 Sun N-tire

38/56


FIGURE 18 Sun ONE N-Tier Network Configuration With Extreme Networks Equipment

Client 1 Client 2

L2-L3edge switch

Webservice

tier

Sun Fire6800

Sun Fire680010.30.0.101

Sun Fire6800

Sun Fire6800

10.30.0.100

Client

access

Extreme

switches

Application

service

tier

Data

service

tier

Naming

servicetier

Extreme switch192.168.10.2



T3

T3

192.168.10.1

10.10.0.1

10.20.0.110.40.0.110.30.0.1

10.50.0.1

10.10.0.100 10.10.0.101 10.10.0.102 10.10.0.103

10.20.0.100 10.20.0.101 10.20.0.102 10.20.0.103

10.40.0.100 10.40.0.101

Extreme switch192.168.10.3Core Core

8/9/2019 Sun N-tire

39/56


FIGURE 19 Sun ONE N-Tier Network Configuration With Foundry Networks Equipment

Client 1 Client 2

Level 2-3 edge switch

Web

service

tier

Sun Fire6800

Sun Fire680010.30.0.101

Sun Fire6800

Sun Fire6800

10.30.0.100

Client

access

Applicationservice

tier

Dataservice

tier

Namingservice

tier

Master core192.168.10.2



T3

T3

192.168.10.1

10.10.0.1

10.20.0.110.40.0.110.30.0.1

10.50.0.1

10.10.0.100 10.10.0.101 10.10.0.102 10.10.0.103

10.20.0.100 10.20.0.101 10.20.0.102 10.20.0.103

10.40.0.100 10.40.0.101

Standby core192.168.10.3

Server load-balancer switches

Foundry

switches

Physical Network Connectivity

8/9/2019 Sun N-tire

40/56


Physical Network ConnectivityFIGURE 20 is a standardized diagram showing the IP addresses for both the ExtremeNetworks and Foundry Networks configurations.

The Extreme Networks configuration would be mapped directly to mls1 and mls2, because these switches incorporate SLB functionality. Foundry, however, requiresadditional load-balancing appliances.

FIGURE 20 elements are described further in TABLE 6.

8/9/2019 Sun N-tire

41/56


FIGURE 20 Physical Network Connections and Addressing

Client1Client2

Edge

mls1 mls2

ge0:172.16.0.101/24ge0:172.16.0.102/24

172.16.0.1/24

192.168.0.1/24

192.168.0.2/24 192.168.0.2/24

192.168.0.101/24 192.168.0.102/24

1 2 3 4

5 67 hme0:10.100.16.101hme0:10.100.16.10210.100.16.1

10.100.168.2

hme010.100.10.101

10.100.168.2

hme0:10.100.10.105

hme0:10.100.20.101 hme0:10.100.20.103

192.168.0.2/24 192.168.0.2/24

hme0:10.100.30.101 hme0:10.100.30.103

ge0:10.10.0.101/24

ge1:10.10.0.102/24

ge0:10.10.0.103/24

ge1: 10.10.0.104/24

ge0:10.10.0.105/24

ge1:10.10.0.106/26

ge0:10.10.0.107/24

ge1:10.10.0.108/26

app1

app2

app1

app2

10.10.0.1/24 10.10.0.1/24

ge0:10.40.0.101/24 ge1:10.40.0.102/24

ge0:10.40.0.103/24 ge1:10.40.0.104/24

ge0:10.40.0.105/24 ge1:10.40.0.106/24

ge0:10.10.0.107/24 ge1:10.40.0.108/24

web1 web1 web1 web1

ds1 ds2

10.20.0.1/24 10.20.0.1/24

ge0:10.20.0.101/24

ge1:10.20.0.102/24 ge1:10.20.0.104/24

ge0:10.20.0.103/24

10.30.0.1/24 10.30.0.1/24

ge1:10.30.0.104/24

db1 db2

ge0:10.30.0.101/24

ge1:10.30.0.102/24

ge0:10.30.0.103/24

8/9/2019 Sun N-tire

42/56


Configuring SwitchesA high-level overview of the switch configuration is shown in FIGURE 21.

TABLE 6 Physical Network Connections and Addressing

Switch Description Port Speed (Mb/s)

Base

Address Netmask

edge Client network to externalnetwork router

1,2,3,4 1000 172.16.0.1 255.255.255.0

edge External network - mls1 5,6 1000 192.168.10.1 255.255.255.0

mls1 External network 1 1000 192.168.10.2 255.255.255.0mls1 Web/application service

router3,4,5,6 1000 10.10.0.1 255.255.255.0

mls1 Directory service router 7,8 1000 10.20.0.1 255.255.255.0

mls1 Database service router 9,10 1000 10.30.0.1 255.255.255.0

mls2 External network 1 1000 192.168.10.2 255.255.255.0

mls2 Web/application servicerouter

3,4,5,6 1000 10.10.0.1 255.255.255.0

mls2 Directory service router 7,8 1000 10.20.0.1 255.255.255.0

mls2 Database services router 9,10 1000 10.30.0.1 255.255.255.0

8/9/2019 Sun N-tire

43/56


FIGURE 21 Collapsed Design Without Layer 2 Switches

Extre

me Ne

tworks Su

mmit 7

i

Extre

me Ne

tworks

- Blac

k Diamond

6808

Slot 1

client

172.16

.0.1

web

10.10.

0.1

Slot 2

ds

10.20.

0.1

Slot 3

db

10.30.0.1

Slot 4

app

10.40.0.1

Slot 5

DNS

10.50.

0.1

Slot 6

Slot 7

Slot 8

mgt

10.100.0.

1

Extre

me Ne

tworks

- Black D

iamond

6808

Slot 1

web

10.10.

0.1

Slot 2

ds

10.20.

0.1

Slot 3

db

10.30.

0.1

Slot 4

app

10.40

.0.1

Slot 5

DNS

10.50.

0.1

Slot 6

Slot 7

Slot 8

mgt

10.10

0.0.1

edge

192.168.0

.2

esrp in

tercon

nect -

web, d

s, db, a

pp, DN

S

edge

192.16

8.0.2

esrp in

tercon

nect -

web, d

s, db, app,

DNS

edge

192.

168.0.2

Configuring the Extreme Networks Switches

8/9/2019 Sun N-tire

44/56


For the Sun ONE N-Tier, two Extreme Networks BlackDiamond switches were usedfor the core switches and one Summit7i switch was used for the edge switch.

Note – Network equipment from Foundry Networks can be used instead. Refer to“Configuring the Foundry Networks Switches” on page 43.

To Configure the Extreme Networks Switches

1. Configure the core switches.

The following example shows an excerpt of the switch configuration file.

#

# MSM64 Configuration generated Thu Dec 6 20:19:20 2001

# Software Version 6.1.9 (Build 11) By Release_Master on 08/30/01 11:34:27

configure slot 1 module g8x






configure slot 7 module f48t

configure slot 8 module f48t

.....................................................

configure dot1q ethertype 8100

configure dot1p type dot1p_priority 0 qosprofile QP1




.....................................................

enable sys-health-check

configure sys-health-check alarm-level log

enable system-watchdog

config qosprofile QP1 minbw 0% maxbw 100% priority Low minbuf 0% maxbuf 0 K

config qosprofile QP2 minbw 0% maxbw 100% priority LowHi minbuf 0% maxbuf 0 K

2. Configure the edge switch.

Th f ll i l h t f th it h fi ti fil

8/9/2019 Sun N-tire

45/56


The following example shows an excerpt of the switch configuration file.

Configuring the Foundry Networks Switches

This section describes the network architecture implementation using FoundryNetworks equipment instead of Extreme Networks equipment. The overall setup isshown in FIGURE 22.

#

# Summit7i Configuration generated Mon Dec 10 14:39:46 2001

# Software Version 6.1.9 (Build 11) By Release_Master on 08/30/01 11:34:27

configure dot1q ethertype 8100


....................................................

enable system-watchdog

config qosprofile QP1 minbw 0% maxbw 100% priority Low minbuf 0% maxbuf 0 K

....................................................

delete protocol ip

delete protocol ipx

delete protocol netbios

delete protocol decnetdelete protocol appletalk

....................................................

# Config information for VLAN Default.

config vlan “Default” tag 1 # VLAN-ID=0x1 Global Tag 1

config vlan “Default” protocol “ANY”

config vlan “Default” qosprofile “QP1”

enable bootp vlan “Default”

....................................................

8/9/2019 Sun N-tire

46/56


FIGURE 22 Foundry Networks Implementation

Client Client Client Client

Servers Servers

Servers Servers

Servers Servers

Servers Servers

Servers Servers

Webservice

tier

Directory

service

tier

Application

service

tier

Database

service

tier

Netscreen

firewall

NS5200

Netscreen

firewall

NS5200

Server loadbalancer

SLB1

BigIronlayer 2/3 switch

MLS0

BigIronlayer 2/3 switch

MLS1

Server loadbalancer

SLB0

Extreme Networks

Summit 7i

S7i

Foundry Networks Master Core Switch Configuration

8/9/2019 Sun N-tire

47/56


The following code box shows part of the configuration file for the BigIron mastercore switch (called MLS0 in the lab).

module 1 bi-jc-8-port-gig-m4-management-modulemodule 3 bi-jc-48e-port-100-module!global-protocol-vlan!vlan 1 name DEFAULT-VLAN by portvlan 10 name refarch by port untagged ethe 1/1 ethe 3/1 to 3/16 router-interface ve 10!vlan 99 name mgmt by port untagged ethe 3/47 to 3/48 router-interface ve 99!hostname MLS0ip default-network 129.146.138.0/16ip route 192.168.0.0 255.255.255.0 172.0.0.1ip route 129.148.181.0 255.255.255.0 129.146.138.1ip route 0.0.0.0 0.0.0.0 129.146.138.1!router vrrp-extendedinterface ve 10 ip address 20.20.0.102 255.255.255.0 ip address 172.0.0.70 255.255.255.0 ip vrrp-extended vrid 1

backup priority 100 track-priority 20 advertise backup ip-address 172.0.0.10 dead-interval 1 track-port e 3/1 enable ip vrrp-extended vrid 2 backup priority 100 track-priority 20 advertise backup ip-address 20.20.0.100 dead-interval 1 track-port e 3/13 enable!interface ve 99 ip address 129.146.138.10 255.255.255.0!end

Foundry Networks Standby Core Switch Configuration

f f f f f

8/9/2019 Sun N-tire

48/56


The following code box shows a partial listing of the configuration file for theBigIron standby core switch (called MLS1 in the lab).

ver 07.5.05cT53!module 1 bi-jc-8-port-gig-m4-management-modulemodule 3 bi-jc-48e-port-100-module!global-protocol-vlan!!vlan 1 name DEFAULT-VLAN by port!vlan 99 name swan by port untagged ethe 1/6 to 1/8 router-interface ve 99!vlan 10 name refarch by port untagged ethe 3/1 to 3/16 router-interface ve 10!!hostname MLS1ip default-network 129.146.138.0/1ip route 192.168.0.0 255.255.255.0 172.0.0.1ip route 0.0.0.0 0.0.0.0 129.146.138.1!router vrrp-extended

interface ve 10 ip address 20.20.0.102 255.255.255.0 ip address 172.0.0.71 255.255.255.0 ip vrrp-extended vrid 1 backup priority 100 track-priority 20 advertise backup ip-address 172.0.0.10 dead-interval 1 track-port e 3/1 enable ip vrrp-extended vrid 2 backup priority 100 track-priority 20 advertise backup ip-address 20.20.0.100 dead-interval 1 track-port e 3/13enableinterface ve 99ip address 129.146.138.11 255.255.255.0

!

!

!

!

!

sflow sample 512

sflow source ethernet 3/1

sflow enable!

!

!

end

Foundry Networks Master Server Load Balancer Configuration

Th f ll i d b h ti l li ti f th fi ti fil d f th

8/9/2019 Sun N-tire

49/56


The following code box shows a partial listing of the configuration file used for themaster Server XL server load balancer (called SLB0 in the lab).

ver 07.3.05T12

global-protocol-vlan

!

!

server source-ip 20.20.0.50 255.255.255.0 172.0.0.10

!

!!

!

server real web1 10.20.0.1

port http

port http url "HEAD /"!

server real web2 10.20.0.2

port http

port http url "HEAD /"

!

!

server virtual WebVip1 192.168.0.100

port http

port http dsr

bind http web1 http web2 http

!

!

vlan 1 name DEFAULT-VLAN by port

no spanning-tree

!

hostname SLB0

ip address 192.168.0.111 255.255.255.0

ip default-gateway 192.168.0.10

web-management allow-no-password

banner motd ^C

Reference Architecture -- Enterprise Engineering^C

Server Load Balancer-- SLB0 129.146.138.12/24^C

!!end

Foundry Networks Standby Server Load Balancer Configuration

The following code box shows a partial listing of the configuration file used for the

8/9/2019 Sun N-tire

50/56


The following code box shows a partial listing of the configuration file used for thestandby Server XL server load balancer (called SLB1 in the lab).

Network SecurityFor the N-Tier network design, firewalls were configured between each service tier toprovide network security. FIGURE 23 shows the relationship between the firewallsand the service tiers.

ver 07.3.05T12

global-protocol-vlan

!

!

server source-ip 20.20.0.51 255.255.255.0 172.0.0.10

!

!!

!

server real s1 20.20.0.1

port http

port http url "HEAD /"!

server real s2 20.20.0.2

port http

port http url "HEAD /"

!

!

server virtual vip1 172.0.0.11

port http

port http dsr

bind http s1 http s2 http

!

!

vlan 1 name DEFAULT-VLAN by port

!

hostname SLB1

ip address 172.0.0.112 255.255.255.0

ip default-gateway 172.0.0.10

web-management allow-no-password

banner motd ^C

Reference Architecture - Enterprise Engineering^C

Server Load Balancer - SLB1 - 129.146.138.13/24^C

!

Cli Cli

8/9/2019 Sun N-tire

51/56

Network Security 49

FIGURE 23 Firewalls Between Service Tiers

In the lab, one physical firewall device was used to create multiple virtual firewalls.Network traffic was directed to pass through the firewalls between the service tiers,as shown in FIGURE 24.

The core switch is only configured for Layer 2 with separate port-based VLANs. Theconnection between the Netscreen device and the core switch uses tagged VLANS.Trust zones are created on the Netscreen device, and they map directly to the taggedVLANs. The Netscreen firewall device performs the Layer 3 routing. This

configuration directs all traffic through the firewall, resulting in firewall protection between each service tier.

Firewall

Firewall

Firewall

Web service

tier

Application service

tier

Database service

tier

Edge

switch

Intranet/Internet

Client Client

Client Client

8/9/2019 Sun N-tire

52/56


FIGURE 24 Virtual Firewall Architecture Using Netscreen and Foundry Networks

Products

Edge

switch

Intranet/Internet

Client Client

Core

switch

Core

switchVLAN*

Database VLAN

Netscreendevice Netscreen

device

VLAN*

Application VLAN

Web VLAN

Database VLAN

Application VLAN

Web VLAN

Web service

tier

Application service

tier

Database service

tier

Traffic

*Web, application and database traffic multiplexed on one VLAN

Netscreen Firewall

The following code box shows a partial example of a configuration file used to

8/9/2019 Sun N-tire

53/56

Network Security 51

e o o g co e o s o s p e p e o co g o e se oconfigure the Netscreen device.

set auth timeout 10

set clock "timezone" 0

set admin format dos

set admin name "netscreen"

set admin password nKVUM2rwMUzPcrkG5sWIHdCtqkAibn

set admin sys-ip 0.0.0.0

set admin auth timeout 0

set admin auth type Local

set zone id 1000 "DMZ1"

set zone id 1001 "web"

set zone id 1002 "appsrvr"

set zone "Untrust" block

set zone "DMZ" vrouter untrust-vr

set zone "MGT" block

set zone "DMZ1" vrouter trust-vr

set zone "web" vrouter trust-vr

set zone "appsrvr" vrouter trust-vr

set ip tftp retry 10set ip tftp timeout 2

set interface ethernet1 zone DMZ1

set interface ethernet2 zone web

set interface ethernet3 zone appsrvr

set interface ethernet1 ip 192.168.0.253/24

set interface ethernet1 route





unset interface vlan1 bypass-others-ipsec

unset interface vlan1 bypass-non-ip

set interface ethernet1 manage ping

unset interface ethernet1 manage scs

unset interface ethernet1 manage telnet

unset interface ethernet1 manage snmp

unset interface ethernet1 manage global

unset interface ethernet1 manage global-pro

unset interface ethernet1 manage sslset interface ethernet1 manage web

unset interface ethernet1 ident-reset

set interface vlan1 manage ping

set interface vlan1 manage scs

set interface vlan1 manage telnet

set interface vlan1 manage snmp

set interface vlan1 manage global

set interface vlan1 manage global-pro

set interface vlan1 manage ssl

set interface vlan1 manage web

set interface v1-trust manage ping

set interface v1-trust manage scs

set interface v1-trust manage telnet

8/9/2019 Sun N-tire

54/56


set interface v1-trust manage snmp

set interface v1-trust manage global

set interface v1-trust manage global-pro

set interface v1-trust manage ssl

set interface v1-trust manage web

unset interface v1-trust ident-reset

unset interface v1-untrust manage ping

unset interface v1-untrust manage scs

unset interface v1-untrust manage telnet

unset interface v1-untrust manage snmp

unset interface v1-untrust manage global

unset interface v1-untrust manage global-pro

unset interface v1-untrust manage ssl

unset interface v1-untrust manage web

unset interface v1-untrust ident-reset

set interface v1-dmz manage ping

unset interface v1-dmz manage scs

unset interface v1-dmz manage telnet

unset interface v1-dmz manage snmp

unset interface v1-dmz manage global

unset interface v1-dmz manage global-pro

unset interface v1-dmz manage ssl

unset interface v1-dmz manage web

unset interface v1-dmz ident-reset







unset interface ethernet2 manage sslunset interface ethernet2 manage web








unset interface ethernet3 manage ssl

unset interface ethernet3 manage web


set interface v1-untrust screen tear-drop

set interface v1-untrust screen syn-flood

set interface v1-untrust screen ping-death

set interface v1-untrust screen ip-filter-src

set interface v1-untrust screen land

set flow mac-flooding

set flow check-session

set address DMZ1 "dmznet" 192.168.0.0 255.255.255.0set address web "webnet" 10.10.0.0 255.255.255.0

set address appsrvr "appnet" 20.20.0.0 255.255.255.0

set snmp name "ns208"

set traffic-shaping ip_precedence 7 6 5 4 3 2 1 0

t ik li h ki

8/9/2019 Sun N-tire

55/56

About the Authors 53

About the AuthorsDeepak Kakadia is a staff engineer and network architect in the ReferenceArchitecture Engineering Group for Sun Microsystems in Menlo Park, California.Deepak has been with Sun since 1994. He has previously worked with CoronaNetworks, Digital Equipment Corp., and Nortel Networks. He has a Bachelor’sDegree in Computer Systems Engineering, a Master’s Degree in Computer Science,has completed Ph.D qualifying exams, and has completed his SCPD certificate inNetworking through the Department of Electrical Engineering at StanfordUniversity. He has filed four patents in the area of Networking and SystemsManagement.

Richard Croucher is Chief Architect of Sun’s Professional Services in EMEA. He isalso an SMI Technical Director. Richard joined Sun in 1995. He has nearly 20 years of experience with UNIX and nearly 30 years of experience in the IT and electronicsindustries. He holds a Higher National Certificate in Applied Physics, a Higher

National Diploma in Electronics, and has received a post-graduate Certificate of Advanced Study in Non-Destructive Testing and Materials Science from BrunelUniversity. He was elected to membership of the British Computer Society in 1993.

set ike policy-checking

set ike respond-bad-spi 1

set ike id-mode subnet

set l2tp default auth local

set l2tp default ppp-auth any

set l2tp default radius-port 1645

set policy id 0 from DMZ1 to web "dmznet" "webnet" "ANY" Permit

set policy id 1 from web to DMZ1 "webnet" "dmznet" "ANY" Permit

set policy id 2 from DMZ1 to appsrvr "dmznet" "appnet" "ANY" Permit

set policy id 3 from appsrvr to DMZ1 "appnet" "dmznet" "ANY" Permit

set ha interface ethernet8

set ha track threshold 255

set pki authority default scep mode "auto"

set pki x509 default cert-path partial

_____________________

Acknowledgements

8/9/2019 Sun N-tire

56/56


AcknowledgementsSincere thanks to Bill Sprouse, Kemer Thomson, Brad Carlile, and many others whoprovided tremendous support, guidance, and direction to this article.

Related ResourcesSun ONE Directory Server 5.1 documentation collection

Note: This product was formerly known as the iPlanet™ Directory Serversoftware. You can locate the documentation collection at:

http://docs.sun.com

Search for the phrase directory server 5.1. Start with the iPlanet Directory

Server 5.1 Deployment Guide.

Ordering Sun Documents

The SunDocsSM

program provides more than 250 manuals from Sun Microsystems,Inc. If you live in the United States, Canada, Europe, or Japan, you can purchasedocumentation sets or individual manuals through this program.

Accessing Sun Documentation OnlineThe docs.sun.com web site enables you to access Sun technical documentationonline. You can browse the docs.sun.com archive or search for a specific book titleor subject. The URL is http://docs.sun.com/

To reference Sun BluePrints™ OnLine articles, visit the Sun BluePrints OnLine web site at:

http://www.sun.com/blueprints/online.html