Session 214552573 City of SacramentoPeopleSoft 8.9 Infrastructure PlanningWednesday, April 16th 9:45am

Presenter Name: Ray Songco

Title: Technical Director

CherryRoad Technologies Inc.

Project Scope

Sizing Methodology

Architecture Design

Q & A

Agenda

The capital of the State of California and the county seat of Sacramento County.

Seventh most populous city in California with a 2007 estimated population of 467,343.

The core cultural and economic center of its four-county metropolitan area (El Dorado, Placer, Sacramento, and Yolo counties) with a combined population of 2,103,956.

The Sacramento Metropolitan Area is the largest in the Central Valley, and is the fourth-largest in California, behind the Greater Los Angeles Area, the San Francisco Bay Area, and the San Diego area.

Greater Sacramento has been cited as one of the five "most livable" regions in America, and the city was cited by Time magazine as America's most integrated.

About the City

Electronic Citywide Accounting and Personnel System (eCAPS) Financials Phase I: Financials 8.9, Portal 8.9

HCM Phase I: HCM 8.9

Financials Phase II: Budgeting, Cash/Deal Management, Vendor Self-Service

HCM Phase II: ELM, Employee Self-Service, Candidate Gateway, Talent Acquisition Manager, eApplications

Project Scope

Other Applications (non-eCAPS) Revenue Management – CIS

Customer Relationship Management – Siebel

Land Management – Accela

Identity Management – TBD

Enterprise Job Scheduling – Tidal

Project Scope

Infrastructure Planning Before

IT Standards

Hardware Sizing

Redundancy

During

Load Testing

Security

After

Production Environment Administration

Multi-Phase Strategy

Project Scope

Project Scope

IT Standards Platform

SUSE Linux/Oracle 10g

HP Proliant Series

High Availability

Multiple Domains

Content Switching

Disaster Recovery

Security

LDAP Integration

Multiple Firewalls

SSL Encryption

Sizing Approach

Step 1: Estimate Peak Utilization (PSTPM)

Step 2: Calculate Database Server Utilization

Step 3: Estimate Database Server Memory

Step 4: Calculate Application Server Utilization

Step 5: Estimate Application Server Memory

Step 6: Calculate Web Server Utilization

Step 7: Estimate Web Server Memory

Step 8: Determine Network Bandwidth Requirements

Sizing Methodology

Step 1: Gather Peak Utilization and Calculate Peak Online PeopleSoft Transactions per Minute (PSTPM):

Heavy

Medium

Light

Self-Service

Sizing Methodology

Step 2: Calculate Database Server Utilization

Add 20% for daytime batch processes

Calculate TPC-C Requirement, Total Peak PSTPM

Sizing Methodology

Database Server

Look up www.tpc.org for TPC-C rating

Sizing Methodology

Database Server

If no exact TPC-C match found, pick a proxy and use www. spec.org to determine % differential:

Sizing Methodology

Database Server

Extrapolate TPC-C Rating from Differential

Calculate Peak CPU Utilization

Sizing Methodology

Step 3: Estimate Database Server Memory

Estimated based on previous customer experience

Stratified ranges based on estimated database size

Headcount

Applications

Sizing Methodology

Step 4: Calculate Application Server Utilization

Get SPEC_Int Rating of Application Server

If no exact match found use proxy (see Database Server Sizing)

Regression: 0.25 SPEC_Int Rating (SIR) per PSTPM

Calculate Peak Utilization at SIR Requirement / SIR Capacity

Sizing Methodology

Step 5: Estimate Application Server Memory

Regression: RAM requirement based on application (e.g., HCM User = 100 MB RAM)

Calculate Memory Requirement - round up to 2 GB increments.

Sizing Methodology

Step 6: Calculate Web Server Utilization

Basic PeopleSoft guidelines set ratio of 1:4 for Application Server to Web Server

Sizing Methodology

Web Server

Regression: each concurrent user consumes 5 MB RAM

Calculate Memory based on Peak Concurrent Users

Sizing Methodology

Network Capacity

PIA Servers - Dual Gigabit

Internal Clients – 100 MBPS

Self-Service – Sufficient ISP Bandwidth:

Sizing Methodology

Total Capacity

Aggregate metrics for each Application Suite

If hardware already procured, calculate peak utilization levels:

ComponentDatabase Server FMS/Portal HCM EPM

CPU Utilization 44.18% 62.05% 45.04%Memory Requirement 12 12 12

Application/Web ServerCPU Utilization 80% 110% 80.11%

Memory Requirement 12 6 8

Peak Utilization

Sizing Methodology

Total Capacity

For Infrastructure Design, provide hardware options based on selected vendor:

Sizing Methodology

Sizing MethodologyFinal Configuration

Lessons Learned

Be conservative! Better to oversize than undersize.

Hardware is driven by vendor models, not metrics.

Go to vendor online stores and “shop” for servers to gain familiarity with servers.

Remember that estimates are exactly that, and are proven out later during load testing.

TPC and SPEC_Int are becoming less reliable. May need to switch to other measure such as Summation of GHz.

Non-production hardware sizing usually based on previous sizing information for similarly-scaled customers.

Sizing Methodology

What is Redundancy? Failover – capability to switch over automatically to a redundant or

standby system upon the failure or abnormal termination of the previously system. Failover happens without human intervention and without warning (vs. Switchover).

Load Balancing - load balancing is a technique (usually performed by load balancers) to spread work between many computers, processes, disks or other resources in order to get optimal resource utilization and decrease computing time.

Architecture Design

Architecture DesignLevels of Redundancy

DataCenter

DataCenter

NetworkNetwork

NetworkNetwork

HardwareComponent

HardwareComponent

HardwareComponent

HardwareComponent

Software-Based

Software-Based

Software-Based

Software-Based

Remote Disaster Recovery site

Load Balancing/Failover appliances, Server farms

Hardware redundancy(multiple NIC’s, UPS, etc.)

Built into middleware(multiple web servers,Domains, etc.)

Architecture DesignTypical PeopleSoft Redundancy

Architecture DesignTypes of Scaling

Horizontal Scaling – Multiple smaller servers grouped together by a load balancer Pros – Cheaper boxes, scalability, no obsolescence, ease of

administration

Cons – Support costs

Vertical Scaling – Single enterprise-class server, upgradeable CPU, memory chips Pros – Lower support cost

Cons – Expensive, parts phased out, not as scalable

PIA Load Balancing/Failover

Content Switches

Internal User Redundancy

External Enterprise Redundancy

Disaster Recovery Site

Phase I: Web and Application Tiers

Phase II: Database Tier (RAC)

Architecture Design

Architecture DesignProposed Enterprise Architecture

Load Testing Objectives

Integrated Workload Testing – take all defined user transactions and mix them into a single workload that is to be representative of a) anticipated production volume, b) peak utilization and c) point of failure.

Long-term Stability Testing – run an 11 hour stability test to simulate a single workday. This will help identify potential memory leaks and the overall stability or decay potential of the application.

Batch Process Testing – use scripts to generate volume transactions to be used to simulate nightly batch processing.

Architecture Design

What is Borland SilkPerformer?

Server-Component Testing tool

Utilizes Virtual Users (VU’s) to simulate utilization

Script-based testing to replicate online processes

Consists of SilkPerformer Server and SilkPerformer Agents

Optional: SAM module to provide detailed metrics

Architecture Design

verify response

s

TrueLog

Agent Health Control

SilkPerformerController

generate requests

System Under Test

Virtual Users

Real-timeServer

Monitoring

SilkPerformerAgents

ViewResults

Borland SilkPerformer Overview

Architecture Design

Prerequisites to Load Testing

Functional PIA Environment on Production (or production-sized) Servers

Redundancy

SilkPerformer Server and Agent Machines

Load Testing Discovery

IT Department Support

Integration Testing

User Security Profile

Architecture Design

250 Virtual User Load Test

Architecture Design

Load Test Response Times

Architecture Design

Performance Tuning Areas

Database Tuning

PIA Parameters

Customization Code

Batch Processes

Architecture Design

Lessons Learned

Determine peak utilization requirements early in order to get minimum number of VU’s for license.

Services + spike license is cost-effective option.

Plan two cycles of Load Testing to incorporate performance tuning in between.

Disperse SilkPerformer agents across multiple locations to also test different networks.

Make sure Integration Testing is complete or nearly complete so scripts work properly.

Define User Security Profile in advance (along with setup click paths) to expedite virtual user generation.

Install redundancy components prior to Load Testing so entire infrastructure is tested.

Architecture Design

Security

Secure Sockets Layer (SSL)

SSL Accelerator in Content Switch

Wildcard Certificate

De-Militarized Zone (DMZ)

Multiple – DMZ + SDMZ

Impact on Architecture

Content Switch Location

Physical vs. Logical Web/App Server

Architecture Design

Multi-Phase Strategy

Architecture Design

PSFDMOPSFDMO

PSFSNDPSFSND

PSFDEVPSFDEV

PSFSTGPSFSTG

PSFCNVPSFCNV

PSFTSTPSFTST

PSFTRNPSFTRN

PSFPRDPSFPRD

Phase IImplementation

PSFDMOPSFDMO

PSFSNDPSFSND

PSFDEVPSFDEV

PSFSTGPSFSTG

PSFCNVPSFCNV

PSFTSTPSFTST

PSFTRNPSFTRN

PSFPRDPSFPRD

Production

PSFDMOPSFDMO

PSFSNDPSFSND

PSFCNVPSFCNV

PSFTRNPSFTRN

PSFPRDPSFPRD

Phase IIImplementation

PSFDMOPSFDMO

PSFSNDPSFSND

PSFDEVPSFDEV

PSFSTGPSFSTG

PSFTSTPSFTST

PSFTRNPSFTRN

PSFPRDPSFPRD

Production

PermanentDatabase

PermanentDatabase

TemporaryDatabase

TemporaryDatabase

Carry OverCopy to Existing

Legend

Copy to New

PSFDEV2PSFDEV2

PSFSTG2PSFSTG2

PSFTST2PSFTST2

FC

S Im

ple

men

tati

on

Ser

ver

sT

est/

UA

T

Ser

ver

sP

rod

uct

ion

S

erv

ers

Lessons Learned Redundancy incorporating DR Site requires load balancing/failover

across ALL tiers

Manage eCAPS architecture against non-eCAPS initiatives

Separate networks can create issues (access, integration constraints, infrastructure management)

Employee Self-Service inherits back-office access

LDAP Integration Increases HCM Security Risk

Load Testing must account for future phases

Plan Change Control to cover both Production and Phase II activities

Architecture Design