Scaling Cloud Across the Enterprise

Chris Haddad Vice President Technology Evangelism, WSO2

Asanka Abeysinghe

Director, Solutions Architecture

Follow me @cobiacomm Read more about PaaS at http://blog.cobia.net/cobiacomm

Try StratosLive https://stratoslive.wso2.com/

Your Presenters Chris Haddad WSO2 Vice President Technology Evangelism Former Burton Group and Gartner research leader whose main role is to raise visibility, awareness, and knowledge of the Platform as Service, Service Oriented Architecture, and API Management Twitter @cobiacomm http://blog.cobia.net/cobiacomm Asanka

WSO2 Director, Solutions Architecture

Twitter @asankama

http://asanka.abeysinghe.org

Scaling Cloud Across the Enterprise

• Where your cloud application strategy may encounter challenges

• When to migrate applications to the cloud

• How to scale your cloud strategy and cloud tactics across the organization.

Application Development Challenges

• Long time and significant effort required to create application project environment(s)

• Complex integration required to extend core application capabilities and bridge consumers

• Difficult to enforce governance best practices and policies

• Time consuming, manual process to obtain project status, dependencies, and change impact

Cloud Governance Challenges

• New data types and metadata

• Interface with new Cloud management infrastructure

• Resurgent focus on bridging silos and creating a hybrid solution requires extensibility

• Emerging specifications and practices requires customization and experimentation

What is Platform as a Service?

Cloud Characteristics and Architecture

Resource pooling

• Multi-tenancy

• Resource utilization

• Shared, virtual infrastructure

• Interoperability

On-demand

self-service

• Flexible workload assignment

• Standard service offerings

• Quick startup and automation

Rapid Elasticity

• Stateless services

• Rapid provisioning

• Flexible topology

• High Quality of Service

Measured service or pay per use

• Fine-grained metering

• Billing • Reporting

Defining Cloud Boundaries • Distributed Topology

– Latency – Stateless, autonomous services – Instrumented, redundant, and expecting failure

• Efficient resource pooling – Consumption aware and multi-tenant – Automated, continuous

• Scale – Asynchronous, event driven callbacks – Parallelizable, workload decomposition, and idempotent

• On-demand self service – Personalization extensibility points – Fine grained, modular – Template driven

Dev Ops Tooling Automated Governance

Service Level Management

Consumption based Funding

Platform as a Service Capabilities

• Resource monitoring

• Resource management

• Performance management

• Traffic orchestration

• Metering

• Reporting

• Billing

• Service catalogue

• Service tiers

• Demand and capacity management

• Lifecycle management

• Infrastructure Authority integration

• Self service configuration

• Automated provisioning

• Policy configuration

• Process automation

Dev Ops Tooling

Automated Governance

Service Level Management

Consumption based Funding

Platform as a Service Capabilities

PaaS Abstraction Levels

• Users, workspaces, transactions

• Services, channels, API

Component Services

• Application session, frameworks

• Application instance, .war files

• Installing application

Application Platform

• Command line interface, bash shell

• Machine size, JVM configuration

• Installing infrastructure

Hardware Infrastructure

Resource Tier

Cloud Scale

Functional Role

Client Tier

Integration Services

Integration Services Resource Services

Functional Role

Presentation and Mashups

Functional Role

Functional code

Presentation Role

Presentation and Mashups

Presentation and Mashups

Resource Services

Private Applications

Public Cloud Services

Business Proces

Business Process

Business Process Business Process and

Business Rules

Bridging Clouds Mobile

Application

Application Portfolio

Hybrid Platform Services

Presentation Services

Resources Services

Functional Services

Business Process and

Rules Services

Integration Services

Composite Application

Business Service Portfolio

Integration Services

Cloud API

Endpoint

Infrastructure as a Service (IaaS)

SaaS Application

Web application Endpoint

Composing a Cloud Platform

Cloud Platform

Cloud Management

PaaS Manager

Cloud Governance

Identity Management

Asset Repositories and Registries(tenant code, service

endpoints, meta-data, configuration, policies)

Infrastructure as a Service (AWS, Eucalyptus, OpenStack,

CloudStack)

Platform as a Service Run-time Framework (i.e. WSO2 Stratos)

Service-aware, tenant-aware

Elastic Load Balancer

Stratos Controller

Asset/Code Deployer

Asset/Code Synchronizer

Metering and Billing

Cloud Native Container(s)

Application Platform Services (web server, database, ESB)

Tenant 1 Tenant (n)

Composing a Cloud Platform

Load Balancer

Manager

Repository

Code Deployer

Platform Services

Metering and Billing

Controller

Cloud Application Platform Architecture

Platform as a Service

Cloud Application Platform Shared Middleware Services

Platform as a Service Architecture What is a tenant? • An isolated or personalized run-time environment context that cannot be

shared across PaaS consumers

• Tenant specific personalization can occur across multiple personalization dimensions • Information access privileges • Information aggregation and composition • Business processes and rules • Service levels and Quality of Service • Security policies, subscriber entitlements, and social network access privileges • Monetization rates

• Personalization may require loading code, configuration files, or data

• Tenant isolation dictated by expected performance, security requirements, and legacy technology. • PaaS security managers, code deployers, and tenant-aware load balancing

influences required container-level isolation

Platform as a Service Architecture

What is a container? • A standalone, Internet addressable node offering

application platform services • Web application hosting, API management, integration

endpoint hosting, ESB mediation, registry services, identity management, relational database

• Containers host tenant resources and context

• Code, configuration files, data, process definitions, rules, policies, entitlements

• Containers may serve

• a single tenant at a time (dedicated), or • multiple-tenants at a time (shared)

Tenant Scaling Method Impacts Value Proposition • Single Application

Tenant per

• Dedicated Application Server Container

Multiple Application Tenants per

Shared Application Platform Service Container

Partitions modify value proposition

• Single, flat namespace Multiple partitions and segmented name space

Partitioning and Tenancy Implications

Tenant Context

Container

Service Partition

Application Solution Tenant Traffic

ESB

ESB-1

Tenant-1 Tenant-3

ESB-2

Tenant-2

Web Applicati

on

AS-1

Tenant-1 Tenant-2

Three (3) Container Instances

Scaling your Cloud

How to size an infrastructure

• Expected Maximum TPS

– Any max latency requirements

• Work done per transaction

• Size of messages

• Availability

• Deployment needs (e.g. specific setup might need based on the infrastructure policy etc.)

Cloud Capacity Planning

• Service types

• System load – Per service type

– Administration

• Scaling – Static

– Dynamic

• PaaS system and deployment architecture

• Limitation of IaaS

• Stratos Controller

- Management, Governance, Monitoring component - System security

• Stratos Service Cluster - Worker nodes provide the runtime for application logic / services - Administrator nodes to create, manage, govern the application

logic / services

Controller

Service Cluster

1:M

WSO2 Stratos Component Architecture

Scaling Stratos Controller

i – Number of WSO2 Carbon instances x – Number of worker nodes per service cluster y – Number of service clusters In most cases each service node will not have equal number of worker nodes.

i = 2 + (2 +x) y

i = 2 + (2 + x1) + (2 + x2) + …….. + (2 + xn)

Stratos Controller

LB + Admin Node

Z

Stratos Controller instances.

Formula for Static Scaling

Optimizing Cloud Performance: Cloud Scaling

Cloud Aware App

Architectural Difference Between Web Application and Cloud Application

Web Application • Synchronous request-reply

interaction • Centralized state (i.e. single

database) and session management

• Clustered server instances • Silo architecture

Cloud Application • Asynchronous interaction • Queues and workers • Scale out across datacenters

and providers • Distributed state and session

management • Autonomous service instances • Tenant context personalization • Shared JVM / Shared Schema • Shared nothing architecture

PaaS Evaluation Framework

Programming Model • Actor model (i.e. message passing instead of function

invocation • RESTful interactions • Dynamic recoverability • Consensus protocols • Asynchronous rather than synchronous interactions • Shared nothing architecture • Data partitioning and sharding • Federated data queries • Service orchestration • Functional programming • MapReduce

Cloud Application Demo • Register a new tenant

– Show the self service aspect, multi tenancy aspect

– Create and provision a new tenant on shared infrastructure

• Demonstrate PaaS facilities available for a tenant (metering, billing, logs)

• Upload an existing web application to the tenant

– Demonstrate the zero cost/easy migration of applications

• Creating a new hello world web application

– Demonstrate the procedure to create an application

– Deploy application to Cloud using WSO2 Dveloper Studio

• Secure the web application(Tie the authentication to underlying user model).

• Create a database

– Relational storage model options

– NoSQL storage model options

• Access a database via web application

• Expose a database table as data services.

• Demonstrate WSO2 Stratos and WSO2 StratosLive application platform services

• Explain the programming model + services

– Access "CarbonContext” within applications

Optimizing Cloud Performance: Asset Lifecycle

Tenant Traffic Balancer

Code Deployer

Activity Metering

Instance Scaling

Code Development and Update

Code Synchronizer

Optimizing Cloud Performance

Stratos Monitoring, Metering, and Billing

• Number of users and applications

• Tenant Storage

• Tenant transaction activity

Cloud-aware Design

Application Architecture Crossroads

Innovation

Familiarity

Migrated or Native

Web Application • Synchronous request-reply

interaction • Centralized state (i.e. single

database) and session management

• Clustered server instances • Silo architecture

Cloud Application • Asynchronous interaction • Queues and workers • Scale out across datacenters

and providers • Distributed state and session

management • Autonomous service instances • Tenant context personalization • Shared JVM / Shared Schema • Shared nothing architecture

Cloud Application Patterns and Anti-Patterns

18

Deterministic performance

Deploy and execute on optimum topology

Separation of concerns

Embarrassingly Parallel / Shared Nothing ArchitectureMinimal

Consumption

Failure Resilient Leaky interfaces

Tightly coupled modules

Monolithic footprint

Single threaded, serial execution

Resource locks Single tenancy model

Cloud Aware Application Use Cases and Underlying Cloud Patterns

• Maximize utilization – Requires deterministic performance – Load balance based on tenant, service, and workload,

context

• Increase reliability, availability, scalability – Shared nothing architecture – Stateless server-side elements – Consensus protocols

• Ecosystem platform – Monetize assets based on business value – Tenant/Consumer personalization and isolation – Sharing domain specific business capabilities

Cloud Architecture Best Practices

Transitioning to a New normal – Traditional practices may not apply • Distributed and federated interactions

– Event based, heterogeneous systems, network latency

• Configurable containers and engines – Declarative data, rules, and process definitions

• De-normalized and simplified data models – Hadoop/BigTable, Hypertext media, simple NoSQL entities

• Expect failure – Systems span transactional control

• Applications decomposed into distinct services – Federated environment drives autonomy, statelessness, and

composition

Cloud-aware Design

Cloud-aware Application

Parallelizable, Shared nothing

Asynchronous, stateless services

Fine grained, modular design

Tenant personalization

Efficient resource consumption

Deterministic performance

Multi-tenant Application Platform Services

ESB

Application Server

Business Process

Registry

Identity Management

Storage

PaaS Framework

Controller Load

balancer

Asset Deployer and Synchronizer

Repositories Metering and

Billing

Scaling Cloud Adoption

Cloud Business Value Blockers

• Silo owners

• Adoption curve

• Trust barrier

• Unknown baseline

• Funding model

Source: http://blog.industryweapon.com/2010/01/beware-of-the-business-blocker/

Architecture Shapes Value

• Resource friendly := small footprint, minimal consumption,

parallelizable workloads

• Poolablility := Non-exclusive use, short usage duration

• Automation := decrease activity time, latency between

steps

• Governance := encourage trust, communication,

coordination

• Distribution friendly:= Stateless, autonomous

DevOp Principles

• Iterative

• Incremental

• Continuous

• Automated

• Self-service

• Collaborative

• Holistic

DevOps Processes

• Self-service configuration

• Automated provisioning

• Continuous build

• Continuous integration

• Continuous delivery

• Automated release management

• Incremental testing

DevOps Tooling

• Self service project and policy configuration – Via project configuration portals

– Security, service levels, frameworks, usage, topology

• Automated platform provisioning – Via service tier templates

– Framework and policy enforcement points (PEPs)

• Process automation – Continuous build, test, and deployment

– Code promotion and synchronization across environments and servers

• Dependency analysis and impact analysis

Defining Governance

• Ensuring people do the right thing

• Make doing the right thing the easy path – via automation

• Spans precepts, policies, people, and process

Governance applies to many domains – Design and development

– Operational processes and run-time environment

– Organizational resources

• Effective governance requires a big picture view of portfolio and dependencies

Cloud Governance extends general purpose Registry & Repository

• Store Cloud Governance Artifacts:

– Contracts, Models, Workflows, Service Definitions, Documentation

– SLAs, Monetization Rates, Usage Limits

– Cloud Service Dependencies and Machine Instance Configurations

– Scale and Resource Pool Policies

Automated Governance

Requires

• Service catalogue and service tiers

• Demand and capacity management

• Lifecycle management and workflow

• Continuous provisioning and deployment

• Cloud management integration

WSO2 AppFactory

A Cloud-based, agile, DevOps approach to application design, development, and delivery

– On-demand self service and Cloud provisioning

– Continuous build, continuous integration, continuous test, and continuous delivery

– Governed, iterative lifecycle management across hybrid clouds and composite applications

WSO2AppFactory

Existing IT infrastructure

Scalable Private Cloud infrastructure

CIO/Management Dashboard

Workflow managedbuild and deploy

Stratos Multi-tenant elasticintegration and ESB runtime

Governanceand Identity

API Management

Lightweight High PerformanceIntegration

Partner and Mobile applications

WSO2 AppFactory

WSO2 AppFactory

• For developers • Simplifies project setup and provisioning

• Fits into their existing IDE (i.e. Eclipse, IDEA, Rational)

• Works with leading development tooling (i.e. SVN, Git, Junit, Selenium, Maven, Jenkins, Bamboo, and JIRA

• For development managers • Manage project lifecycle with well defined checkpoints and phases

• Ensures projects follow governance model and best practices

• Provides consistent frameworks and architecture

• For the CIO and Executive IT management • Development dashboards deliver at a glance view of projects’ status, lifecycle,

activity, and health

WSO2 App Factory

7/25/2012 55

Overcoming Cloud Value Blockers: WSO2 AppFactory

Configuration Governance • Governs any kind of server/system configuration

• Manages Versions and Revisions with checkpointing & rollback

• Manages the full promotion lifecycle spanning across development, testing, staging and production environments

• Deeply integrates with all WSO2 Carbon middleware products

• Integration to any 3rd party product via command-line clients

Key Metrics

• Foundation • Time to create new application environment

• Time to redeploy application

• Optimize • Minimum and maximum scale

• Scale frequency (i.e. time to scale up/down)

• Transformation • Time and effort required integrating business process, event

processor – creating a complex app.

• Time and effort required to apply policy across tenant(s)

• Cost to operate application per user or transaction

Quick Start Use Cases

• DevOps Tooling and On-demand self-service

• Automated Governance

• Service level management and elastic scale

• Consumption based pricing and billing

Resources • Try StratosLive right now:

– https://stratoslive.wso2.com/

• Read about Stratos: – http://wso2.com/cloud/stratos/ – Source Download available

• White Paper – Selecting Platform as a Service

• Blog Articles – What is Platform as a Service? – PaaS Evaluation Framework for CIOs and Architects – How to simplify Platform as a Service Complexity – Searching for Cloud Reference Architecture

• Contact us: – bizdev@wso2.com

Contact us:

http://wso2.com/contact/

Follow us:

http://twitter.com/#!/wso2