Seasonal burst handling using hybrid cloud

infrastructureFrancois Lascelles

Chief Architect

Layer 7 Technologies

@flascelles

The peak

Peak frequency, peak deviation

f(Frequency, Deviation)

All the time

Once a day

Once a month

Once per season

Once per year

One time only

50% over avg

2X avg

10X avg

1000X avg

∞ avg

Service zones

Private

- Private trust

Public cloud

- Elasticity1-n

API-based integration

Coordinating service zones

Declarative abstraction layer

Handle at perimeter:

- Secured communication

- Distributed policy enforcement

- Data Distribution

home

public cloud

region 1

public cloud

region 2

Hybrid cloud infrastructure based coordination

Apply to traffic in both directions:

- Routing rules

- Translation rules

- Caching rules

- Access rules

- Security rules

API/Service Gateway at perimeter of each zone

Trust management across zones

SAML

OAuth

OpenID Connect

XACML

Federate authentication to on-

premise identity authority

PKI-based trust

Authenticate identities

Issue claims, tokens

Centralized authorization rules

trust

SAML

SAML Based authentication, authorization

- User redirection

- API based authorization and attribute statements

Establish session

Authenticate, request attributes

Get back SAML Authorize access to

service based on SAML

assertions associated with

session

Authenticate credentials

against id provider++SAML

OAuth across zones

Decouple AS and RS

- authorization server on premise and

resource server on cloud

- If opaque tokens and support for

revocation requires call back to issuing

side (less scaling)

- Otherwise use self-signed tokens, no

revocation or pushed revocation, short

token lifespan

Each zone manages its own tokens

- Still delegate authentication

- Scale best

- No centralized token management,

localized revocation

Which ID?

Which scope?

Still valid?

abc123

OAuth Authorization Server

OAuth Resource Server

abc123

Identities

Backends

Federate identity from public to private zone

OpenID Connect

- Standardizes how an OAuth handshake is used to delegate authentication

- Standardizes the API to discover an API authenticated with OAuth

- JWT instead of SAML

/userinfo

Authorization: Bearer [oauth access token]

{

"user_id": "248289761001",

"name": "Jane Doe",

"given_name": "Jane",

"family_name": "Doe",

"preferred_username": "j.doe",

"email": "janedoe@example.com",

}

• OAuth Authorization Server

• UserInfo endpoint

Scaling up on public cloud

AWS-based case study

- Front-end load distribution

- Back-end load distribution

- Distributed policy configuration

- Service endpoint discovery

- Perimeter gateway resize

ELB

Perimeter gateway cluster

RDS

Service endpoint instances

Auto Scale

Ramp-up rate

Time to instantiate a new instance: 5 minutes

Time to instantiate 10 instances: 5+ minutes

Nr instances per increment controls how fast you meet increased demand

In some cases, you can’t react fast enough

t

Sudden demand increase example

t: tickets for U2 concert go on sale

t+3m: sold out

t+3 minutes

Capacity planning and automatic scaling

Planned capacity adjustments

- Capacity planning: understand your traffic patterns, regular patterns and growth

over time

- Adjustment in capacity can be automated (e.g. autoscaling scheduled actions)

- Stay ‘ahead’ of demand

- Future demand cannot always be accurately predicted

Automatic scaling

- Temporary increase in capacity to address unexpected change in demand

- Best practice: limited by a ceiling

- Reaction time in minutes

- Notify provider so that unexpected peak be analyzed and further action determined

Perimeter gateway cluster on public cloud

Every node from cluster share same persistence layer instance

- Leverages RDS

Traffic is distributed across nodes

- ELB

…

RDS instance for this cluster

ELB vip

Scale-up perimeter gateways

Adding new gateway instances needs to be automated, no manual steps

New gateway instantiated, need to discover RDS target and credentials

- Cannot be part of AMI snapshot, nor passed as user data

Admin gateway instance holds this information for new gateway instantiated in such

fashion

Current cluster

AutoScale Group

++node

Scale controller

Master instance

(discover RDS

API)

AWS API

Is this requester

really part of the

AS group?RDS host?

Credentials?

Configuration across regions

home

public cloud

region 1

public cloud

region 2

rds

rdsUAT

Localize and

push to each

zone

Promote

Automated change

provisioning and mapping

between environments

- Policy updates

- New services

Push+cache

home

++backend

++cache

Localize and

push to each

zone

++new or changed content

++backend

++cache

public cloud

region 1

public cloud

region 2

Backend load-balancing pool management

Backend endpoints scale using typical AutoScale Group strategy

Distribution from perimeter to backend is direct (no ELB)

Gateways discover backend instances by pulling

backends

…

Regular pull

through API

call

distribute

Monitoring and

scaling

infrastructure

Backend load-balancing pool management

Pushed updates

- Gateway expose API to receive backend pool information

- Refresh pushed by same process which managed the pool in first place

…

Monitoring and

scaling

infrastructure

Push change

in backend

pool

distribute

backends

Application-aware auto-scaling

Monitor over time

- Concurrency

- Backend response time

- Nr queued

- Per service, per operation

- Application-level aware monitoring

++instance

--instancecloud director

Backend/data

instances

API traffic

API traffic

Manage backend instances scale-up and

scale-down

Interface with vCloud director API or

equivalent

Backend load-balancing pool management

Multi-pool coordination

- Perimeter gateway calculates effective service backend pool based on preferential

instances combined with varying availability

10.7.2.91

Primary pool

10.7.2.92

10.7.2.93

10.7.2.94

Secondary pool

10.7.2.95

10.7.2.96

10.7.2.91

Effective pool

10.7.2.92

10.7.2.93

10.7.2.95

Traffic compression across zones

Add header

- accept-encoding: gzip

Unzip content Zip content

Add header:

- content-encoding: gzip

Service zone A Service zone Brequest

response

Compression applied at perimeter

Also pre-emptive compression on

requests with payloads

Thank you