“try not. do, or do not. there is no try.” - yoda yoda finally admits he does not understand...

66
Architecting to be Cloud Native Guest lecture at Dino Konstantopoulos’ BU MET CS755 Cloud Computing class 17-April-2014 (7:00 – 9:00 PM EDT) HELLO my name is Bill Wilder Aligning your application’s architecture with the architecture of the cloud… FTW! But the cloud is a friendly place for non- native apps too!

Upload: reanna-boord

Post on 01-Apr-2015

253 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Architecting to be Cloud Native

Guest lecture at Dino Konstantopoulos’ BU MET CS755 Cloud Computing class 17-April-2014 (7:00 – 9:00 PM EDT)

                                        

HELLOmy name isBill Wilder

Aligning your application’s architecture with the architecture

of the cloud… FTW!But the cloud is a friendly place for

non-native apps too!

Page 2: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

My name is Bill Wilder

HELLO

my name is

Bill Wilder

[email protected]@codingoutloud

www.devpartners.com

Page 3: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Who is Bill Wilder?

www.devpartners.com

www.bostonazure.org

www.cloudarchitecturepatterns.com

Page 4: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

I will ass-u-me…

1. You know what “the cloud” is2. You have an inkling about Amazon Web Services and

Windows Azure cloud platforms3. You understand that such cloud platforms include

compute services [like hosted virtual machines (VMs), in both IaaS and PaaS modes], SQL and NoSQL database services, file storage services, messaging, DNS, management, etc.

4. You are interested in understanding cloud-native applications and why that’s better than deploying my old-school app to the cloud “as is”

Page 5: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Roadmap for rest of talk… …

1. Lightning-fast overview of Windows Azure2. Cover three specific patterns for building

cloud-native applications3. Mention some other patterns along the way

• Q&A during talk is okay (time permitting)• Q&A at end with any remaining time• Okay to reach out through email or twitter

?

Page 6: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Windows Azure Portal

General informationhttp://www.windowsazure.com

Management Portalhttp://manage.windowsazure.com

Page 7: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

“Bring Your Own” ____ as a Service

BYO UsersBYO

Applications

BYO Virtual Machines

PaaS

IaaS

SaaS

more

less

Responsibility &

Flexibility

NIST: http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf

Most productive

platforms for

Cloud-Native

Apps

Page 8: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

NIST Terminology

• SaaS = Software as a Service (BYO users)• PaaS = Plaform as a Service (BYO apps)• IaaS = Infrastructure as a Service (BYO VMs)

Simplicity

ComplexityFlexibility

RigidityPower?

Power?http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf

Page 9: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

So Architecting for the (Windows Azure, AWS, GAE, …) Cloud is Different…

WHY DID THEY (Microsoft, Amazon, Google, …) DO THIS TO US?

But Why?

Page 10: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Know the rules

“If I had asked people what they wanted, they would have said faster horses.”

- Henry Ford

Faster h

orses w

ould not have

addressed th

e horse m

anure problem

late 1800s..

150k horses in

NYC

x 20 lbs m

anure/day/h

orse

= 3 milli

on lbs o

f manure per d

ay

Page 11: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Know the rules

“If I had asked IT departments what they wanted, they would have said IaaS.”

- Henry Cloud

Page 12: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Cloud Platform Characteristics• Scaling – or “resource allocation” – is horizontal

– and ∞ (“illusion of infinite resources”)

• Resources are easily added or released– self-service portal or API; cloud scaling is automatable

• Pay only for currently allocated resources– costs are operational, granular, controllable, and transparent

• Optimized for cost-efficiency– cloud services are MT, hardware is commodity– MTTR over MTTF

• Rich, robust functionality is simply accessible– like an iceberg

Page 13: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Cloud-Native Application Characteristics

• Application architecture is aligned with the cloud platform architecture–uses the platform in the most natural way– lets the platform do the heavy lifting

Page 14: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

The term “cloud” is nebulous…

The definition of “Cloud” is nebulous…

Page 15: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling
Page 16: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

What is different about the cloud?

What's different about the cloud?^public

Page 17: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

1/9th above w

ater

TTM & Sleeping well= SOA

Page 18: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

MTBF MTTR

commodity hardware + multitenant services= cost-efficient cloud

failure is routine(so you better be good at

handling it)

Architectural Assumptions

Page 19: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Loosely Coupled &Eventually Consistent

Data & WorkflowArchitecture

Page 20: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

This bar is always open

*and*

has an APIPay by the Drink

$

Page 21: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

• Resource allocation (scaling) is:– Horizontal– Bi-directional– Automatable

The “illusion of infinite resources”

Resource Allocation

Page 22: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Integrated Surface Area

Page 23: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

www.pageofphotos.com• Simple idea, simple app• Two-tiers: web tier (one server) + database• What’s the problem?

• But… what’s WRONG with this architecture?

• Different ≠ WRONG. Use the right tool for the job. Some apps are simply not good fit for cloud.

?

Page 24: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

www.pageofphotos.com• Simple idea, simple app• Two-tiers: web tier (one server) + database• What can go wrong

• We’ll reexamine1. Scaling the web tier2. Scaling the service tier3. Scaling the data tier4. Handling failure5. Operational efficiency (scale the app, not the team!)

Page 25: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Horizontal Scaling Compute Pattern

pattern 1 of 3

Page 26: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

What’s the difference between performance

and scale??

Page 27: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Common Terminology:Scaling Up/Down Vertical ScalingScaling Out/In Horizontal “Scaling” But really is Horizontal Resource Allocation

• Architectural Decision– Big decision… hard to change

Scale Up (and Scale Down??)vs. Horizontal Resourcing

Page 28: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Vertical Scaling (“Scaling Up”)

.

Resources that can be “Scaled Up”• Memory: speed, amount • CPU: speed, number of CPUs• Disk: speed, size, multiple controllers• Bandwidth: higher capacity pipe• … and it sure is EASY

Downsides of Scaling Up• Hard Upper Limit• HIGH END HARDWARE HIGH END CO$T• Lower value than “commodity hardware”• May have no other choice (architectural)

Page 29: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Scaling Horizontally: Adding BoxesAutonomous nodes

for scalability(stateless web servers, shared

nothing DBs, your custom code in

QCW)

Autonomous nodes*and*

Homogeneous nodes for operational simplicity

*and*Anonymous nodes

don‘t get emotionally involved!

This is how the CLOUD works *and*This is how YOUR CLOUD-NATIVE APP WORKS

Page 30: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Load Balancer(Cloud Service)

Managed VMs(Cloud Service)

Example: Web Tier www.pageofphotos.com

Page 31: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

1. Auto-Scale • Bidirectional

2. Nodes can fail• Auto-Scale is only one cause• Handle shutdown signals• Stateless (“like a taxi”)

vs. Sticky Sessions• Stateless nodes

vs. Stateless apps• N+1 rule

vs. occasional downtime (UX)

Horizontal Scaling Considerations

Page 32: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

How many users does your cloud-native

application need before it needs to be able to

horizontally scale??

Page 33: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Queue-Centric Workflow Pattern

(QCW for short)

pattern 2 of 3

Page 34: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Extend www.pageofphotos.com example into Service Tier

• QCW enables applications where the UI and back-end services are Loosely Coupled

• (Compare to CQRS at end if there is interest)

Page 35: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW Example: User Uploads Photo www.pageofphotos.com

Web Server

Compute ServiceReliable Queue

Reliable Storage

Page 36: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW

WE NEED:• Compute (VM) resources to run our code

• Reliable Queue to communicate

• Durable/Persistent Storage

Page 37: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Where does Windows Azure fit?

Page 38: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW [on Windows Azure]

WE NEED:• Compute (VM) resources to run our code

Web Roles (IIS) and Worker Roles (w/o IIS)• Reliable Queue to communicate

Azure Storage Queues• Durable/Persistent Storage

Azure Storage Blobs & Tables; WASD

Page 39: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW on Azure: User Uploads a Photo

WebRole(IIS)

WorkerRoleAzure Queue

Azure Blob

UX implications: user does not wait for thumbnail(architecture!)

ww

w.p

ageo

fpho

tos.

com

push pull

Page 40: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW enables Responsive UX

• Response to interactive users is as fast as a work request can be persisted

• Time consuming work done asynchronously• Comparable total resource consumption,

arguably better subjective UX• UX challenge – how to express Async to users?

– Communicate Progress– Display Final results– Long Polling/Web Sockets (e.g., SignalR or Node.io)

Page 41: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW enables Scalable App

• Decoupled front/back provides insulation– Blocking is Bane of Scalability– Order processing partner doing maintenance– Twitter down– Email server unreachable– Internet connectivity interruption

• Loosely coupled, concern-independent scaling– (see next slide)– Get Scale Units right

–Key to optimizing operational CO$T$

Page 42: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

General Case: Many Roles, Many Queues

WebRole(IIS)

WorkerRole

WebRole(IIS)

WebRole

(Public)

WorkerRoleWorker

RoleWorker

Role Type 1

WorkerRoleWorker

RoleWorkerRoleWorker

Role Type 2

Queue Type 1

Queue Type 2

Queue Type 1

Queue Type 2

Queue Type 3

• Scaling best when Investment α Benefit• Optimize for CO$T EFFICIENCY

• Logical vs. Physical Architecture depends on current scale

WorkerRole

Type 2

WorkerRole

Type 2

WorkerRole

Type 2

WebRole

(Admin)

Page 43: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Reliable Queue & 2-step Delete

(IIS)WebRole

WorkerRole

var url = “http://pageofphotos.blob.core.windows.net/up/<guid>.png”;queue.AddMessage( new CloudQueueMessage( url ) );

var invisibilityWindow = TimeSpan.FromSeconds( 10 );CloudQueueMessage msg = queue.GetMessage( invisibilityWindow );

(… do some processing then …)queue.DeleteMessage( msg );

Queue

Page 44: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW requires Idempotent

• Perform idempotent operation more than once, end result same as if we did it once

• Example with Thumbnailing (easy case)• App-specific concerns dictate approaches

– Compensating action, Last write wins, etc.• PARTNERSHIP: division of responsibility

between cloud platform & app– Far cry from database transaction

Page 45: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW expects Poison Messages

• A Poison Message cannot be processed– Error condition for non-transient reason– Use dequeue count property

• Be proactive– Falling off the queue may kill your system

• Determine a Max Retry policy per queue– Delete, put on “bad” queue, alert human, …

Page 46: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

QCW requires “Plan for Failure”

• VM restarts will happen– Hardware failure, O/S patching, crash (bug)

• Bake in handling of restarts into our apps– Restarts are routine: system “just keeps working”– Idempotent support needed important– Event Sourcing (commonly seen with CQRS) may

help• Not an exception case! Expect it!• Consider N+1 Rule

Page 47: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Typical Site Any 1 Role Inst Overall System

Operating System Upgrade

Application Code Update

Scale Up, Down, or In

Hardware Failure

Software Failure (Bug)

Security Patch

What’s Up? Reliability as EMERGENT PROPERTY

Page 48: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

What about the DATA?

• You: Azure Web Roles and Azure Worker Roles– Taking user input, dispatching work, doing work– Follow a decoupled queue-in-the-middle pattern– Stateless compute nodes

• Cloud: “Hard Part”: persistent, scalable data– Azure Queue & Blob Services– Three copies of each byte– Blobs are geo-replicated– Busy Signal Pattern

Page 49: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Database Sharding Pattern

pattern 3 of 3

Page 50: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Database Sharding Pattern

Most Cloud Applications don’t care (much) about (very high) scale

But they do care about developer productivity and operational efficiency

pattern 3 of 3

Page 51: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

foo.com

Site

-to-

Site

Vir

tual

Net

wor

k

VNET in cloud, connected to on-prem

On-premdatabase

On-prem API

bar.com as Azure Cloud Service

TDS

(nati

ve S

QL

Serv

er

TCP-

base

d w

ire p

roto

col)

SOA

P /

REST

/ H

TTP

Azure Cloud

On-prem

Dev Team(Point-to-Site VPN from CoLo Router into Azure)

Off-site/Travel Dev Team(Point-to-Site VPN from laptop to Azure)

foo.com as Azure Web Site running CMS

dedicated MySQL Database to run CMS

bar.com

Global CDN

Public Internet

Blob Storag

e

Blob Stora

ge

Content Editing & Site Admin

Dev Team

Page 52: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Azure SQL Database (WASD)is SQL Server Except…

Common

SQL ServerSpecific(for now)

SQL DatabaseSpecific

“Just change the connection

string…”

• Full Text Search• Transparent Data

Encryption (TDE)• Many more…Limitations• You need to run it• Max VM size

Limitations• 500 GB size limit• Busy Signal PatternExtra Capabilities• Managed Service• Highly Available• Rental model• Premium (reserved)

http://msdn.microsoft.com/en-us/library/ff394115.aspxAdditional information on Differences:

Page 53: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

My database instance is limited to 500 GB.

∞ ∞ ∞Does that mean the

cloud doesn’t really offer the illusion of infinite

resources??

Page 54: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Pre-Cloud vs. Cloud-Native

Old-School vs.

Cloud-Native

Control Efficiency

Stable/Static Hardware Dynamic/∞ Resources

Fixed/CapEx Variable/OpEx

Vertical Scaling Horizontal Resourcing

Minimize MTBF Minimize MTTR

Data Storage = RDBMS Scenario-specific Storage

Manage Infrastructure Managed Infrastructure

arc

hit

ect

ura

l co

nce

rns

Page 55: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Pre-Cloud vs. Cloud-Native

Lessons:

beingCloud-Native

1:15,000 Efficiency

Auto-Scaling via API Dynamic/∞ Resources

Pay-As-You-Go Variable/OpEx

Stateless, Autonomous Horizontal Resourcing

N+1, Idempotent Minimize MTTR

SQL, NoSQL, Blob Scenario-specific Storage

VM, Storage, LB, DR Managed Infrastructure

Page 56: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Know the rules

“Know the rules well, so you can break them effectively.”

- Dalai Lama XIV

Page 57: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Integrated Surface Area

Page 58: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Cloud Architecture Patterns bookPrimer Chapters

1. Scalability2. Eventual Consistency3. Multitenancy and

Commodity Hardware4. Network Latency

Page 59: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Cloud Architecture Patterns book Pattern Chapters

1. Horizontally Scaling Compute Pattern2. Queue-Centric Workflow Pattern3. Auto-Scaling Pattern4. MapReduce Pattern5. Database Sharding Pattern6. Busy Signal Pattern7. Node Failure Pattern8. Colocate Pattern9. Valet Key Pattern10. CDN Pattern11. Multisite Deployment Pattern

Page 60: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Questions?Comments?

More information?

?

Page 61: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Business Card

Page 62: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

BostonAzure.org

• Boston Azure cloud user group• Focused on Microsoft’s Public Cloud Platform

• Monthly, 6:00-8:30 PM in Boston area– Food; wifi; free; great topics; growing community

• Follow on Twitter: @bostonazure • More info or to join our Meetup.com group:

http://www.bostonazure.org

Page 63: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

Looking for …• consulting help with Windows Azure Platform? • someone to bounce Azure or cloud questions off?• a speaker for your user group or

company technology event?Just Ask!

Bill Wilder@codingoutloudhttp://blog.codingoutloud.comcommunity inquiries: [email protected] inquiries: www.devpartners.com book: www.cloudarchitecturepatterns.com

Contact Me

Find this slide deck here

Page 64: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling
Page 65: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling

DONE

Page 66: “Try not. Do, or do not. There is no try.” - Yoda Yoda finally admits he does not understand exception handling