decomposing applications for scalability and deployability (devnexus 2013)
DESCRIPTION
Today, there are several trends that are forcing application architectures to evolve. Users expect a rich, interactive and dynamic user experience on a wide variety of clients including mobile devices. Applications must be highly scalable, highly available and run on cloud environments. Organizations often want to frequently roll out updates, even multiple times a day. Consequently, it’s no longer adequate to develop simple, monolithic web applications that serve up HTML to desktop browsers. In this talk we describe the limitations of a monolithic architecture. You will learn how to use the scale cube to decompose your application into a set of narrowly focused, independently deployable back-end services and an HTML 5 client. We will also discuss the role of technologies such as NodeJS and AMQP brokers. You will learn how a modern PaaS such as Cloud Foundry simplifies the development and deployment of this style of application.TRANSCRIPT
@crichardson
Decomposing applications for deployability and scalability
Chris Richardson
Author of POJOs in Action
Founder of the original CloudFoundry.com
@[email protected] http://plainoldobjects.com
@crichardson
Presentation goalHow decomposing applications
improves deployability and scalability
and
How Cloud Foundry helps
@crichardson
About Chris
@crichardson
(About Chris)
@crichardson
About Chris()
@crichardson
About Chris
@crichardson
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
@crichardson
vmc push About-Chris
Developer Advocate
Signup at http://cloudfoundry.com
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson
Let’s imagine you are building an e-commerce application
@crichardson
Tomcat
Traditional web application architecture
Browser
WAR
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFrontUI
developtest
deploy
Simple to
Apache
scale
@crichardson
But there are problems with a monolithic architecture
@crichardson
Users expect a rich, dynamic and interactive experience
Java Web ApplicationBrowser
HTTP Request
HTML/Javascript
Old style UI architecture isn’t good enough
Real-time web ≅ NodeJS
@crichardson
Intimidates developers
@crichardson
Obstacle to frequent deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often
e.g. Makes A/B testing UI really difficult
@crichardson
Overloads your IDE and container
Slows down development
@crichardson
Shipping team
Accounting Engineering
Obstacle to scaling development
E-commerce application
@crichardson
WAR
Shipping
Accounting
InventoryService
StoreFrontUI
Shipping team
Accounting
Inventory team
UI team
Obstacle to scaling development
@crichardson
Lots of coordination and communication required
Obstacle to scaling development
I want to update the UI
But the backend is not working
yet!
@crichardson
Requires long-term commitment to a technology stack
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson
@crichardson
The scale cube
X axis - horizontal duplication
Z ax
is - d
ata pa
rtition
ing
Y axis - functional
decomposition
Scale
by sp
litting
similar
thing
s
Scale by splitting
different things
@crichardson
Y-axis scaling - application level
WAR
ShippingService
AccountingService
InventoryService
StoreFrontUI
@crichardson
Y-axis scaling - application level
Store front web application
shipping web application
inventory web application
Apply X axis cloning and/or Z axis partitioning to each service
AccountingService
StoreFrontUI
accounting web application
ShippingService
InventoryService
@crichardson
Partitioning strategies...
Partition by verb, e.g. shipping service
Partition by noun, e.g. inventory service
Single Responsibility Principle
Unix utilities - do one focussed thing well
@crichardson
...Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art
@crichardson
Example micro-servicerequire 'sinatra'
post '/' do phone_number = params[:From] registration_url = "#{ENV['REGISTRATION_URL']}?phoneNumber=#{URI.encode(phone_number, "+")}" <<-eof <Response> <Sms>To complete registration please go to #{registration_url}</Sms> </Response> eofend
For more on micro-services see @fgeorge52
@crichardson
Service deployment options
VM
Linux Container/LXC
JVM
JAR/WAR/OSGI bundle/...
Isolation, manageability
Density/efficiency
@crichardson
Real world examples
http://highscalability.com/amazon-architecture
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
@crichardson
There are drawbacks
@crichardson
Complexity
See Steve Yegge’s Google Platforms Rant re Amazon.com
@crichardson
Multiple databases &
Transaction management
@crichardson
Implementing features that span multiple services
@crichardson
When to use it?In the beginning: •You don’t need it •It will slow you down
Later on:•You need it•Refactoring is painful
@crichardson
But there are many benefitsScales development: develop, deploy and scale each service independently
Update UI independently
Improves fault isolation
Eliminates long-term commitment to a single technology stack
Modular, polyglot, multi-framework applications
@crichardson
Two levels of architectureSystem-level
ServicesInter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each serviceEach service could use a different technology stack
Pick the best tool for the jobRapidly evolving
@crichardson
If services are small...
Regularly rewrite using a better technology stack
Adapt system to changing requirements and better technology without a total rewrite
Pick the best developers rather than best <pick a language> developers ⇒ polyglot culture
@crichardson
The human body as a system
@crichardson
50 to 70 billion of your cells die each day
@crichardson
Yet you (the system) remain you
@crichardson
Can we build software systems with these characteristics?
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson
Inter-service communication options
Synchronous HTTP ⇔ asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Even via the database
Asynchronous is preferredJSON is fashionable but binary format
is more efficient
@crichardson
StoreFrontUI
wgrus-store.war
AccountingService
wgrus-billing.war
InventoryService
wgrus-inventory.war
ShippingService
wgrus-shipping.war
MySQL
RabbitMQ(Message Broker)
Asynchronous message-based communication
@crichardson
Benefits
Decouples caller from server
Caller unaware of server’s coordinates (URL)
Message broker buffers message when server is down/slow
Supports a variety of communication patterns, e.g. point-to-point, pub-sub, ...
@crichardson
Drawbacks
Additional complexity of message broker
Request/reply-style communication is more complex
@crichardson
Writing code that calls services
@crichardson
The need for parallelism
Service A
Service B
Service C
Service D
b = serviceB()
c = serviceC()
d = serviceD(b, c)
Call in parallel
@crichardson
Java Futures are a great concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
@crichardson
Using Java Futurespublic class Client {
private ExecutorService executorService; private RemoteServiceProxy remoteServiceProxy;
public void doSomething() throws ... { Future<Integer> result =
executorService.submit(new Callable<Integer>() { @Override public Integer call() throws Exception { return remoteServiceProxy.invokeRemoteService(); } });
/// Do other things
int r = result.get(500, TimeUnit.MILLISECONDS);
System.out.println(r);
}}
Eventually contains result
When needed wait for result
@crichardson
Guava ListenableFutures = better
Scala’s composable Futures = really good
Java 8 CompletableFuture = great
Better future implementations
@crichardson
Composable Futures
val f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
Transforms Future
Combines two futures
@crichardson
Using Scala futuresdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...
val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
Two calls execute in parallel
And then invokes D
Get the result of D
@crichardson
Spring IntegrationProvides the building blocks for a pipes and filters architecture
Enables development of application components that are
loosely coupled
insulated from messaging infrastructure
Messaging defined declaratively
@crichardson
Handling partial failures
Service A Service B
Down?Slow?
Down?Slow?
@crichardson
About Netflix> 1B API calls/day
1 API call ⇒ average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
@crichardson
How to run out of threads
Tomcat
Execute thread pool
HTTP Request
Thread 1
Thread 2
Thread 3
Thread n
Service A Service B
If service B is down then thread
will be blocked
XXXXX
Eventually all threads will be blocked
@crichardson
Their approachNetwork timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
https://github.com/Netflix/Hystrix
See my talk tomorrow for more details
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson
Browser
WAR
StoreFrontUI
Model
View Controller
Presentation layer evolution....
HTML / HTTP
@crichardson
Browser Web application
RESTfulEndpointsModel
View Controller
...Presentation layer evolution
JSON-REST
HTML 5 - JavaScript
No elaborate, server-side web framework required
Event publisher
Events
Static content
@crichardson
How to publish events to browser?
@crichardson
NodeJS is the fashionable technology
@crichardson
Why NodeJS?Familiar Javascript
High-performance, scalable event-driven, non-blocking I/O model
Compact runtime
Over 17,000 modules developed by the community
Simple event publishing using socket.io or SockJS
@crichardson
Why not NodeJS?
a.k.a. callback hell
@crichardson
A modern web applicationBrowser
Service 1
Service 2
Service 3
HTML 5/JavaScriptSocket.io
client
Events
RESTful WS
Server application
Socket.ioserver
Node JS
@crichardson
NodeJS - using RESTful WS and AMQP
Node JS
Service
RabbitMQ Service
REST
AMQP AMQP
RESTRequests
Events
socket.io
@crichardson
Socket.io server-sidevar express = require('express') , http = require('http') , amqp = require(‘amqp’) ....;
server.listen(8081);...var amqpCon = amqp.createConnection(...);
io.sockets.on('connection', function (socket) { function amqpMessageHandler(message, headers, deliveryInfo) { var m = JSON.parse(message.data.toString()); socket.emit(‘tick’, m); }; amqpCon.queue(“”, {}, function(queue) { queue.bind(“myExchange”, “”); queue.subscribe(amqpMessageHandler); });});
Handle socket.io
connection
Subscribe to AMQP queue
Republish as socket.io
event
https://github.com/cer/nodejs-clock
@crichardson
Socket.io - client side
var socket = io.connect(location.hostname);
function ClockModel() { self.ticker = ko.observable(1); socket.on('tick', function (data) { self.ticker(data); });};
ko.applyBindings(new ClockModel());
<html><body>
The event is <span data-bind="text: ticker"></span>
<script src="/socket.io/socket.io.js"></script><script src="/knockout-2.0.0.js"></script><script src="/clock.js"></script>
</body></html>
clock.js
Connect to socket.io
Subscribe to tick event
Bind to model
Update model
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
How do services communicate?
Presentation layer design
How Cloud Foundry helps
@crichardson
Tomcat
Original architecture
Browser
WAR
MySQL DatabaseApache
Shipping
AccountingService
InventoryService
StoreFrontUI
@crichardson
Modern architecture
RabbitMQ
NodeJS
Inventory Service
Shipping Service
Billing Service Redis Inventory Database
Mongo Order Database
Standalone“headless”
Spring/Java applications
Spring/Scala web application
MySQL Customer Database
Desktop Browser Native Mobile application
HTML5 mobile application
StoreUI StoreUI StoreUI
StoreUI
JavascriptAsynchronous, scalable
communication
How
do w
e dev
elop,
test
and d
eplo
y th
is?
@crichardson
Traditional tools: monolithic applications
@crichardson
Developing modular apps is more difficult
Many more moving parts to manage
Platform services: SQL, NoSQL, RabbitMQ
Application services: your code
Who is going to setup the environments:
the developer sandbox?
...
But Cloud Foundry helps...
@crichardson
Applica'on Service Interface
Data Services
Other Services
Msg Services
Easy polyglot application deployment and service provisioning
vFabric Postgres
vFabric RabbitMQTM
Additional partners services …
OSS community
Private Clouds
PublicClouds
MicroClouds
@crichardson
Creating a platform service instance
$ vmc create-service mysql mysql1Creating Service: OK
$ vmc services......
=========== Provisioned Services ============
+-------------+---------+| Name | Service |+-------------+---------+| mysql1 | mysql |+-------------+---------+
@crichardson
Multi-application manifest - part 1
---
applications:
inventory/target:
name: inventory
url: cer-inventory.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services:
si-rabbit:
type: :rabbitmq
si-mongo:
type: :mongodb
si-redis:
type: :redis
Path to application
Required platform services
@crichardson
Multi-application manifest - part 2 store/target:
name: store
url: cer-store.chrisr.cloudfoundry.me
framework:
name: spring
info:
mem: 512M
description: Java SpringSource Spring Application
exec:
mem: 512M
instances: 1
services:
si-mongo:
type: :mongodb
si-rabbit:
type: :rabbitmq
Path to application
Required platform services
@crichardson
One command to create platform services and deploy application
$ vmc push
Would you like to deploy from the current directory? [Yn]:
Pushing application 'inventory'...
Creating Application: OK
Creating Service [si-rabbit]: OK
Binding Service [si-rabbit]: OK
Creating Service [si-mongo]: OK
Binding Service [si-mongo]: OK
Creating Service [si-redis]: OK
Binding Service [si-redis]: OK
Uploading Application:
Checking for available resources: OK
Processing resources: OK
Packing application: OK
Uploading (12K): OK
Push Status: OK
Staging Application 'inventory': OK
Starting Application 'inventory': OK
Pushing application 'store'...
vmc push:•Reads the manifest file•Creates the required platform services•Deploys all the applications
@crichardson
Micro Cloud Foundry: new developer sandbox
Open source Platform as a Service project
App Instances Services
10.04
A PaaS packaged as a VMware Virtual Machine
Use as a developer sandbox• Use the services from Junit integration tests
• Deploy your application for functional testing
• Remote debugging from STS
@crichardson
Using Caldecott to tunnel into your services
@crichardson
Caldecott = TCP over HTTP
Your computer
Caldecott gem
Cloud Foundry
Caldecott application Service
HTTP nativeprotocol
Port NNN
Service client
nativeprotocol
@crichardson
Using Caldecott…$ vmc tunnel1: mysql-135e02: mysql1
Which service to tunnel to?: 2Password: ********
Stopping Application: OKRedeploying tunnel application 'caldecott'.Uploading Application:
Checking for available resources: OK Packing application: OK
Uploading (1K): OK Push Status: OK
Binding Service [mysql1]: OKStaging Application: OK Starting Application: OK
Getting tunnel connection info: OK
Service connection info: username : uMe6Apgw00AhS password : pKcD76PcZR7GZ
name : d7cb8afb52f084f3d9bdc269e7d99ab50
Starting tunnel to mysql1 on port 10000.1: none2: mysql
Which client would you like to start?: 2
@crichardson
…Using CaldecottLaunching 'mysql --protocol=TCP --host=localhost --port=10000 --user=uMe6Apgw00AhS --password=pKcD76PcZR7GZ d7cb8afb52f084f3d9bdc269e7d99ab50'
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 10944342Server version: 5.1.54-rel12.5 Percona Server with XtraDB (GPL), Release 12.5,
Revision 188
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respectiveowners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
@crichardson
Running JUnit test with Caldecott
Configure your test code to use port + connection info
@crichardson
Summary
@crichardson
Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
@crichardson
Apply the scale cube
Modular, polyglot, and scalable applications
Services developed, deployed and scaled independently
@crichardson
Cloud Provider Interface
Applica'on Service Interface
Private Clouds
PublicClouds
MicroClouds
Data Services
Other Services
Msg Services
.js
Cloud Foundry helps