decomposing applications for deployability and scalability (cfopentour india)
DESCRIPTION
30 minute version of the talk given in Bangalore and India in Sept 2012TRANSCRIPT
Decomposing applications for deployability and scalability
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected]://plainoldobjects.com/
1
Presentation goal
2
How decomposing applications improves
deployability and scalability
vmc push About-Chris
3
Developer Advocate for CloudFoundry.com
Signup at http://cloudfoundry.com cfopentour2012
4
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?
Let’s imagine you are building an e-commerce application
5
Tomcat
Traditional web application architecture
6
Browser
WAR
MySQL Database
ShippingService
AccountingService
InventoryService
StoreFrontUI
developtestdeploy
Simple to
Apache
scale
But there are problems with a monolithic architecture
7
Users expect a rich, dynamic and interactive experience
8
Java Web Application
Browser
HTTP Request
HTML/Javascript
Old style UI architecture isn’t good enough
Real-time web ≅ NodeJS
Obstacle to frequent deployments§Need to redeploy everything to change one component§Interrupts long running background (e.g. Quartz) jobs§Increases risk of failure
Fear of change
§Updates will happen less often§e.g. Makes A/B testing UI really difficult
9
Overloads your IDE and container
10Slows down development
11
Shipping team
Accounting team
Engineering
Obstacle to scaling development
E-commerce application
12
WAR
Shipping
Accounting
InventoryService
StoreFrontUI
Shipping team
Accounting team
Inventory team
UI team
Obstacle to scaling development
13
Lots of coordination and communication required
Obstacle to scaling development
14
Requires long-term commitment to a technology
stack
15
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?
16
The scale cube
17
X axis - horizontal duplication
Z axis
- data
parti
tionin
g
Y axis - functionaldecomposition
Scale
by sp
litting
simila
r
thing
s
Scale by splitting different things
Y-axis scaling - application level
18
WAR
ShippingService
AccountingService
InventoryService
StoreFrontUI
Y-axis scaling - application level
19
Store front web application
shipping web application
inventory web application
Apply X axis cloning and/or Z axis partitioning to each service
ShippingService
AccountingService
InventoryServiceStoreFrontUI
accounting web application
Partitioning strategies§Partition by verb, e.g. shipping service§Partition by noun, e.g. inventory service§Single Responsibility Principle§Unix utilities - do one focussed thing well
20
Something of an art
Real world examples
21
http://highscalability.com/amazon-architecture
Between 100-150 services are accessed to build a page.
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
There are drawbacks
22
Complexity
23
See Steve Yegge’s Google Platforms Rant re Amazon.com
Multiple databases =
Transaction management challenges
24
When to use it?
25
In the beginning: •You don’t need it •It will slow you down
Later on:•You need it•Refactoring is painful
But there are many benefits§Scales development: develop, deploy and scale each service independently§Update UI independently§Improves fault isolation§Eliminates long-term commitment to a single technology stack
26
Modular, polyglot, multi-framework applications
Two levels of architecture
27
System-level
ServicesInter-service glue: interfaces and communication mechanismsSlow changing
Service-level
Internal architecture of each serviceEach service could use a different technology stackPick the best tool for the jobRapidly evolving
If services are small...§Regularly rewrite using a better technology stack§Adapt system to changing requirements and better technology without a total rewrite§Pick the best developers rather than best <pick a language> developers ⇒ polyglot culture
28
Fred George “Developer Anarchy”
The human body as a system
29
50 to 70 billion of your cells die each day
30
Yet you (the system) remain you
31
Can we build software systems with these characteristics?
32
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
33
Agenda§The (sometimes evil) monolith§Decomposing applications into services§How do services communicate?
Inter-service communication options§Synchronous HTTP ⇔ asynchronous AMQP
§Formats: JSON, XML, Protocol Buffers, Thrift, ...
§Even via the database
34
Asynchronous is preferred
JSON is fashionable but binary format is more efficient
StoreFrontUI
wgrus-store.war
AccountingService
wgrus-billing.war
InventoryService
wgrus-inventory.war
ShippingService
wgrus-shipping.war
MySQL
35
RabbitMQ(Message Broker)
Asynchronous message-based communication
Benefits§Decouples caller from server§Caller unaware of server’s coordinates (URL)§Message broker buffers message when server is down/slow
36
Drawbacks§Additional complexity of message broker §RPC using messaging is more complex
37
Writing code that calls services
38
The need for parallelism
39
Service A
Service B
Service C
Service D
b = serviceB()
c = serviceC()
d = serviceD(b, c)
Call in parallel
Java Futures are a great concurrency abstraction
40
http://en.wikipedia.org/wiki/Futures_and_promises
Akka’s composable futures are even better
41
Using Akka futures
42
def callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...
val future = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
val result = Await.result(future, 1 second)
http://doc.akka.io/docs/akka/2.0.1/scala/futures.html
Two calls execute in parallel
And then invokes D
Get the result of D
Spring Integration§Implements EAI patterns§Provides the building blocks for a pipes and filters architecture§Enables development of application components that are•loosely coupled•insulated from messaging infrastructure
§Messaging defined declaratively
43
Handling failure
44
Service A Service B
Errors happen in distributed systems
About Netflix
45
> 1B API calls/day1 API call ⇒ average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Use timeouts and retries
46
Never wait forever
Errors can be transient ⇒ retry
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Service A
Service B
Use per-dependency bounded thread pool
47http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
Runnable 1
Runnable 2
Runnable ...
bounded queue
Task 1
Task 2
Task ...
bounded thread pool
Limits number of outstanding requests
Fails fast if service is slow or down
Use a circuit breakerHigh error rate ⇒ stop calling temporarily
Down ⇒ wait for it to come back up
Slow ⇒ gives it a chance to recover
48http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
On failureReturn cached data
Return default data
Fail fast
49http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
AvoidFailing
Summary
50
Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
51
Apply the scale cube
52
X axis - horizontal duplicationZ ax
is - d
ata pa
rtitio
ning
Y axis - functionaldecomposition
§Modular, polyglot, and scalable applications§Services developed, deployed and scaled independently
Questions?
@crichardson [email protected]://slideshare.net/chris.e.richardson/
www.cloudfoundry.com promo code: cfopentour2012