decompose that war! architecting for adaptability, scalability, and deployability (jmaghreb,...
DESCRIPTION
It’s no longer acceptable to develop large, monolithic, single-language, single-framework Web applications. In this session, you will learn how to use the scale cube to decompose your monolithic Web application into a set of narrowly focused, independently deployable services. The presentation discusses how a modular architecture makes it easy to adopt newer and better languages and technologies. You will learn about the various communication mechanisms—synchronous and asynchronous—that these services can use.TRANSCRIPT
@crichardson
Chris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com @crichardson [email protected] http://plainoldobjects.com
Decompose That WAR! Architecting for Adaptability, Scalability, and Deployability
@crichardson
Presentation goalHow decomposing applications
improves deployability and scalability
and simplifies the adoption of new
technologies
@crichardson
(About Chris)
@crichardson
About Chris()
@crichardson
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
@crichardson
About Chris
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
API gateway design
Refactoring the monolith
@crichardson
Let’s imagine you are building an online store
@crichardson
Tomcat
Traditional application architecture
Browser
WAR/EAR
MySQL Database
Review Service
Product InfoService
Recommendation Service
StoreFrontUI
developtest
deploy
Simple to
Apache/Load
balancer
scale
Spring MVC
SpringHibernate
Order Service
@crichardson
But there are problems with a monolithic architecture
@crichardson
Users expect a rich, dynamic and interactive
experience
@crichardson
Browser
WAR
StoreFrontUI
Model
View Controller
Presentation layer evolution....
HTML / HTTP
Old style UI architecture isn’t good enough
@crichardson
Browser - desktop and mobile Web application
RESTfulEndpoints
Model
View Controller
...Presentation layer evolution
JSON-REST
HTML 5 - JavaScript
Native mobile clientIoS or Android
Event publisher
Events
Static content
No elaborate, server-side web framework required
@crichardson
Intimidates developers
@crichardson
Obstacle to frequent deployments
Need to redeploy everything to change one component
Interrupts long running background (e.g. Quartz) jobs
Increases risk of failure
Fear of change
Updates will happen less often - really long QA cycles
e.g. Makes A/B testing UI really difficult
Eggs in one basket
@crichardson
Overloads your IDE and container
Slows down development
@crichardson
Shipping team
Accounting Engineering
Obstacle to scaling development
E-commerce application
@crichardson
WAR
Review service
Product Info service
Recommendation service
StoreFrontUI
Reviews team
Product Info team
Recommendations team
UI team
Obstacle to scaling development
Order serviceOrders team
@crichardson
Lots of coordination and communication required
Obstacle to scaling development
I want to update the UI
But the backend is not working
yet!
@crichardson
Requires long-term commitment to a technology stack
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
API gateway design
Refactoring the monolith
@crichardson
@crichardson
The scale cube
X axis - horizontal duplication
Z axis
- data
partit
ioning
Y axis - functional
decomposition
Scale b
y split
ting s
imilar
thing
s
Scale by splitting
different things
@crichardson
Y-axis scaling - application level
WAR
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
@crichardson
Y-axis scaling - application level
Store front application
reviews application
recommendations application
Apply X axis cloning and/or Z axis partitioning to each service
product info application
ReviewService
Product InfoService
RecommendationService
StoreFrontUI
OrderService
orders application
@crichardson
Partitioning strategies...
Partition by noun, e.g. product info service
Partition by verb, e.g. shipping service
Single Responsibility Principle
Unix utilities - do one focussed thing well
@crichardson
Partitioning strategies
Too few
Drawbacks of the monolithic architecture
Too many - a.k.a. Nano-service anti-pattern
Runtime overhead
Potential risk of excessive network hops
Potentially difficult to understand system
Something of an art
@crichardson
Inter-service communication options
Synchronous HTTP ⇔ asynchronous AMQP
Formats: JSON, XML, Protocol Buffers, Thrift, ...
Asynchronous is preferred but REST is popular too
JSON is fashionable but binary format is more efficient
@crichardson
Example micro-serviceclass RegistrationSmsServlet extends RegistrationSmsScalatraStack {
post("/") { val phoneNumber = request.getParameter("From") val registrationUrl = System.getenv("REGISTRATION_URL") + "?phoneNumber=" + encodeForUrl(phoneNumber) <Response> <Sms>To complete registration please go to {registrationUrl} </Sms> </Response> } }
For more on micro-services see http://oredev.org/2012/sessions/micro-
service-architecture
@crichardson
Real world examples
http://highscalability.com/amazon-architecture
http://techblog.netflix.com/
http://www.addsimplicity.com/downloads/eBaySDForum2006-11-29.pdf
http://queue.acm.org/detail.cfm?id=1394128
@crichardson
There are drawbacks
@crichardson
Complexity
See Steve Yegge’s Google Platforms Rant re Amazon.com
@crichardson
Multiple databases &
Transaction management
@crichardson
Implementing features that span multiple services
@crichardson
When to use it?In the beginning: •You don’t need it •It will slow you down
Later on:•You need it•Refactoring is painful
@crichardson
But there are many benefits
@crichardson
Smaller, simpler apps
Easier to understand and develop
Reduced startup time - important for GAE
Less jar/classpath hell
Who needs OSGI?
@crichardson
Scales development: develop, deploy and scale each service independently
@crichardson
Improves fault isolation
@crichardson
Eliminates long-term commitment to a single technology stack
Modular, polyglot, multi-framework applications
@crichardson
Two levels of architectureSystem-level
ServicesInter-service glue: interfaces and communication mechanisms
Slow changing
Service-level
Internal architecture of each serviceEach service could use a different technology stack
Pick the best tool for the jobRapidly evolving
@crichardson
Easily try other technologies
... and fail safely
@crichardson
The human body as a system
@crichardson
50 to 70 billion of your cells die each day
@crichardson
Yet you (the system) remain you
@crichardson
Can we build software systems with these characteristics?
http://dreamsongs.com/Files/WhitherSoftware.pdf
http://dreamsongs.com/Files/DesignBeyondHumanAbilitiesSimp.pdf
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
API gateway design
Refactoring the monolith
@crichardson
Directly connecting the front-end to the backend
Model
View Controller Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
Model
View Controller
Browser/Native App
Traditional web application
Chatty API
Web unfriendly protocols
@crichardson
Use an API gateway
Model
View Controller Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
APIGateway
Model
View Controller
Browser/Native App
Single entry point
Client specific APIs
Protocol translation
Traditional web application
@crichardson
Optimized client-specific APIs
DesktopBrowser
MobileApp
NodeJS
APIGateway
RESTproxy
Event publishing
Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
getProductInfo()getRecomm...()getReviews()
getProductDetails()
@crichardson
Netflix API Gateway
http://techblog.netflix.com/2013/01/optimizing-netflix-api.html
Device specific end points
@crichardson
Developing an API gateway
Java EE web technologies
Netty-based technology stack
Non-Java options: e.g. Node.JS
@crichardson
API gateway design issues
@crichardson
The need for parallelism
ProductDetails
API
Product Info
Recommendations
Reviews
getProductDetails()
getRecomendations()
getReviews()
Call in parallel
get ProductDetails
@crichardson
Futures are a great concurrency abstraction
An object that will contain the result of a concurrent computationhttp://en.wikipedia.org/wiki/Futures_and_promises
Future<Integer> result = executorService.submit(new Callable<Integer>() {... });
Java has basic futures. We can do much better....
@crichardson
Better: Futures with callbacks
val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }
Guava ListenableFutures, Java 8 CompletableFuture, Scala Futures
@crichardson
Even better: Composable Futuresval f1 = Future { ... ; 1 }val f2 = Future { ... ; 2 }
val f4 = f2.map(_ * 2)assertEquals(4, Await.result(f4, 1 second))
val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))
def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))
Scala Futures
Transforms Future
Combines two futures
Transforms list of futures to a future containing a list
@crichardson
Composing concurrent requests using Scala Futures
class ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }
}
Asynchronously creates a Future containing result
@crichardson
Handling partial failures
ProductDetails
Controller
Product Info
Recommendations
Reviews
getProductDetails()
getRecomendations()
getReviews()
get ProductDetails X
@crichardson
About Netflix> 1B API calls/day
1 API call ⇒ average 6 service calls
Fault tolerance is essential
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
@crichardson
How to run out of threads
Tomcat
Execute thread pool
HTTP Request
Thread 1
Thread 2
Thread 3
Thread n
API Gateway
Code
RecommendationService
If service is down then thread will
be blocked
XXXXX
Eventually all threads will be blocked
@crichardson
Their approachNetwork timeouts
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
https://github.com/Netflix/Hystrix
@crichardson
Agenda
The (sometimes evil) monolith
Decomposing applications into services
API gateway design
Refactoring the monolith
@crichardson
How do you decompose your big, scary monolithic
application?
@crichardson
Strategy #1: Stop digging
@crichardson
New functionality = service
Monolith ServiceAnti-corruption layer
Glue code
Pristine
@crichardson
Sounds simple but...
Dependencies between monolith and service
e.g. Common entities
Building the anti-corruption layer can be challenging
Must replicate data between systems
...
@crichardson
Strategy #2: Split front-end & backend
@crichardson
Split front-end & backend
MonolithPresentation
layer
Backend
Front-end
Backend
Independently deployable
@crichardson
Strategy #3: Decompose front-end
@crichardson
Decompose front-endFront-end
Catalog Checkout Account Orders
Catalog Checkout Account Orders
/catalog /checkout /account /orders
@crichardson
Issues and challenges
Server-side Session state:
Need to use a separate session state server
...
@crichardson
Strategy #4: extract services
@crichardson
Module ⇒ service ...
WAR
Module
@crichardson
... Module ⇒ service
WAR ServiceAnti-corruption layer
@crichardson
What to extract?
Have the ideal partitioned architecture in mind:
Partitioned by verb or noun
Start with troublesome modules:
Frequently updated
Stateful components that prevent clustering
Conflicting resource requirements
@crichardson
Service
Domain model 2
Monolith
Untangling dependencies
API A API B API C
Domain model 1
A C
B
X
Z
Y
Trouble!
@crichardson
Useful idea: Bounded contextDifferent services have a different view of a domain object, e.g.
User Management = complex view of user
Rest of application: User = PK + ACL + Name
Different services can have a different domain model
Services exchange messages to synchronize data
@crichardson
Customer management
Untangling orders and customers
Order management
Order Service
placeOrder()
Customer Service
availableCredit()updateCustomer()
Customer
creditLimit...
has ordersbelongs toOrder
total
Invariant:sum(order.total) <= creditLimit
available credit= creditLimit - sum(order.total)Trouble!
@crichardson
Customer management
Replicating the credit limit
Order management
Order Service
placeOrder()
Customer
creditLimit...
Order
total
Customer’
creditLimit
CreditLimitChangedEvent
sum(order.total) <= creditLimit
Customer Service
updateCustomer()
Simplified
@crichardson
Customer management
Maintaining the openOrderTotal
Order management
Order Service
placeOrder()
Customer Service
availableCredit()
Customer
creditLimit
...
Order
customerIdtotal
= creditLimit - openOrderTotal
OpenOrderTotalUpdated
openOrderTotal
@crichardson
Refactoring a monolith is not easy
BUT
the alternative is far worse
@crichardson
Summary
@crichardson
Monolithic applications are simple to develop and deploy
BUT have significant drawbacks
@crichardson
Apply the scale cube
Modular, polyglot, and scalable applications
Services developed, deployed and scaled independently
@crichardson
Use a modular, polyglot architecture
Model
View Controller Product Infoservice
RecommendationService
Reviewservice
REST
REST
AMQP
APIGateway
Model
View Controller
Traditional web application