best practices building cloud scale apps with microservices

2417 Best Practices in Building Cloud-Scale Applications with Microservices

Rob Phippen – STSM, Smart Hybrid Connectivity ArchitectJim Zhang – STSM, Lead Architect, IBM App Connect

Please Note:

2

• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision.

• The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

• Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

3

Agenda

• Overview• Introducing Microservices

– what are they – and how do they relate to what has gone before?• Core Principles Microservices• Putting The Principles To Work

Introducing MicroservicesWhat are they – and how do they relate to what has gone before?

What is a Microservices architecture?

Monolithicapplication

Microservicesapplication

Silo

Microservice(component)



A microservice is a granular decoupled component within a broader application

AgilityScalabilityResilience

Simplistically, microservices architecture is about breaking down large silo applications into more manageable fully decoupled pieces

6

Microservices and SOA

• Both SOA and microservices deal with a system of services communicating over a network, but…

• The Focus of SOA is on reuse and discovery– This tends to align with a centrally funded model– SOA services tend to be ‘servants of many masters’– This means that a change to a SOA service may impact multiple consumers

• The focus of microservices is on breaking a potentially monolithic application down into smaller, more manageable components– With the objective of more flexible, decoupled, faster development– Challenges here relate to needing very good dev/ops, management views

and controls

7

What are the fundamental questions that drive a design?

Who owns it?Who creates and prioritises the requirements?

How long does it last?What is the runtime lifespan of its instances?

How often does it change?How frequently do its requirements change?

Microservices: Why now? (technical standpoint)• Ease/feasibility of distributing components

– Internet/intranet/network maturity

– RESTful API conventions (perceived simplicity…)

• Ease/simplicity of hosting

– Lightweight runtimes (node.js, WAS Liberty etc.)

– Simplified infrastructure

• Virtualisation (hypervisors), containerisation (Docker), infrastructure as a service (cloud infrastructure)

– Platform as a service

• Auto-scaling, SLA management, messaging, caching, build management.

• Agile Development Methods

– Agile, DevOps, TDD, etc

– Standardised code management (Github, …)

8

Encapsulation is key. Related logic and data should remain together, and which means drawing strong boundaries between microservices.

Microservicecomponent



Monolithic application

Microservices application

Silo logic

Silo data

Example operating system boundaries

What that might look like in IBM technology?(simplistic representation – NOT a reference architecture!)




WAS LibertyNode.js

Monolithic application


WAS ND

DB2

MongoDb

MongoDb

Cloudant

Node.js

IBM Bluemix(PaaS)

MQ Light,MessageHub


Microservices inter-communication


REST

/HTT

P Messaging

REST/HTTPMessa

ging



Lightweight protocols • REST (JSON/HTTP)• Messaging (e.g. Kafka)

Aim is complete decoupling, achieved by:• Messaging wherever possible• Service registry/discovery• Load balancing• Circuit breaker patterns

Heterogeneous on the inside, homogeneous on the outside

Freedom to choose runtimes, languages, datastores etc.• Wise to encourage preferred technologies. • Convergence often happens naturally.

Commonality is in the framework in terms of:• Interconnectivity• Scalability• Resilience

12


Microservice(Node.js)

Microservice(Java +

MongoDB)Microservice

(Go + CloudantDB)

Common services useful in building Microservices

Microservices simplify the creation and maintenance of applications by inheriting key non-functional characteristics from their environment. These common services are• agnostic to the runtime/language used

by each microservice• Available on the network – rather than

being built into a specific runtime

Bluemix provides a very rich collection of these capabilties Microservices application



MicroservicecomponentMicroservice

componentMicroservicecomponentMicroservice

component

Framework provides• Scaling• Availability• Caching• Messaging• Connectivity• …and more

13

Enterprise Boundary

SoESoE

SoRSoRSoR

An evolving integration architecture

Integration Hub

Integration HubConnector Connector

Exposure Gateway (internal)

Exposure Gateway (external)

Connector Connector“Hub and Spoke” integration

“Enterprise Service Bus”

“API Gateway”

Systems of record (SoR)• Older technology• Harder to change• Challenging integration points

Systems of engagement (SoE)• Modern languages/runtimes• Agile updates• Simple modern connectivity

Enterprise Boundary

SoESoE

SoRSoRSoR

An evolving integration architecture

Integration Hub

Integration HubConnector Connector



Connector Connector“Hub and Spoke” integration

“Enterprise Service Bus”

“API Gateway”

Event-drivenLargely asynchronousPossibly bi-directionalThroughput based metrics

Where do microservices fit in a complex integration landscape

16

Syst

ems

of R

ecor

d(In

tern

al c

ore

appl

icat

ions

)

Integration Hub

Integration HubAdapter Adapter

Enga

gem

ent

App

licat

ion

s

Mic

rose

rvic

eap

plic

atio

ns

SaaS

App

licat

ions

(ext

erna

l)

Adapter

Externally Exposed Services/APIs



Bus

ines

s Pa

rtne

rs

17

A few things to read…

http://12factor.net/ http://bit.ly/ibmredmicrohttp://bit.ly/microsvc http://bit.ly/oreillymicro

Core Principles for MicroservicesWhat’s the ideal?

18

19

Engineering Principles

• Building them– Loosely Coupled– Horizontal Scaling– Eventually Consistent

• Deploying them– DevOps & CI– Test automation– Auto-scaling

• Operating them– Logging and monitoring– Self-recovering

20

Organizational/Cultural Principles

• Small teams– Decisions can be made timely within the team– Minimize external communications required for decision-making– Rule of thumb: two-pizza size

• Agile all the way– Don’t end up being a “waterscrumfall” organization– Each microservice team truly owns that service, and responsible for the

entire lifecycle of it: build, deploy, operate, change, update, etc.

Putting the principles to work

21

IBM App Connect

22

Sugar CRM

workday

Salesforce.com

MarketoHubspot

Insightly

Island Pacific SAP ORACLEManhattan

Temenos

22

“App awareness” allows you to connect your apps in minutes

Automation of manual tasks – notifications, events & updates

Connect your applications wherever they are… cloud or local

Keep customer and other data in sync between multiple apps

Build new connections quickly – tooling for your tech support team to connect to the apps you care about in hours

• applications shown are for discussion purposes only no claim is made of inclusion in product at launch

App Connect Agenda

23

23

Act on simple events• LoB professional using events to

automate daily tasks and ensure timely reaction to changes

• Trigger campaigns in a marketing automation system or ensure that new client appears everywhere

• One record at a time

Data Sync• Operations or administrator

ensuring that systems of record are in sync ( eg SFDC, SAP)

• Supporting changes happening in one or both systems

• Data models, selection and mapping of data important

• Typically on a schedule

Data Copy• Move a table of data from a

CRM or Marketing platform to a spread sheet or database

• Pull a segment list from CRM

• Data movement in one direction – insert into existing data structure or create new copy

“When a new contact appears in App A then update my App B address book..”

“I need a copy of all the new sales today dumped into a spreadsheet for my managers sales report”

“Keep the customer deals info in Salesforce.com in sync with the core SAP system…”

SF SFGS SAPHS

Try at the Lab Under development

App Connect Architecture

24Bluemix

node.js buildpacks

Dashboard & Tools

UI Views

User A&A

Flow authoring & Mgmt

Backend discovery

API Gateway

Flow engine(StrongLoop-based)

Connectors(Loopback-based)

services

Health monitor

Cloudant

Message Hub

Secure Gateway

Cloud Marketplace (SCX) Blue ID Success360

DevOps (Jenkins + Docker + mocha/chai/sinon + Selenium)

QRadar

Selenium Grid (SauceLabs.com)

ELK Digital Marketing

25

1. Don’t Start by Building Microservices

• It’s difficult to build a GOOD microservice– Loose coupling: minimize knowledge one service has of another– High cohesion: ALL the logic related to a behavior are enclosed in one

service• Start by building monolith• Until the bounded context for the microservices are properly identified

and proven stable• Take your time down this journey:

– Create modules from the monolith code– Test bounded contexts around modules for stability– Create services out of the modules

26

2. Writing Stateless Servers

• Key to being able to scale out and be fault tolerant– Every instance of the cluster are exactly the same– Any instance can be rebooted and be exactly as the same before– As a system, “states” have to be tracked somewhere. Use persistence

(databases like Cloudant, MongoDB; key-value pair stores like Redis)• Stateless web servers• Stateless worker servers

27

Stateless Web Servers

• Tracking user identity– Server-side session object: makes it a stateful server, don’t use it– Use identity tokens: take advantage of the modern browsers

• httponly, secure cookies• JWT (JSON Web Token)

• Handling HTTP requests– Use database or caching service to save state– Trade-off b/w latency (re-using in-memory objects across requests) and

memory consumption– Recycle instances to reclaim memory (kill and restart)

28

Stateless Worker Servers

• Runs background jobs– e.g. an integration flow execution

• Conventional mindset is “deploy the job so it’s ready to take input and process”– You end up with a stateful server because the job is deployed to a particular

server instance, which is the ONLY instance that can do the requested processing

– e.g. Node-RED (http://nodered.org/)• Instead, deploy the job on-demand along with the input to be

processed– Any of the instances can fulfill this request

http://nodered.org/

http://nodered.org/

29

3. Inter-Services Communication

• Use REST, avoid RPC (for synchronous communications)– RPC causes tight coupling

• Asynchronous, event/command based communications– Better throughput compared to synchronous APIs– Better at smoothing out spiky workloads– Better aligned with the “eventual consistency” design principle

30

More on Async Communication

• Sync request response– Tight coupling

• Coupled at runtime – consumer and provider must be available it the same time

• Coupled in terms of format and data structure – in both diredtions

• Async request-response– Less tightly coupled

• No runtime coupling

– BUT still have format and data coupling

• Event-based– Minimal coupling

• No runtime coupling

• Reduced format/data coupling (event consumer can decide how much of the event they need to understand)

31

Avoid Building Singleton Processes

• Consider this scenario:– “I need to deploy a process that keeps polling on a REST API every 5 minutes and

generate a message on a queue”

• If you deploy a process into a server and have that process run a timer:– It’s a singleton: can only have one of them running at any time– Can’t horizontally scale it up

• Build an asynchronous message-driven API instead with an external timer/job scheduler

32

4. Isolate Failure

MicroServices aren’t reliable by default Think about the potential for cascading failures Example

The worst kind of failure – ‘slow death’ Consider the implications of a single backend that starts to respond very slowly Dependent Services that interact synchronously will themselves start to respond

slowly Resources used up

Strategies Circuit breaker pattern Async Communication

33

5. API Gateway

• Service discovery• Service invocation

– Synchronous API calls• Request -> locate service -> call service -> relay response• Reverse proxy

– Asynchronous API calls• Message production upon request• Return status checking URL• Check results of async job (e.g. from database) and return in response to status

checking calls

34

6. Database Design for Microservices

• Shared database creates tight coupling• Schema change affects all microservices because they all have to

model the same schema in the shared database• Techniques to split databases

– Replace foreign key relationships with API calls to another service– Replace shared lookup tables with configuration files on individual services– Replace shared data with shared model and a microservice responsible for

managing the full lifecycle of that model• Single schema -> split schemas -> split microservices with dedicated

databases

35

7. Don’t Build Transactions Across Microservices

• Distributed Transactions are very difficult to build in a Microservices environment, and can seriously impact the system’s scalability

• Build instead for eventual consistency– Through re-try of failed operations– Build idempotent APIs– Otherwise rollback

36

8. Maximize Automation

• Automate deployment of microservices– Repeatable– Reliable

• Automate tests– Unit tests– service tests (API contract)– User journey tests

37

9. Logging and Monitoring

• Need a common and comprehensive logging and monitoring solution across all services

• Use log analysis to make sense of the interactions across all microservices

ELK (ElasticSearch, Logstash, Kibana)• Monitoring

– Graphite/Graphana

38

10. Security

• Single Sign On among microservices– SSO gateway: authenticates incoming requests and inserts headers for roles

• Service-to-service calls are made inside the network perimeter• Trust all calls from inside the perimeter?

– Confused Deputy Problem: can use authenticated UI session to perform operations or access data that the user aren’t supposed to be allowed to

• https basic authentication: inject id and password in headers for downstream services to validate

• Alternative Approaches– OAuth accounts for services– SSL mutual auth– API Keys

39

11. How To Run Microservices in a Cloud Scale

• Generally there are two approaches– Using a PaaS: CloudFoundry (IBM Bluemix and others)– DIY with Containers: mesos, marathon, kubernettes, swarm

• What might drive you to DIY– Need for very flexible scale up/scale down– Fine grain control of resources– Control over placement of processing

Summary

Principles of Microservices

Nurture a culture of

automation

Deploy Independently

Build to for monitoringIsolate Failure

Model around the business

domain

Notices and Disclaimers

41

Copyright © 2016 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law

Notices and Disclaimers Con’t.

42

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained h erein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

http://www.ibm.com/legal/copytrade.shtml

Thank YouYour Feedback is Important!

Access the InterConnect 2016 Conference Attendee Portal to complete your session surveys from your

smartphone, laptop or conference kiosk.