modeling microservices

@aahoogendoorn

MODELING MICROSERVICESSander Hoogendoornditisagile.nlMentoring ▪ Consulting ▪ Training ▪ Agile ▪ Software architecture ▪ Code

MICROSERVICES? STAIRWAY TO HEAVEN OR HIGHWAY TO HELL?©2015 ditisagile.nl. All Rights Reserved 2@aahoogendoorn

www.ditisagile.nl

Sander Hoogendoorn

Me Dad Mentor, trainer, software architect, programmer Books and +250 articles Keynotes and talks at +150 conferences

Work Owner ditisagile.nl CTO Klaverblad Insurances

Web www.sanderhoogendoorn.com www.smartusecase.com www.speedbird9.com @aahoogendoorn [email protected]


www.ditisagile.nl

@aahoogendoorn

MONOLITHS Hard to deliver, even harder to test and impossible to maintain


www.ditisagile.nl

WHO OF YOU HAS A SYSTEM THAT IS TOO BIGAND THAT YOU WOULD RATHER BREAK UP?


www.ditisagile.nl

Monoliths

Advantages A single (layered) architecture

A single technology stack

A single code base maintained by multiple teams

Disadvantages All parts are interconnected

Many other systems are connected to your system

Hard to change, hard to maintain

Long time between releases, thereby increasing risks

Slow innovation

Hard to move to newer technologies

Doesn’t scale very well

Product Account

Order Customer

Products Accounts

Orders Customers


www.ditisagile.nl

Dependencies will kill you

@aahoogendoorn

A BRIEF HISTORY OF COMPONENTS AND SERVICES


www.ditisagile.nl

Client server


www.ditisagile.nl

Component based development


www.ditisagile.nl

Service oriented architecture


www.ditisagile.nl

Microservices

@aahoogendoorn

MICROSERVICESBeyond the hype?


www.ditisagile.nl

Microservices. Beyond the hype?


www.ditisagile.nl

Gartner hype cycle

@aahoogendoorn

MICROSERVICESThe clear benefits


www.ditisagile.nl

BUT FIRST… A DEFINITION


www.ditisagile.nl

In short, the microservice architectural style is an approach to developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API.

These services are built around business capabilities and independently deployable by fully automated deployment machinery. There is a bare minimum of centralized management of these services, which may be written in different programming languages

and use different data storage technologies.Martin Fowler


www.ditisagile.nl

Monoliths. Scalability

Product Account

Order Customer

Product Account

Order Customer

Product Account

Order Customer


www.ditisagile.nl

Microservices. Scalability

Product Account

OrderCustomer


www.ditisagile.nl

Microservices. Scalability

Product Account

OrderCustomer

Product

CustomerCustomer Customer

Account Account


www.ditisagile.nl

Microservices. Scalability & containers

Product Account

Order

Product

Customer

Account Account

CustomerCustomer


www.ditisagile.nl

Monoliths. Persistence

Product Account

Order Customer

Products Accounts

Orders Customers


www.ditisagile.nl

Microservices. Polyglot persistence

Product Account OrderCustomer

MongoDB

Customers

MongoDB

Orders

Active Directory

Accounts

Oracle

Products


www.ditisagile.nl

Microservices. Promises

Products not projects

Scalable

Decentralized governance

Replaceable parts

High performance

Technology independent

Polyglot persistence

Easy to build

Easy to test

Easier deployment than monoliths

ProductAccount

Order

Customer


www.ditisagile.nl

Microservices. But…

What is a microservice exactly?

How small is a microservice?

Requirements in a microservice world

Components or services

Who owns a microservice?

What technologies do you use?

What protocols do you apply?

How to define messages

How to test microservices

How to coordinate when business services run across components?

How to build deployment pipelines?

ProductAccount

Order

Customer


www.ditisagile.nl

Opinions, opinions, opinions

@aahoogendoorn

ARE MICROSERVICESA STAIRWAY TO HEAVEN?

@aahoogendoorn

OR A HIGHWAY TO HELL?

@aahoogendoorn

A REAL WORLD CASE


www.ditisagile.nl

A major insurance company


www.ditisagile.nl

A major insurance company

We have Most functionality on an expensive mainframe

A wide variety of large systems written in Java that are hard to maintain and to test, and that are very hard to replace

Individual systems that cover large areas of functionality, usually coupled to departments

Aging technology

No mobile strategy, allowing for new business or new services to clients, and intermediaries

We need to Get rid of the mainframe

Shorten time-to-market

Lower TCO

Uphold a fully secure systems landscape


www.ditisagile.nl

Where do we come from?


www.ditisagile.nl

Outsourcing didn’t work


www.ditisagile.nl

Where do we go?


www.ditisagile.nl

FOR THE THINGS WE HAVE TO LEARN BEFORE WE CAN DO THEM, WE LEARN BY DOING THEM

Aristotle


www.ditisagile.nl

MICROSERVICES REQUIRE AN EVOLUTIONARY ARCHITECTURE

@aahoogendoorn

START WITH SOME GUIDING PRINCIPLES


www.ditisagile.nl

Questions, questions, questions

Communication architecture. The glueHow do we define interfaces between apps and component? How do we arrange messaging?

How do we glue together rapidly changing apps and component?

Application architectureEnd user facing Different users, different fast evolving needs

Which technology is the best for which purpose?

Component architectureComponents and services are evolving rapidly. How do we decide which components we need? How do we deal with versioning? How do we deal with distributed processes and transactions?


www.ditisagile.nl

Case. Guiding principles

We decided to go from here Client thinks in business processes, so we implement

business processes We move away from the mainframe, to a new systems

landscape, consisting of micro-applications and micro-components

Requirements and documentation are modeled rather than written

Applications implement a single (elementary) business process

Components serve a single purpose and offer services Applications and components all have their own bounded

context – a domain model Applications and components will have an similar internal

software architecture to facilitate ease of maintenance and allow for harvesting re-use

Communication between applications and components will use a simple open protocol - REST

@aahoogendoorn

APPLICATIONS


www.ditisagile.nl

Presentation

Process

Domain

Services

Outside world

PagesGrids / Panels, Controls

Use casesFlow

Domain objects, Factories / RepositoriesEnums / Value objects / Tupels / Reference objects

Service gateways, Service clientsInfo objects / Search objects

ComponentsRelations Dossiers Intermediaries Accounts Rates

@aahoogendoorn

COMPONENTS


www.ditisagile.nl

Service interface

Process

Domain

Data / Services

Outside world

ResourcesRepresentations

Use casesFlow


Storage gateways, Storage clientsInfo objects / Search objects

StorageRelations Dossiers Intermediaries DB2 MongoDB


www.ditisagile.nl

NOW WHAT DOES ALL THIS HAVE TO DO WITH MODELING?


www.ditisagile.nl

DOING BIG UP-FRONT DESIGN IS DUMBDOING NO DESIGN IS EVEN DUMBER

Dave Thomas

@aahoogendoorn

BUSINESS PROCESSES FIRST


www.ditisagile.nl

Different levels of processes (and requirements)


www.ditisagile.nl

A high-level business process

Step 1 Step 2 Step 3 Step 4 Step 5


www.ditisagile.nl

Split into work processes

Step 3.1 Step 3.2 Step 3.3 Step 3.4

Step 3


www.ditisagile.nl

Split into elementary processes - OTOPOP

Step 3.3.1 Step 3.3.2 Step 3.3.3 Step 3.3.4

Step 3.3

Step 3.3.5


www.ditisagile.nl

Smart use cases

Modeling applications

Each elementary process is implemented in a single application

The requirements are modeled using smart use cases

Each application consists of a single sea-level use case and a number of fish-level use cases

Additionally we add the services that are required to implement the applications to the model

Doing so, we can easily do impact mapping on our services

Also, the smart use cases form a strong foundation for integration testing


www.ditisagile.nl

Different levels of use cases

Traditionaluse cases

Smartuse cases

Format Textual Visual

Granularity Different Unified

Estimate Hard Easy

Unit of work Lousy Good

Reuse Incidental Normal

Traceability Possible Normal

Testability Poor Good


www.ditisagile.nl

Smart use cases


www.ditisagile.nl

57Smart use cases


www.ditisagile.nl

Wireframes with use cases

@aahoogendoorn

MODULAR DESIGN / DESIGNING MICROSERVICES


www.ditisagile.nl

Single responsibility principle (SRP)

SOLID Single Responsibility Principle

Open Closed Principle

Liskov Substituion Principle

Interface Segregation Principle

Dependency Inversion Principle

Single Responsibility Principle Every module should have responsibility over a single part of the

functionality provided by the software,

That responsibility should be entirely encapsulated by the class

All its services should be narrowly aligned with that responsibility

Therefore Group together things that change together

Separate things that change for different reason


www.ditisagile.nl

Bounded context

Domain driven design The paradigm of designing software based on models

of the underlying domain

The domain model helps the business and the developers to reason about the functionality

A model needs to be unified – internally consistent without contradictions

Bounded context The bounded context is a central pattern in domain

driven design

When you model larger domains, it becomes progressively harder to create this single unified model

So, instead of creating a single unified model, you create several, all valid within their bounded context


www.ditisagile.nl

The single unified domain model

Product

Vendor

StockOrder

Client

Delivery

Payment


www.ditisagile.nl

Bounded contexts

Product

Vendor

StockOrder

Client

Delivery

Payment

Product


www.ditisagile.nl

Bounded contexts

Modeling Modeled in a (single) domain model

Model contains entities, domain objects

Usually evolves around a single main entity (aggregate root)

Services Interrogate the bounded context

Resources usually follow the aggregate root

Services post and put representations

Representations are mapped to the bounded context

Validation Representations can be validated before being mapped

Bounded context can be validated as a whole e.g. before storing

Business rules

Properties and property types

Product

Vendor

Stock


www.ditisagile.nl

Properties and property types

Basic types string, integer, date, datetime Include nullable wrapping

Enumerations Set up at design time, unchangeable at run-time Genders, Categories

Value objects No specific instances Isbn, Email, Url, Money

References (code tables) Changeable at run-time, such as contract types

Associations To cached entities such as Country, Nationality To first level citizens such as Customer, Product


www.ditisagile.nl

66

Every application,worker and componenthas its own bounded context

@aahoogendoorn

AND THE REST ISCOMMUNICATION (OVER HTTP)


www.ditisagile.nl

HTTP RETURN CODES CHEAT SHEET1**. Hold on

2**. Here you go 3**. Go away

4**. You fucked up 5**. I fucked up


www.ditisagile.nl

HTTP GET

GET

Retrieve whatever information is identified by the request URI in the form of an entity

The entity is usually a single or a list of objects (of the type provided by the service, often JSON, or XML)

GET is safe (retrieval only) and idempotent

Unfortunately there are many ways of creating GET requests (see examples below)

Returns 200 (found), possibly 400 (bad request) or 404 (not found)

Examples

Get an entire collectionlocalhost:8080/countries

Find objects in the collectionlocalhost:8080/countries?name=“stan”

Find an object in the collection by IDlocalhost:8080/countries/38

Find a sub-object in the collection by IDlocalhost:8080/countries/38/capital

Find object in the collection by ISOlocalhost:8080/countries?isocode=“GRC”

Find object in the collection by ISOlocalhost:8080/countries/isocode/GRC

Find object in the collection by ISOlocalhost:8080/countries/GRC


www.ditisagile.nl

HTTP POST

POST

Request that the server accepts the entity enclosed in the request as a new subordinate of the resource identified by the request URI

The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it

Returns the location header for the new resource, possibly with created entity in the body

Returns 201 (created), possibly 500 (server error)

Examples

Post to the collection (with entity in body)localhost:8080/countries

Returning location headerlocalhost:8080/countries/38


www.ditisagile.nl

HTTP PUT

PUT

Requests that the enclosed entity is stored under the supplied request URI

If the request URI refers to an existing resource, the enclosed entity is an modified version of the resource

POST identifies the resource that will handle the enclosed entity

PUT identifies the entity enclosed with the request

Returns 200 or 204 (when resource is modified), possibly 201 (if a resource is created), possibly 500 (server error)

Examples

Put with ID (with entity in body)localhost:8080/countries/38

Update capital (with entity in body)localhost:8080/countries/38/capital


www.ditisagile.nl

HTTP DELETE

DELETE

Requests that the resource identified by the request URI is deleted.

Returns 200 or 204 (when resource is deleted), possibly 500 (server error)

Examples

Deletelocalhost:8080/countries/38

@aahoogendoorn

MODELING RESOURCES


www.ditisagile.nl

Modeling resources

Root resources (component)/qa

/questionnaires

GET

Questionnaire

Id, Name, Description

/qa

GET the collection, but only limited to this representation (but with locations likely)/qa/questionnaires

/questionnaires

GET

/{id}Questionnaire

Id, Name, Description

Question

Type, Name, Description

Answer

Name, Value

GET a single item from the collection, but with representation/qa/questionnaires/334532


www.ditisagile.nl

75Resource model


www.ditisagile.nl

76Service use cases


www.ditisagile.nl

77Component bounded context

@aahoogendoorn

TESTING MICROSERVICES


www.ditisagile.nl

A service development lifecycle

Code Developer Test Test Integration Test Acceptance Test LivePrepare & Design


www.ditisagile.nl

What to test


DevelopersUnit tests

DevelopersQ & A

TestersScenario’s & API’s


Product ownerProduct


www.ditisagile.nl

Even though you might have brilliant testers


www.ditisagile.nl

… please automate your tests


DevelopersUnit tests

DevelopersQ & A



Product ownerProduct


www.ditisagile.nl

Developer Q&A Sonarqube


www.ditisagile.nl

Developer Q&A Sonar


www.ditisagile.nl

Presentation

Process

Domain

Services

Outside world

PagesGrids / Panels, Controls

Use casesFlow


Service gateways, Service clientsInfo objects / Search objects

ComponentsRelations Dossiers Intermediaries Accounts Rates

Integration testsBDD & Selenium

Unit tests

Unit tests

Unit tests


www.ditisagile.nl

Service interface

Process

Domain

Data / Services

Outside world

ResourcesRepresentations

Use casesFlow


Storage gateways, Storage clientsInfo objects / Search objects

StorageRelations Dossiers Intermediaries DB2 MongoDB

Integration testsUsing BDD

Unit tests

Unit tests

Unit tests


www.ditisagile.nl

WHAT ABOUT BEING INDEPENDENTLY DEPLOYABLE?

@aahoogendoorn

DEPLOYING MICROSERVICESContinuous integration, build pipelines and continuous delivery


www.ditisagile.nl

A typical build pipeline



www.ditisagile.nl

ProductionAcceptanceIntegrationTestDevelopment

A typical build pipeline



www.ditisagile.nl

Build pipelines in Jenkins


www.ditisagile.nl

Microservices. Building a deployment pipeline

Code Developer Test Test Acceptance Test Acceptance Live





www.ditisagile.nl

Microservices. Pipeline hell?

Code v.2 Developer Test v.2 Test v.2 Acceptance Test v.2 Acceptance v.2


Test v.2 Acceptance Test v.2 Acceptance v.2 Live v.2

Developer Test Test Acceptance Test Acceptance Live

Code v.3 Developer Test v.3

Live

Code v.2


www.ditisagile.nl

Dealing with breaking changes

Customer

v1

App1 App2

Customer

v1

App1 App2

v2

Customer

v1

App1 App2

v2

Customer

App1 App2

v2

@aahoogendoorn

DO WE REALLY NEED PROJECTS?From projects to releases to continuous delivery


www.ditisagile.nl

Do we really need projects?


www.ditisagile.nl

Planning?


www.ditisagile.nl

Or roadmap?


www.ditisagile.nl

Minimal viable product


www.ditisagile.nl

Maintenance

From projects to continuous delivery?

Project

MaintenanceMVP

MaintenanceContinuous delivery


www.ditisagile.nl

Moving towards DevOps is not easy

@aahoogendoorn


www.ditisagile.nl

WORDLY WISDOM TEACHES US THAT IT IS BETTER FOR REPUTATION TO FAIL CONVENTIONALLY THAN TO SUCCEED UNCONVENTIONALLY

John Maynard Keynes

@aahoogendoorn

SOME LAST THOUGHTS


www.ditisagile.nl

Microservices are not for everyone

What are we going to benefit from the most?

Polyglot persistence?

Being able to multiple stacks?

Independentlydeployable?

Enforcemodular design?

ContinuousDelivery?


www.ditisagile.nl

Minimal viable product

@aahoogendoorn

ALLOW THE TEAM TO LEARN CONTINUOUSLY…


www.ditisagile.nl

Will it work?


www.ditisagile.nl

The hockey stick model

@aahoogendoorn

… AND HAVE FUN

@aahoogendoorn

THIS IS AGILE

www.createspace.com/4747266

Password: agilescrum

Discount code: KGNWKKWG

@aahoogendoorn

www.sanderhoogendoorn.comwww.smartusecase.comwww.speedbird9.com

[email protected]

@aahoogendoorn

REFERENCES AND QUESTIONS