event sourcing with microservices

Event Sourcing and Microservices

Why & How?

Ralph Winzinger Taipei, April 2016

Senior Technical Leader

MicroservicesAgile

Architecture

Digital

RalphWinzinger

Cloud

Microservices• small - not only in terms of „lines of code“ but regarding

its functionality and the effort it takes to maintain

• self-contained, autonomous

• independent deployment unit

• fault-tolerant / resilient

• lightweight, having a small footprint

• exclusive access to its datastore

Microservices

• self contained

• fault-tolerant / resilient

• independent components

Microservices

• self contained

• fault-tolerant / resilient

• independent components

TransactionService

CustomerService

AccountService

Microservices

• self contained

• fault-tolerant / resilient

• independent components

TransactionService

CustomerService

AccountService

Microservices

• self contained ???

• fault-tolerant / resilient ???

• independent components ???

TransactionService

CustomerService

AccountService

Microservices

• self contained ???

• fault-tolerant / resilient ???

• independent components ???

TransactionService

CustomerService

AccountService

highly distributed systems!

Microservices

• self contained ???

• fault-tolerant / resilient ???

• independent components ???

TransactionService

CustomerService

AccountService

highly distributed systems!

Availability?

Microservices

• self contained ???

• fault-tolerant / resilient ???

• independent components ???

TransactionService

CustomerService

AccountService

highly distributed systems!

Availability?

Latency?

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

1 year = 8760 hours = 525600 minutes

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

1 year = 8760 hours = 525600 minutes

99% 88,5 hours downtime

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

1 year = 8760 hours = 525600 minutes

99,9% 8,8 hours downtime

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

1 year = 8760 hours = 525600 minutes

99,99% 53 minutes downtime

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

1 year = 8760 hours = 525600 minutes

99,999% 5 minutes downtime

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime)

1 year = 8760 hours = 525600 minutes

99,999% 5 minutes downtime

„five nines“high availability

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime) AS = AC(n-1) + ((1-AC(n-1))*ACn)

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime) AS = AC(n-1) + ((1-AC(n-1))*ACn)

S1

S2

S3

DB

FW

LB

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime) AS = AC(n-1) + ((1-AC(n-1))*ACn)

S1

S2

S3

DB

FW

LB

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime) AS = AC(n-1) + ((1-AC(n-1))*ACn)

S1

S2

S3

DB

99,5%

FW

LB 99,99%

99,99%

99,9%

99,9%

99,99%

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime) AS = AC(n-1) + ((1-AC(n-1))*ACn)

S1

S2

S3

DB

99,5%

FW

LB 99,99%

99,99%

99,9%

99,9%

99,99%

99,27% 64h downtime

Availabilityfor eCommerce 24/7 systems

AV = Uptime / (Uptime + Downtime) AS = AC(n-1) + ((1-AC(n-1))*ACn)

S1

S2

S3

DB

99,5%

FW

LB 99,99%

99,99%

99,9%

99,9%

99,99%

99,27% 64h downtimewe don’t want dependencies

& communication

LatencyThe more dependencies we have, the longer

our requests will take

LatencyThe more dependencies we have, the longer

our requests will take

S1

S2

S3

DB

FW

LB

LatencyThe more dependencies we have, the longer

our requests will take

S1

S2

S3

DB

FW

LB

LatencyThe more dependencies we have, the longer

our requests will take

S1

S2

S3

DB

100 msec

FW

LB 5 msec

5 msec

100 msec

100 msec

5 msec

LatencyThe more dependencies we have, the longer

our requests will take

S1

S2

S3

DB

100 msec

FW

LB 5 msec

5 msec

100 msec

100 msec

5 msec

315 ms

LatencyThe more dependencies we have, the longer

our requests will take

S1

S2

S3

DB

100 msec

FW

LB 5 msec

5 msec

100 msec

100 msec

5 msec

315 mswe don’t want dependencies

& communication

only data, not logicMicroservices contain all necessary

logic to process a request

only data, not logicMicroservices contain all necessary

logic to process a request

But maybe we need some additional data from other services, some „facts“

only data, not logic

• access the other service’ database

• keep data redundant

• low level database sync

• event sourcing

Microservices contain all necessary logic to process a request

But maybe we need some additional data from other services, some „facts“

only data, not logic

• access the other service’ database

• keep data redundant

• low level database sync

• event sourcing

Microservices contain all necessary logic to process a request

But maybe we need some additional data from other services, some „facts“

NO !

only data, not logic

• access the other service’ database

• keep data redundant

• low level database sync - also introduces coupling

• event sourcing

Microservices contain all necessary logic to process a request

But maybe we need some additional data from other services, some „facts“

NO !

only data, not logic

• access the other service’ database

• keep data redundant

• low level database sync - also introduces coupling

• event sourcing

Microservices contain all necessary logic to process a request

But maybe we need some additional data from other services, some „facts“

NO !

Domain Events• An important change in the state of our system.

Important with regard to business functionality.

• It happened in the past and it is immutable

• It is persistent

• The „sum“ of all events is the current system state

• It’s public for all services

Domain Events• An important change in the state of our system.

Important with regard to business functionality.

• It happened in the past and it is immutable

• It is persistent

• The „sum“ of all events is the current system state

• It’s public for all services

CustomerCreatedEvent

AccountOpenedEvent

TransactionProcessedEvent

CustomerAddressChangedEvent

Account BalanceMicroservices have exactly one thing to do.

They have one business functionality.

Account BalanceMicroservices have exactly one thing to do.

They have one business functionality.

Transactionsamount

Customer[accounts]

Accountbalance

Account BalanceThe balance depends on transactions.

Where can I find the current account balance?

Transactionsamount

Customer[accounts]

Accountbalance

Account BalanceThe balance depends on transactions.

Where can I find the current account balance?

• select balance from acc where accno=18773

Account BalanceLet’s assume we have a TransactionProcessedEvent

that is created each time, the TransactionService processed a transaction

Account BalanceLet’s assume we have a TransactionProcessedEvent

that is created each time, the TransactionService processed a transaction

TransactionService

DB

Online Banking

Mobile Banking

Account BalanceLet’s assume we have a TransactionProcessedEvent

that is created each time, the TransactionService processed a transaction

TransactionService

DB

Online Banking

Mobile Banking TransactionProcessedEvent

source = 18773 dest = 20770 amount = - 8.000 NTD

Account BalanceLet’s assume we have a TransactionProcessedEvent

that is created each time, the TransactionService processed a transaction

TransactionService

DB

Online Banking

Mobile Banking TransactionProcessedEvent

source = 18773 dest = 20770 amount = -8.000 NTDTransactionProcessedEvent

source = 170805 dest = 18773 amount = +1.500 NTD

Account BalanceThe balance depends on transactions.

Where can I find the current account balance?

• select balance from acc where accno=18773

• replay all events • use „empty“ / initial account • sum up all TransactionProcessedEvents

Account Balance

Accountbalance = 0

Account Balance

TPE (+120.000 NTD)

Accountbalance = 120.000

Account Balance

TPE (+120.000 NTD)

Accountbalance = 118.500

TPE (- 1.500 NTD)

Account Balance

TPE (+120.000 NTD)

Accountbalance = 110.500

TPE (- 1.500 NTD)

TPE (- 8.000 NTD)

Account Balance

TPE (+120.000 NTD)

Accountbalance = 127.500

TPE (- 1.500 NTD)

TPE (- 8.000 NTD)

TPE (+ 17.000 NTD)

Account Balance

TPE (+120.000 NTD)

Accountbalance = 677.500

TPE (- 1.500 NTD)

TPE (- 8.000 NTD)

TPE (+ 17.000 NTD)

TPE (+ 550.000 NTD)

Account Balance

TPE (+120.000 NTD)

Accountbalance = 652.500

TPE (- 1.500 NTD)

TPE (- 8.000 NTD)

TPE (+ 17.000 NTD)

TPE (+ 550.000 NTD)

TPE (- 25.000 NTD)

Account Balance

TransactionServiceAccountService

getTransactions()

Account Balance

TransactionServiceAccountService

getTransactions()

TPE

Account Balance

TransactionServiceAccountService

getTransactions()

TPE DB

ReadModel

Account Balance

TransactionServiceAccountService

getTransactions()

TPE DB

ReadModel

no more synchronous dependency!

!! Eventual Consistency !!

!! Eventual Consistency !!

TransactionServiceAccountService

TPE DB

ReadModel

!! Eventual Consistency !!

TransactionServiceAccountService

TPE DB

ReadModel t = 100 msec

!! Eventual Consistency !!

TransactionServiceAccountService

TPE DB

ReadModel t = 100 msec

The transaction was already processed but the AccountService still uses the old balance

!! Eventual Consistency !!

TransactionServiceAccountService

TPE DB

ReadModel t = 100 msec

The transaction was already processed but the AccountService still uses the old balance

Inconsistency! (but that’s how distribution works)

Technical Realization• Domain Events are very similar to messages

• publish / subscribe & fire forget

Technical Realization• Domain Events are very similar to messages

• publish / subscribe & fire forget

• Domain Events need a special semantic

• as a producer I want „fire & forget“

• as a consumer I want „exactly once“

Technical Realization• Domain Events are very similar to messages

• publish / subscribe & fire forget

• Domain Events need a special semantic

• as a producer I want „fire & forget“

• as a consumer I want „exactly once“

• even if a consumer is down for some time

• even if a consumer is late to the game

Technical Realization• Domain Events are very similar to messages

• publish / subscribe & fire forget

• Domain Events need a special semantic

• as a producer I want „fire & forget“

• as a consumer I want „exactly once“

• even if a consumer is down for some time

• even if a consumer is late to the game

multiple implementations available, e.g.:

Redis Persistent Queues

EventStore

Akka Persistence + MongoDB

Time Traveling

Time Traveling

AML Service

check limit

Time Traveling

AML Service

TPE

check limit

Time Traveling

AML Service

TPE

check limit suspicious

transactions

Time Traveling

AML Service

TPE

check limit suspicious

transactions

Analytics Alerting

Time TravelingAssume we will finish this service in two years.

How do we start it?

Time TravelingAssume we will finish this service in two years.

How do we start it?

A - start up & wait for new events

Time TravelingAssume we will finish this service in two years.

How do we start it?

A - start up & wait for new events

B - start up & process old events & wait for new events

Time TravelingAssume we will finish this service in two years.

How do we start it?

A - start up & wait for new events

B - start up & process old events & wait for new events

Now, we can apply new software (or changes) to the past!

bug fixes, new features, new analytics, … agile!

Time Traveling

AML Service

check limit suspicious

transactions

Analytics Alerting

TPE (+120.000 NTD)

TPE (- 1.500 NTD)

TPE (- 8.000 NTD)

TPE (+ 17.000 NTD)

TPE (+ 550.000 NTD)

TPE (- 25.000 NTD)

Time Traveling

AML Service

check limit suspicious

transactions

Analytics Alerting

TPE (+120.000 NTD)

TPE (- 1.500 NTD)

TPE (- 8.000 NTD)

TPE (+ 17.000 NTD)

TPE (+ 550.000 NTD)

TPE (- 25.000 NTD)

TPE (+ 550.000 NTD)

!! Race-Conditions !!We need „exactly once“!

!! Race-Conditions !!We need „exactly once“!

AccountService

TPE

!! Race-Conditions !!We need „exactly once“!

AccountService

TPE

!! Race-Conditions !!We need „exactly once“!

AccountService

AccountServiceAccountService

Account Service Cluster

TPE

!! Race-Conditions !!We need „exactly once“!

AccountService

AccountServiceAccountService

Account Service Cluster

TPE

!! Race-Conditions !!We need „exactly once“!

AccountService

AccountServiceAccountService

Account Service Cluster

TPE

coor

dina

tor

!! Race-Conditions !!We need „exactly once“!

AccountService

AccountServiceAccountService

Account Service Cluster

TPE

coor

dina

tor

multiple implementations available, e.g.:

Akka, …

What is the result?

What is the result?

Transactions

Customer

Account

Bank

AML

XYZ

What is the result?

Transactions

Customer

Account

Bank

AML

XYZ

lots of dependencies high latency & low availability

What is the result?

Transactions

Customer

Account

Bank

AML

XYZ

ReadModel

ReadModel

ReadModel

ReadModel

What is the result?

Transactions

Customer

Account

Bank

AML

XYZ

ReadModel

ReadModel

ReadModel

ReadModel

lots of independent services low latency & high availability

Sample Code

Sample Code

http://bit.ly/1RYRY6p

https://github.com/senacor/CUB-codesamples-1/tree/es-akka-eventstore

Event Sourcing …

… is not easy

But if you start thinking about this pattern, you will like how powerful it is

Any Questions …… on Event Sourcing?

… on my project at Cathay United Bank?

Right now or later - just as you want.

Any Questions …… on Event Sourcing?

… on my project at Cathay United Bank?

Right now or later - just as you want.