reactive design patterns

Reactive Design PatternsDr. Roland Kuhn

@rolandkuhn — Akka Tech Lead

Reactive Design Patterns

• currently in MEAP

• all chapters done (to be released)

• use code 39kuhn (39% off)

2

Reactive?

Elasticity: Performance at Scale

4

Resilience: Don’t put all eggs in one basket!

5

Result: Responsiveness

• elastic components that scale with their load

• responses in the presence of partial failures

6

Result: Decoupling

• containment of • failures

• implementation details

• responsibility

• shared-nothing architecture, clear boundaries

• Microservices: Single Responsibility Principle

7

Result: Maintainability

• decoupled responsibility—decoupled teams

• develop pieces at their own pace

• continuous delivery

8

Implementation: Message-Driven

• focus on communication between components

• model message flows and protocols

• common transports: async HTTP, *MQ, Actors

9

Reactive Traits

10

elastic resilient

responsive maintainable extensible

message:driven

Value

Means

Form

Architecture Patterns

Simple Component Pattern

12

«A component shall do only one thing,

but do it in full.»

Simple Component Pattern

• Single Responsibility Principle formulated by DeMarco in «Structured analysis and system specification» (Yourdon, New York, 1979) • “maximize cohesion and minimize coupling”

• “a class should have only one reason to change”(Uncle Bob Martin’s formulation for OOD)

13

Example: the Batch Job Service

• users submit jobs

• planning and validation rules

• execution on elastic compute cluster

• users query job status and results

14


15


16


17

Let-It-Crash Pattern

18

«Prefer a full component restart to

complex internal failure handling.»


• Candea & Fox: “Crash-Only Software”(USENIX HotOS IX, 2003)

• transient and rare failures are hard to detect and fix

• write component such that full restart is always o.k.

• simplified failure model leads to more reliability

19


• Erlang philosophy from day one

• popularized by Netflix Chaos Monkey • make sure that system is resilient by arbitrarily performing

recovery restarts

• exercise failure recovery code paths for real

• failure will happen, fault-avoidance is doomed

20

Implementation Patterns

Circuit Breaker Pattern

22

«Protect services by breaking the

connection during failure periods.»

Circuit Breaker Pattern

• well-known, inspired by electrical engineering

• first published by M. Nygard in «Release It!»

• protects both ways: • allows client to avoid long failure timeouts

• gives service some breathing room to recover

23

Circuit Breaker Example

24

private object StorageFailed extends RuntimeExceptionprivate def sendToStorage(job: Job): Future[StorageStatus] = { // make an asynchronous request to the storage subsystem val f: Future[StorageStatus] = ??? // map storage failures to Future failures to alert the breaker f.map { case StorageStatus.Failed => throw StorageFailed case other => other }}

private val breaker = CircuitBreaker( system.scheduler, // used for scheduling timeouts 5, // number of failures in a row when it trips 300.millis, // timeout for each service call 30.seconds) // time before trying to close after tripping

def persist(job: Job): Future[StorageStatus] = breaker .withCircuitBreaker(sendToStorage(job)) .recover { case StorageFailed => StorageStatus.Failed case _: TimeoutException => StorageStatus.Unknown case _: CircuitBreakerOpenException => StorageStatus.Failed }

Saga Pattern

25

«Divide long-lived distributed

transactions into quick local ones with

compensating actions for recovery.»

Saga Pattern: Background

• Microservice Architecture means distribution of knowledge, no more central database instance

• Pat Helland: • “Life Beyond Distributed Transactions”, CIDR 2007

• “Memories, Guesses, and Apologies”, MSDN blog 2007

• What about transactions that affect multiple microservices?

26

Saga Pattern

• Garcia-Molina & Salem: “SAGAS”, ACM, 1987

• Bank transfer avoiding lock of both accounts: • T₁: transfer money from X to local working account

• T₂: transfer money from local working account to Y

• C₁: compensate failure by transferring money back to X

• Compensating transactions are executed during Saga rollback

• concurrent Sagas can see intermediate state

27

Saga Pattern

• backward recovery:T₁ T₂ T₃ C₃ C₂ C₁

• forward recovery with save-points:T₁ (sp) T₂ (sp) T₃ (sp) T₄

• in practice Sagas need to be persistent to recover after hardware failures, meaning backward recovery will also use save-points

28

Example: Bank Transfer

29

trait Account { def withdraw(amount: BigDecimal, id: Long): Future[Unit] def deposit(amount: BigDecimal, id: Long): Future[Unit]}

case class Transfer(amount: BigDecimal, x: Account, y: Account)

sealed trait Eventcase class TransferStarted(amount: BigDecimal, x: Account, y: Account) extends Eventcase object MoneyWithdrawn extends Eventcase object MoneyDeposited extends Eventcase object RolledBack extends Event


30

class TransferSaga(id: Long) extends PersistentActor { import context.dispatcher

override val persistenceId: String = s"transaction-$id"

override def receiveCommand: PartialFunction[Any, Unit] = { case Transfer(amount, x, y) => persist(TransferStarted(amount, x, y))(withdrawMoney) }

def withdrawMoney(t: TransferStarted): Unit = { t.x.withdraw(t.amount, id).map(_ => MoneyWithdrawn).pipeTo(self) context.become(awaitMoneyWithdrawn(t.amount, t.x, t.y)) }

def awaitMoneyWithdrawn(amount: BigDecimal, x: Account, y: Account): Receive = { case m @ MoneyWithdrawn => persist(m)(_ => depositMoney(amount, x, y)) }

...}


31

def depositMoney(amount: BigDecimal, x: Account, y: Account): Unit = { y.deposit(amount, id) map (_ => MoneyDeposited) pipeTo self context.become(awaitMoneyDeposited(amount, x))}

def awaitMoneyDeposited(amount: BigDecimal, x: Account): Receive = { case Status.Failure(ex) => x.deposit(amount, id) map (_ => RolledBack) pipeTo self context.become(awaitRollback) case MoneyDeposited => persist(MoneyDeposited)(_ => context.stop(self))}

def awaitRollback: Receive = { case RolledBack => persist(RolledBack)(_ => context.stop(self))}


32

override def receiveRecover: PartialFunction[Any, Unit] = { var start: TransferStarted = null var last: Event = null

{ case t: TransferStarted => { start = t; last = t } case e: Event => last = e case RecoveryCompleted => last match { case null => // wait for initialization case t: TransferStarted => withdrawMoney(t) case MoneyWithdrawn => depositMoney(start.amount, start.x, start.y) case MoneyDeposited => context.stop(self) case RolledBack => context.stop(self) } }}

Saga Pattern: Reactive Full Circle

• Garcia-Molina & Salem note: • “search for natural divisions of the work being performed”

• “it is the database itself that is naturally partitioned into relatively independent components”

• “the database and the saga should be designed so that data passed from one sub-transaction to the next via local storage is minimized”

• fully aligned with Simple Components and isolation

33

Conclusion

Conclusion

• reactive systems are distributed

• this requires new (old) architecture patterns

• … helped by new (old) code patterns & abstractions

• none of this is dead easy: thinking is required!

35