agile lab_bigdata_meetup_akka
TRANSCRIPT
Reactive and Parallel Programming
Presents...
Enrico PigozziVito Ressa
Who is Agile Lab?
GO BIG (data) or GO HOME
Summary
• Concurrency problemRecurrent issuesHandle concurrency
• AkkaOverviewHow it worksAkka clusterExpansions and use-casesHow we use Akka
• Demo time
Parallel and
concurrent Programming
Ⓒ2016 Agile Lab S.r.l.
“We need to stop coding our concurrency and just let our code run concurrently”
Shared/mutable state
• Threads (pool) usage to make code “faster” + synchronization features (primitives, locks, defensive copies, etc.) to make it “safe”.
• We tend to create the problem first, and (don’t) fix it up later (N.B. no real concurrency allowed)
Issues
• Threads
• expensive (startup, memory, context switching)
• limited (by OS)
• CPU Vs. Synchronus IO balance required
• optimizated usage requires non-blocking API (cumbersome to write and test)
• Design
• bottlenecks, race conditions, deadlocks
• traditional errors management approches (exceptions and error codes) don’ttranslate well
• maintenance / refactoring costs can increase to a fairy high degree (and thereforeavoided)
Software Transactional Memory
• Turns memory heap in a transactional data set (Atomic, Consistent, Isolated)
• Based on Transactional References:
Memory cells, holding an immutable value (separates identity from value)
Implement CAS (Compare-And-Swap) semantic (old value still availabeafter change)
Can only be altered within a transaction
Very fast (near constant time)
Thread and iteration safe
Futures
future(a,b)futureB(a)
Asynchronous values
Composable functional abstraction
Error propagation and recover allowed
Callback on termination available
for {
a ← futureA()
b ← futureB(a)
c ← futureC(a,b)
} yield c
futureA()
Actors & Message passing• Unit of code organization: “policy” separated from “logic”, incapsulate state,
behaviour, mailbox
• Weakly / Adaptive typed: achitectural flexibility enhanced (decupled design)
• Safe: one thing a time in isolated context (actor accessed by reference, interaction by messages)
• Immutable structures/messages managed by actors in isolated/safe context
Finite State Machine
• Applied to actor: must be in one of the defined (in type system) state all the time
• A data object can be attached to each state (M:N relation)
• A starting state (and eventually data) has to be defined
• Event insted of simple messages: provides current state info
• 'When/goto/stay' model: define actor reaction (partial function) to eventfor a state
S1S2
S3S4S5
A
B A
A A
A
B
BB
BA,B = possible messages
S1,…S5 = States
Microservices
• Bounded Context: Each element of functionality into a separate service• Fault tolerance• Deployment independency• DataStore independency• Loose coupling• Service Discovery• Sync/async messaging between services• Scalability
Akka
Ⓒ2016 Agile Lab S.r.l.
Overview
Event-driven systems with
Actors
High Performance
Easy concurrency handling
Simple distribution
High flexibility
ResilienceReactive Manifesto
http://www.reactivemanifesto.org/
Commercial Support
Strenghts
How it works?
• Actor as an essential code unit• Messages for actors interaction• Hierarchical organization• Supervision Hierarchy
Actors Lifecycle
• Actor instance is hidden• Actor restarting
transparent to other actors• Mailbox bounded to the
incarnation
Messages
• Any kind of message but immutable!• Hint: In scala use case classes and take advantage of
pattern matching (see next slide)
Tell or Ask
Mailbox• Landing point for messages• Bounded to the incarnation of the actor (not the instance!)• Allows different configurations (FIFO, LIFO, custom priority, ...)• A mailbox can be shared in case of balancing pools
MAILBOX STASH
unstash()
Behaviors
Message handling
Behaviors management
Akka
Cluster
Ⓒ2016 Agile Lab S.r.l.
Akka Cluster
Node A
Node B
Node C
Node D
LeaderCLUSTER
Cluster membership service• Peer to peer• Decentralized• Fault tolerant• No single point of failure• No single point of bottleneck
A Node is identified by a tuple
hostname:port:uid
• Gossip Protocol• Automatic failure detector
Failure detection
An implementation of The Phi Accrual Failure Detector
Failure Detector
input
Nodes heartbeats phi-value
Is a node unreachable or not? Depends on a treshold
Extensions and
use-cases
Ⓒ2016 Agile Lab S.r.l.
Extensions
Community projects available at http://akka.io/community/
• Akka persistence - http://doc.akka.io/docs/akka/current/scala/persistence.html• Akka HTTP - http://doc.akka.io/docs/akka-stream-and-http-
experimental/current/scala/http/• Akka Streams - http://doc.akka.io/docs/akka-stream-and-http-
experimental/current/scala/http/• Akka Camel - http://doc.akka.io/docs/akka/current/scala/camel.html• Akka FSM - http://doc.akka.io/docs/akka/current/scala/fsm.html
Use cases: GearPump
Ref: https://www.typesafe.com/resources/case-studies-and-stories/intels-gearpump-real-time-streaming-engine-using-akka
• Real-time streaming engine• Multiple stream datasources• Need of HA, scalability, performance• "Once at a time" transactional message processing
With 4 nodes GearPump can process up to 11 million
messages per seconds with an average latency of 17ms
(100 bytes messages)
Use cases: Paycasso
• Three-factor authentication platform• High speed authentication is mandatory• Scala, Akka and Cassandra at the core as a reactive stack• Make use of Akka HTTP, Akka Persistence, Akka Cluster Sharding.
Ref: https://www.typesafe.com/resources/case-studies-and-stories/paycasso-conquers-speed-gap-in-three-factor-authentications-with-reactive-stack
Use cases: SAMI• Data broker in the cloud for IoT• Need for reactiveness and resiliency• Microservices-based platform with Akka
Akka
usages
• Actors dedicated to clients for each WebSocket.
• At the transformation layer, where data is
streamed into the system and normalized.
Ref: https://www.typesafe.com/resources/case-studies-and-stories/samsung-strategy-and-innovation-center-executes-iot-vision-at-startup-speed-with-reactive-architecture
Other use cases
Interesting Discussion
http://stackoverflow.com/questions/4493001/good-use-case-for-akka
Online Gaming
Finance
StatisticsBetting
Social Media
Telecom
Web Media
How we use
Akka
Ⓒ2016 Agile Lab S.r.l.
Akka in Action with
Microservices Mosaico RCS
Reverse Proxy
Akka
Srv1
Akka
Srv2
Akka
Srv3
Akka
Srv4Akka
Srv5
Akka
Srv6
Mongo
DB
Mongo
DB
My
SQLS3
Akka Custering
Akka in Action with BigData
Wasp AgileLab
Akka Custering
master
Producer
Guardian
Consumer
Gardian
API
Guardian
E
PE
PE
PE
P
K
RK
R
K
R
R
T
S
P
S
PWS TCP
Legend:EP: EndPoint DataInKR:KafkaRouterSP: Spark PipelineRT: Real TimeWS: Web Socket API
DataIn
ControlPlane
Demo time...
Ⓒ2016 Agile Lab S.r.l.
