the strange world of javascript and all its little asynchronous beasts

The strange world of javascript and

all its littleasynchronous beasts

Federico Galassi

@federicogalassi

http://federico.galassi.net

I wrote this

http://www.jsbestpractices.it

asynchronous

non-blocking

event-driven

confused!

callbacks...

Rob Pike

Concurrencyis not

Parallelism

http://vimeo.com/49718712

Rob Pike

Concurrency is a way tostructure a program bybreaking it into piecesthat can be executedindependently

Javascriptis

concurrent !

All in one threadNo it’s not.

Joe Armstrong

I want to disappear

Unlike utopian worlds...I live in

synchronousbliss!!

Javascriptcan learn from

concurrentlanguages

Go has gophers

Gophers block onchannels

c := make(chan int)go func() { for i := 0; i < 100; i++ { c <- i }}()go func() { for { i := <- c fmt.Println(i) }}()

Erlang has actors

Actors block onreceive

P = fun Producer(N, C) when N < 100 -> C ! {N, self()}, Producer(N + 1, C)end.

C = fun Consumer() -> receive {N, Pid} -> io:format("received ~p from ~p~n", [N, Pid]), Consumer() endend.

Cons = spawn(C).P(0, Cons).

The gold rule is

You must be able toBLOCK

Javascriptwas

concurrentby

necessity

Brendan Eich

Key pressed

Do nothing Button.onclickExec.

Eventqueue

TimeKey pressed

the Event Loop

ClickKey pressed

Click

User click

button.onclick = function() { element.style.color = "red"})

Continuationpassing style

refuel()startEngine()takeOff()land()// ... done

Continuationpassing style

refuel(function() { startEngine() takeOff() land() // ... done})

Continuationpassing style

refuel(function() { startEngine(function() { takeOff() land() // ... done })})

Continuationpassing style

refuel(function() { startEngine(function() { takeOff(function() { land() // ... done }) })})

The Pyramid of Doom

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

The Pyramid of Doom

Loss of control flow

images.forEach(function(url) { var image = download(url) image.show()})

Loss of control flow

images.forEach(function(url) { download(url, function(image) { image.show() })})

Loss of control flow

var showImages = function(images, callback) { var url = images.shift() if (url) { download(url, function(image) { image.show() showImages(images, callback) }) } else { callback() }})

Loss of control flow

Loss of error handling

try { download(url, function(image) { image.show() })} catch(e) { // never executed!! console.log("Cannot show image")}

Sync/Async Ambiguity

try { // is download Asynchronous?!?! download(url, function(image) { image.show() })} catch(e) { // never executed!! console.log("Cannot show image")}

Sync/Async Ambiguity

try { // Is download Asynchronous?!?! download(url, function(image) { image.show() })} catch(e) { // never executed!! console.log("Cannot show image")}

It’s not really like this

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

More like this

// ... After a while ...

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

Where do we land?

// ... After a while ...

???

???

???

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

What do we know?

When an async call completesI’m time warped back in timeto the callback code then backto wherever I came from.I find this very difficult tounderstand

http://joearms.github.io/2013/04/02/Red-and-Green-Callbacks.html

Joe Armstrong

It’s even worse, every javascript programmer who has a concurrent problem to solve must invent their own concurrency model

Joe Armstrong

http://joearms.github.io/2013/04/02/Red-and-Green-Callbacks.html

In javascript youCAN’T BLOCK

ES6 to rescue us!

with Generators

function* upTo(end) { for (var i = 0; i <= end; i++) { yield i }}

Generators make iterators

var counter = upTo(100)

counter.next() // => Object {value: 0, done: false}counter.next() // => Object {value: 1, done: false}counter.next() // => Object {value: 2, done: false}

// ...

counter.next() // => Object {value: 99, done: false}counter.next() // => Object {value: undefined, done: true}

function* upTo(end) { for (var i = 0; i <= end; i++) {

yield i

}}

Generators rememberexecution stack

restartsHere!

Yieldcan receive values

function* upTo(end) { for (var i = 0; i <= end; i++) { var newI = yield i if (newI) i = newI }}

var counter = upTo(100)

counter.next() // => Object {value: 0, done: false}counter.next() // => Object {value: 1, done: false}counter.next(10) // => Object {value: 11, done: false}counter.next() // => Object {value: 12, done: false}

Yieldcan receive errors

function* upTo(end) { for (var i = 0; i <= end; i++) { yield i }}

var counter = upTo(100)

counter.next() // => Object {value: 0, done: false}counter.next() // => Object {value: 1, done: false}counter.throw(new Error("argh")) // => Error: argh

Yes, this isBlocking!!

Blocking for sequence

async(function*() { yield refuel() yield startEngine() yield takeOff() yield land()})

Blocking forcontrol flow

async(function*() { images.forEach(function(url) { var image = yield download(url) image.show() })})

Blocking forerror handling

async(function*() { try { var image = yield download(url) image.show() } catch(e) { console.log("Cannot show image") }})

What is async() ?

https://github.com/kriskowal/q/tree/v1/examples/async-generators

http://pag.forbeslindesay.co.uk/#/22

// Implementation by Lindesay Forbesfunction async(makeGenerator){ return function (){ var generator = makeGenerator.apply(this, arguments) function handle(result){ // { done: [Boolean], value: [Object] } if (result.done) return result.value return result.value.then(function (res){ return handle(generator.next(res)) }, function (err){ return handle(generator.throw(err)) }) } return handle(generator.next()) }}

async is too complex

Generators support

http://kangax.github.io/compat-table/es6/#Generators_(yield)

I think this is the way

or there is the dark way

inventing your own concurrency

modelJoe Armstrong

it will be leaky

Still you can buildyour little paradise

Functional compositionwith Async.js

https://github.com/caolan/async

CaolanMcmahon

functions in Async.js

function(callback) { download(url, function() { callback() })}

error handling the node.js way

function(callback) { download(url, { success: function(result) { // success, null error then result callback(null, result) }, error: function(err) { // failure, error and no result callback(err) } })}

The Pyramid of Doom

refuel(function() { startEngine(function() { takeOff(function() { land(function() { // ... done }) }) })})

Demolished withsequential composition

async.series([ refuel, startEngine, takeOff, land], function(err, result) { // done!})

Pretty control flow

images.forEach(function(url) { download(url, function(image) { image.show() })})

Prettyfunctional composition

async.map(images, function(callback) { download(url, function(image) { callback(null, image) })}, function(err, results) { results.forEach(function(image) { image.show() })})

Composition of composition

async.waterfall([ function(callback) { async.map(files, fs.readFile, callback) }, function(contents, callback) { async.map(contents, countWords, callback) }, function(countedWords, callback) { async.reduce(countedWords, 0, sum, callback) },], function(err, totalWords) { // The number of words in files is totalWords})

Composition of composition is the

real power

Functionallack of state is the

weakness

No decoupling

async.series([ refuel, startEngine, takeOff, land], function(err, result) { // done!})

call start

callbackbinding

async.series([ refuel, startEngine, takeOff, land], function(err, result) { // done! share = result})

globalvariableto share

No decoupling

async.series([ refuel, startEngine, takeOff, land], function(err, result) { // done! logger.log(err) stats.update(result) spinner.hide()})

divergentchange

No decoupling

No decouplingis bad

We need first classasynchronous calls

to keep the stateof the computation

OOP compositionwith Promises

Promise is an objectrepresenting an

async computation

var promise = new Promise(function(resolve, reject) { // do asynchronous computation ... if (succeeded) { resolve(result) } else { reject(new Error(err)) }})

http://promisesaplus.com

https://www.promisejs.org/

The computation will eventually produce

an outcome

var promise = new Promise(function(resolve, reject) { // do asynchronous computation ... if (succeeded) { resolve(result) } else { reject(new Error(err)) }})

http://promisesaplus.com

https://www.promisejs.org/

Promise hasstates

var promise = new Promise(function(resolve, reject) { // do asynchronous computation ... if (succeeded) { resolve(result) } else { reject(new Error(err)) }})

pending

fulfilled

rejected

Promise is thenable

promise.then( function(result) { // promise fulfilled }, function(err) { // promise rejected })

Promise remembers its final state

promise // after some time ... .then(function(result) { // fulfilled, result is "hello" })

// after a while ...

promise .then(function(result) { // fulfilled, result still "hello" })

Promise remembers its final state

promise // after some time ... .then(function(result) { // fulfilled, result is "hello" })

// after a while ...

promise .then(function(result) { // fulfilled, result still "hello" })

Thenable are chainable

promise .then(function() { /* do something */ }) .then(function() { /* do something */ }) .then(function() { /* do something */ }) .then(function() { /* do something */ })

Sequential composition

promise .then(refuel) .then(startEngine) .then(takeOff) .then(land)

Pyramid demolished

promise .then(refuel) .then(startEngine) .then(takeOff) .then(land)

Promisify

function after(time) { return new Promise(function(resolve) { setTimeout(resolve, time) })}

after(5000) .then(function() { // five seconds gone! })

Promise propagation

promise .then(function() { /* called immediately */ }) .then(function() { /* called immediately */ })

Promise propagation

promise .then(function() { return after(5000) })

// returns a promise .then(function() { /* called after 5 secs */ }) .then(function() { /* called after 5 secs */ })

Promise propagation

promise .then(function() { return 5 })

// returns a value .then(function(result) { /* result == 5 */ })

Promise propagationpromise .then(after(5000)) .then(function() { throw new Error("argh") })

// throws an error .then(function() { /* never called */ }) .then(null, function(err) {

// err == Error(“argh”)})

Lovely error handling

promise .then(refuel) .then(startEngine) .then(takeOff) .then(land) .catch(function(err) { // deal with err })

More compositionvar one = after(1000).then(function() { return 1 })var two = after(2000).then(function() { return 2 })

// parallel, wait for allPromise.all([one, two]).then(function(result) { // after 2 seconds // result == [1, 2]})

// parallel, wins the firstPromise.race([one, two]).then(function(result) { // after 1 second // result == 1})

Stateful joy

Promises have limited vision

Promises have limited vision

setInterval(function() { // does success/error make sense? // what about next intervals ??}, 1000)

Promises have limited vision

$button.on("click", function() { // does success/error make sense? // what about next events ??})

Streams are weird beasts to promises

Reactive programming

https://github.com/Reactive-Extensions/RxJS

Eric Meijer

Reactive programming

Observables are collections over time

Iterators are pull

iterator.next() // => valueiterator.next() // => valueiterator.next() // => value

// it can return an erroriterator.next() // => Error("argh")

// it can enditerator.hasNext() // => falseiterator.next() // => null

observable.subscribe( function(value) { // next value }, function(err) { // failure }, function() { // completed })

Observables are push

Observables can dosetInterval

interval(1000).subscribe( function(value) { // value == undefined // value == undefined // ... }, function(err) { // never called }, function() { // when the interval is cleared })

Observables can doevents

click("#button").subscribe( function(value) { // value == { x: 458, y: 788 } // value == { x: 492, y: 971 } // ... }, function(err) { // never called }, function() { // when click is unsubscribed })

Observables can doasync calls

download(url).subscribe( function(image) { // once! // image == Image object }, function(err) { // once! // if error }, function() { // once! // when either succeeded or failed }

Observables areall-around

OOP structurefunctional compositionvar mouseup = Rx.Observable.fromEvent(dragTarget, 'mouseup')var mousemove = Rx.Observable.fromEvent(document, 'mousemove')var mousedown = Rx.Observable.fromEvent(dragTarget, 'mousedown')

var mousedrag = mousedown.selectMany(function (md) {

// calculate offsets when mouse down var startX = md.offsetX, startY = md.offsetY

// Calculate delta with mousemove until mouseup return mousemove.select(function (mm) { return { left: mm.clientX - startX, top: mm.clientY - startY } }).takeUntil(mouseup)})

// Update positionvar subscription = mousedrag.subscribe(function (pos) { dragTarget.style.top = pos.top + 'px' dragTarget.style.left = pos.left + 'px'})

Powerful model

Used at scale byNetflix

https://github.com/Netflix/RxJava/wiki

Take Away

Blockif you can

Choose the concurrencymodel that

fits you

@federicogalassi