meta programming with streams and pipes

  Essentially, programs that can be created by themselves.

  We’re twisting the term a little for streams

  We will be defining streaming constructs using streaming constructs

  By streams I mean beams which are just streams and pipes for compute as opposed to IO

Any info? Please contact #nodeland police at @cianomaidin on twitter…

  Readable Streams   IO / throughput oriented

  Events – data, end, error, close

  Methods – pause, resume, end, destroy

  Writeable Streams   IO / throughput oriented

  Events - drain, pause, resume, error, close

  Methods – write, end, destroy

  IO Oriented

  Finite, not infinite

  Asynchronous

  Lazy

  Assumes in transit data, not in memory forms

  Great for IO. Not efficient for compute

  Sources   Where data pours in

  Sinks   Where results pop out

  Operators   Filters – Drop or pass on data based on a UDF   Transforms – Mutate data based on a UDF   Branch with Filters   Combine with Sinks

  Compute Oriented

  Infinite, not finite

  Synchronous

  Eager

  Assumes in memory form, not encoded or wire data

  Convenient for compute. Not good for IO

  Eager: a.pipe(operation).pipe(z).pipe(a);

  Implies:

RangeError: Maximum call stack size exceeded

  npm install beam   Compute oriented streams   Branch, Combine, Filter, Transform data   Extend with goto and modules.

  npm install eep   Beam will be integrated into eep soon   Eep provides aggregate windows   Slide, Tumble, Monotonic, Periodic   Stats Library   Ordering now supported

  npm install nodesapiens   Robosapiens V1 for node.js

  Code on github   https://github.com/darach