How to make a new language and make the world better

...or, let's talk about eDSLs

Greg Wengabout.me/snowmantw

snowmantw@gmail.com

bit.ly/edsl-intro

Make a new programming language? Why?

Make a new programming language? Why?

You may have heard somebody claim that we already have TOO MUCH programming languages

in the world

Make a new programming language? Why?

var i, divs = document.getElementsByTagName('div');for(i = 0; i < divs.length; i++) { divs[i].onclick = function() { this.style.backgroundColor = 'red'; }}

var nextElement = document.getElementById("wrap").nextSibling;

var map_r = [ ];for( var i = 0; i < foodParcel; i++) { map_r[i] = foodParcel[i].contents.split(',') }

var flattern_r = [ ];// Omit annoying flattern code...

var reduce_r = 0;// ...

$('div').click(function() { $(this).css('background-color', 'red');});

var nextElement = $("#wrap").next();

_(foodParcel).chain() .map(function(type) { return type.contents.split(','); }) .flatten() .reduce(function(counts, item) { counts[item] = (counts[item] || 0) + 1; return counts; }, {}).value();

From this...

...To this

IMHO, programmer's life always become better and better with every single new language

Assembly

JavaScript

C++ JavaC

HaskellPython

Hey! They're LIBRARIES, not LANGUAGES!

var i, divs = document.getElementsByTagName('div');for(i = 0; i < divs.length; i++) { divs[i].onclick = function() { this.style.backgroundColor = 'red'; }}

var nextElement = document.getElementById("wrap").nextSibling;

var map_r = [ ];for( var i = 0; i < foodParcel; i++) { map_r[i] = foodParcel[i].contents.split(',') }

var flattern_r = [ ];// Omit annoying flattern code...

var reduce_r = 0;// ...

$('div').click(function() { $(this).css('background-color', 'red');});

var nextElement = $("#wrap").next();

_(foodParcel).chain() .map(function(type) { return type.contents.split(','); }) .flatten() .reduce(function(counts, item) { counts[item] = (counts[item] || 0) + 1; return counts; }, {}).value();

From this...

...To this

Not really.They're actually eDSLs,

not only libraries.

embedded DSL means

"...implemented as libraries which exploit the syntax of their host general purpose language or a subset thereof, while adding domain-specific language elements (data types, routines, methods, macros etc.)."

From Wikipedia (eDSL)

You might already used some of these eDSLs ...

$('#message').val("Winston Smith...").fadeOut('slow').hide( ).val("Big Brother Is Watching You").css('font-color', 'red').show( )

var $message = document.getElementById('message')$message.value = "Winston Smith..."fadeOut($message, 'slow')hide($message)$message.value = "Big Brother is Watching You"$message.style.frontColor = 'red'show($message)

jQuery

You might already used some of these eDSLs ...

var stooges = [{name : 'curly', age : 25}, {name : 'moe', age : 21}, {name : 'larry', age : 23}]

var youngest = _.chain(stooges) .sortBy(function(stooge) { return stooge.age }) .map(function(stooge) { return stooge.name + ' is ' + stooge.age }) .first().value();

var stooges = [{name : 'curly', age : 25}, {name : 'moe', age : 21}, {name : 'larry', age : 23}]

stooges.sort( function(stooge) { return stooge.age } )var sorted = [ ]stooges.forEach( function(e,i,x) { result[i] = e.name + 'is' + e,age } )var yougest = sorted[0]

underscore.js

You might already used some of these eDSLs ...

query.from(customer) .orderBy(customer.lastName.asc() ,customer.firstName.asc()) .list(customer.firstName ,customer.lastName);

// Well, I don't want to handle SQL strings in Java // manually...// The left will generate SQL like this:

SELECT c.first_name, c.last_nameFROM customer c ORDER BY c.last_name ASC, c.first_name ASC

LINQ (Java porting)

You might already used some of these eDSLs ...

select $from $ \(s, e) -> dowhere_ (s ^. StockId ==. e ^. EndOfDayStockId &&. s ^. StockTicker ==. val ticker &&. s ^. EndOfDayTradeDate ==. val stockDate)return (e ^. EndOfDayClosingPrice, e ^. EndOfDayTradeDate)

SELECT end_of_day.closing_price, end_of_day.trade_dateFROM stock, end_of_dayWHERE stock.stock_id = end_of_day.stock_id AND (stock.ticker = ? AND end_of_day.trade_date = ?)

esqueleto(Haskell)

You might already used some of these eDSLs ...

var mtxA = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]var mtxB = [ [ -1], [ 0], [ 1] ]

var mtxC = mul( mtxA, mtxB)

var mtxA = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]var mtxB = [ [ -1], [ 0], [ 1] ]

var mtxC = mul( mtxA, mtxB)

Matrix Manipulations

eDSLs may (or may not) make your code shorter and more elegant.

But the most important thing is it helps you to focus on the current domain problem with right tools.

$('#message').val("Winston Smith...").fadeOut('slow').hide( ).val("Big Brother Is Watching You").css('font-color', 'red').show( )

var $message = document.getElementById('message')$message.value = "Winston Smith..."fadeOut($message, 'slow')hide($message)$message.value = "Big Brother is Watching You"$message.style.frontColor = 'red'show($message)

eDSLs and their domain problems

jQuery DOM manipulation

Underscore.js Computation

LINQ Querying

esqueleto Database Querying

Matrix Manipulations

Arithemetic (Matrix)

My small but useful (?) eDSL

Gist: bit.ly/sntw-mtx

mtx( 1, 2, 3) ( 4, 5, 6) ( 7, 8, 9) (10,11,12)..mul(11,12,13,14) (14,15,16,15) (17,18,19,16).mul( 3) ( 4) ( 5) ( 6).get()

1. Because using Arrays to represent Matrices is too mainstream.

2. You don't need the stupid [[outer] [brackets]] anymore!

3. To test my "inifinite curry" in this example.

Syntax

Well, it might be more formal and like a real language if we can show

something hard to understand...

jsprog := JSStatements prog JSStatementsprog := mtx manipulations getmtx := mtx rowsrows := row rows | rowrow := ( JSNumber, ..., JSNumber )get := .get ()mul := .mul rowsmanipulations := mul

/* Can add more manipulations if we need */

And the bigger one

GitHub: bit.ly/fluorine-js

Motivationtry to implement tolerable Monad-like

mechanism in JavaScript, and make it more functional

The triangle relation between following 3 eDSLs:

Or at least its my goal...

jQueryUI, IO and other computations with side-effects

Underscore.jsPure computations like map-reduce

FluorineIsolate impure computation and combine them with pure ones reasonably.

The triangle relation between following 3 eDSLs:

Or at least its my goal...

// Data -> UI DOMfunction recover(dataset){ return $('#form-wrapper') .find('#cover') .hide() .end() .find('#form') .find('input') .each(/*fill dataset in*/) .end() .fadeIn('slow') .end()}

// Use function as argument.fetch_data(URL, recover)

// URL -> IO Datafunction fetch_data(url, cb){ let parse = function(html,cb) { $vals = $(html) .find('input') .val() return _ .chain($vals) .map(/*...*/) .reduce(/* ... */) .value() cb($vals) } $.get(url, parse) // IO is async}

Impure

Pure

The triangle relation between following 3 eDSLs:

Or at least its my goal...

// Data -> UI DOMfunction recover(dataset){ return $('#form-wrapper') .find('#cover') .hide() .end() .find('#form') .find('input') .each(/*fill dataset in*/) .end() .fadeIn('slow') .end()}

// Use function as argument.fetch_data(URL, recover)

// URL -> IO Datafunction fetch_data(url, cb){ let parse = function(html,cb) { $vals = $(html) .find('input') .val() return _ .chain($vals) .map(/*...*/) .reduce(/* ... */) .value() cb($vals) } $.get(url, parse) // IO is async}

// URL -> IO DOMvar recover = IO(URL) .get() .tie(function(html) { return UI() .find('input') .val() }) ._(parse) // IO HTML -> IO Data .tie(recover) // IO Data -> IO DOM .idgen()

// recover :: ( ) -> IO DOM// recover():: IO DOMvar main = recover()

// Run the "Monad", runIOmain()

JavaScript lacks these features to become more "functional"

PDF: bit.ly/js-fun

From one of my representations in JS Group

1. (Has) First class function & anonymous function2. (Could) Curry & Partial Application3. (Poorly) Supports recursion

4. (Isn't) Pure5. (Isn't) Lazy

The most lethal one is that you can't isolate impure code as nature as in Haskell.

With Fluorine you can:

GitHub: bit.ly/fluorine-js

1. Isolate impure parts in the program

2. Mix pure/impure when necessary

3. Flow-Control, to avoid callback hell

4. Laziness (well, sort of)

(Cheating) Demo

GitHub: bit.ly/fluorine-js

Principles

GitHub: bit.ly/fluorine-js

1. Define impure parts with contexts

2. foo().bar() ≌ foo >>= bar

3. Don't extract contexts unless necessary

4. Use generators wisely

5. "Making Wrong Code Look Wrong"

Some implementation details that might interest you

UI context

Customized Process

initialize

Step #1

Step #2

Step #3

Step #4

......

done

Process The type of our contexts are actually

m (Process a)

rather than

m a

They all have implicit process

UI context

Customized Process

initialize

Step #1

Step #2

Step #3

Step #4

......

done

Process

IO context

initialize

Step #1

get

tie

post

......

done

Process

flattern callback hell reasonably

Process helps us

(Pure code never goes async)

Customized ProcessIO context

initialize

Step #1

get

tie

post

......

done

Process You can extract result from UI and other context (remember Maybe or List?)

var foo = IO().get()....done()().extract()// Don't do this.

Because of its internal aschronous process may return wrong value

However, IO is not "co-context"You should never extract things from IO

Customized ProcessIO context

initialize

Step #1

get

tie

post

......

done

Process For example, this is legal and safe:

var foo = IO().get()....done()().extract()

However, you should never:

var foo = UI('#foo').$().done()().extract()

m = head ([1,2,3] >>= return.odd)

Just like

Just like

m = unsafePerformIO (getChar >> getChar)

Customized ProcessIO context

initialize

Step #1

get

tie

post

......

done

Process In Haskell, things should never escape from IO monad due to the side-effects

It's occasionally true in Fluorine. And we both have very good reasons to say that

The definition and running stageIO context

initialize

Step #1

get

tie

post

......

done

Process IO.o.prototype.get

definition

➔ Setup the step from the context

➔ Push the step to the process (runtime stack)

➔ Return this to keep chaining

running

➔ Pop one step from the stack

➔ Execute it with the environment

➔ Capture its return value and pass to the next

➔ Call next when this one is done

The definition and running stageIO context

initialize

Step #1

get

tie

post

......

done

Process This stage would capture information from the context to create the step

It's also possible to do more check before we enter the running stage

IO().get('/todo').tie(renew_ui).post('/ok') .done()

For example, typed JavaScript?

Thus we don't have compilation time,we have definition time

The definition and running stageIO context

initialize

Step #1

get

tie

post

......

done

Process The key is "to call the next when this one is done" in runtime.

Thus, the |get| step can call the next |tie| step only after its remote request has returned success.

This empower us to make customized binding function, just like the different>>= among Haskell monads.

TyingIO context

initialize

Step #1

get

tie

post

......

done

Process Tying means you tied another context into the current one

In theory, it's very simple: just push another context generator into the stack

It's an unsuccessful attempt to mimic the >>= function in Monad Transformer

tie:: m a -> (a -> n b) -> m b

>>=:: M m a -> (a -> M m b) -> M m b

TyingIO context

initialize

Step #1

get

tie

post

......

done

ProcessNote the tied context would take over the control of the whole process:

IO().get('/todo') .tie(function(todos) { return Event('user-request') .tie(ui_render(todos)) .done() }) .post('/ok') .done()

The |post| won't be executed til the event |user-request| fired, because it's Event's default behavior

Something about Lint(s)

Lint hates my eDSL...

Demo

What will you get and lose if you force your eDSL compatible with harsh Lint(s)

Too bad...

1. You CAN'T figure out what's going wrong at first look

2. You CAN'T indetify symbols and strings anymore

3. Damn semicolons in a language needn't them

Here are you only choices...

To be or not to be, that is the question

(Hard work) Make your own Lint

(Bad) Ignore the Lint

Conclusion

Languages which allow AMAP ways to say the same thing arenot guilty

Instead of, they will become useful and graceful Swiss knifes

Of course you must provide a serious tool, not dangerous IEDs

Of course you must provide a serious tool, not dangerous IEDs

Remember, you are desinging a REAL programming language !

In my (painful) experience:

1. debugger, debugger, debugger2. useful examples3. simple introducation & detailed API/technical documents4. eat your own dog food

Even one of those math-like most language allows users create and use eDSL

JavaScript, Ruby, Java...

Why forcing people to drive a car only as a car, when it's actually a Transformer ?

Make your own language!