1
Lecture 12
Implementing Small Languagesinternal vs. external DSLs, hybrid small DSLs
Ras Bodik Ali and Mangpo
Hack Your Language!CS164: Introduction to Programming Languages and Compilers, Spring 2013UC Berkeley
Overview
Implementation of DSLs
External vs. internal DSLsinternal == embedded in the host language
Deep embedding
Shallow embedding
Examples
2
Styles of DSL implementations
External DSL- own syntax (and parser)- dedicated interpreter
Internal DSL: deep embedding in the host language
- deep embedding: program exists as data (eg AST)
- the host language implements an interpreter
Internal DSL: shallow embedding in the host language
- shallow embedding: DSL constructs are composed purely of host language constructs
- we are using the interpreter of the host language
note: instead of interpreter, we can use a compiler
3
Architecture of external-DSL implementations
External DSL is a standalone language- just like “big” languages, Java, Python, C, …- domain programming abstractions with own
syntax
Similar interpreter/compiler architecture parser, optional analysis of AST, interpretation/compilation
If compiled, DSL need not be translated into code
- instead, it can be translated to a data structure which will be used by some other program P
- yes, P can be viewed as an interpreter of the DSL, and the data structure is an “AST”
5
Examples
Interpreted external DSLs
External DSLs compiled to code
External DSLs compiled to a data structure
6
Embed your DSL into a host language
Shallow embedding: the new language is a library written in the host language
sometimes called a “framework”
When DSL is implemented as a library, we often don’t think of it as a language
even though it defines own abstractions and operations
But the library implementation goes very far
8
Example 1: sockets
Sockets are library with own abstractions and rules
Socket f = new Socket(mode)f.connect(ipAddress)f.write(buffer)f.close()
Programming abstractions:socket, address, buffer
Usage rules:- do not write into a closed socket- do not close a socket twice
9
Usage rules (side note)
How abstractions can be combined is part of the language.
With network sockets, these are enforced at runtime:
ex: sock.write() checks that socket has been open
This is analogous to dynamic (runtime) type checking
ex: in Python, 1 + “a” will fail at runtime, when types of 1 and “a” are checked in the + operation
10
Usage rules (side note)
Could we enforce these socket rules at compile time?
This would be analogous to static type checking.
ex: in Java, int i; String s; i+s; fails at compile time even knowing what specific values i and s will take at runtime.
11
Usage rules (side note)
Could we enforce these socket rules at compile time?
Yes, use static types (as in say Java) and refine Socket into Socket, OpenSocket, ClosedSocket
How do we rewrite this code to use refined types?
Socket f = new Socket(mode)f.connect(ipAddress)f.write(buffer)f.close()
12
Example 2: regex matching
The library defines:
i) functions that DSL programmers use to define a pattern
ex: pattern ("abc"|"de")."x" can be defined as follows:
def patt = seq(alt(prim("abc"),prim("de")),prim("x"))
ii) a function matches of a string against a pattern:
match(string, patt)
13
Example 3:
rfig: formatting DSL embedded into Ruby.see slide 8 in
http://cs164fa09.pbworks.com/f/01-rfig-tutorial.pdf
14…
The animation in rfig, a Ruby-based language
slide!('Overlays',
'Using overlays, we can place things on top of each other.', 'The pivot specifies the relative positions', 'that should be used to align the objects in the overlay.',
overlay('0 = 1', hedge.color(red).thickness(2)).pivot(0, 0),
staggeredOverlay(true, # True means that old objects disappear 'the elements', 'in this', 'overlay should be centered',
nil).pivot(0, 0),
cr, pause, # pivot(x, y): -1 = left, 0 = center, +1 = right
staggeredOverlay(true, 'whereas the ones', 'here', 'should be right justified',
nil).pivot(1, 0), nil) { |slide| slide.label('overlay').signature(8) }
15
DSL as a framework
It may be impossible to hide plumbing in a procedure
these are limits to procedural abstraction
We can use advanced language features, such as closures, coroutines
Framework, a library parameterized with client code• typically, you register a function with the library• library calls this client callback function at a
suitable point• ex: an action to perform when a user clicks on
DOM node
16
Example 4: jQuery
Before jQuery
var nodes = document.getElementsByTagName('a'); for (var i = 0; i < nodes.length; i++) { var a = nodes[i]; a.addEventListener('mouseover', function(event)
{ event.target.style.backgroundColor=‘orange'; }, false ); a.addEventListener('mouseout', function(event)
{ event.target.style.backgroundColor=‘white'; }, false ); }
jQuery abstracts iteration and events
jQuery('a').hover( function() { jQuery(this).css('background-color', 'orange'); }, function() { jQuery(this).css('background-color', 'white'); } );
17
Embedding DSL as a language
Hard to say where a framework becomes a language
not too important to define the boundary precisely
Rules I propose: it’s a language if 1) its abstractions include compile- or run-time
checks --- prevents incorrect DSL programsex: write into a closed socket causes an error
2) we use syntax of host language to create (an illusion) of a dedicated syntax
ex: jQuery uses call chaining to pretend it modifes a single object: jQuery('a').hover( … ).css( …)
18
rake
rake: an internal DSL, embedded in RubyAuthor: Jim Weirich
functionality similar to make– has nice extensions, and flexibility, since it's
embedded– ie can use any ruby commands
even the syntax is close (perhaps better):– embedded in Ruby, so all syntax is legal Ruby
http://martinfowler.com/articles/rake.html19
Example rake file
task :codeGen do # do the code generationend
task :compile => :codeGen do # do the compilationend
task :dataLoad => :codeGen do # load the test dataend
task :test => [:compile, :dataLoad] do # run the testsend 20
How is rake legal ruby?
Deconstructing rake (teaches us a lot about Ruby):
task :dataLoad => :codeGen do # load the test dataend
task :test => [:compile, :dataLoad] do # run the testsend
22
Two kinds of rake tasks
File task: dependences between files (as in make)
file 'build/dev/rake.html' => 'dev/rake.xml' do |t|
require 'paper' maker = PaperMaker.new t.prerequisites[0], t.name
maker.runend
23
Two kinds of tasks
Rake task: dependences between jobs
task :build_refact => [:clean] do target = SITE_DIR + 'refact/' mkdir_p target, QUIET require 'refactoringHome' OutputCapturer.new.run {run_refactoring}end
24
Rake can orthogonalize dependences and rules
task :second do #second's bodyend
task :first do #first's bodyend
task :second => :first
25
General rules
Sort of like make's %.c : %.o
BLIKI = build('bliki/index.html')
FileList['bliki/*.xml'].each do |src|file BLIKI => src
end
file BLIKI do #code to build the blikiend
26
Deep embedding
Represent the program as an AST or with some other data structure
This AST can be interpreted or compiled
28
Acknowledgements
This lecture is based in part on
Martin Fowler, “Using the Rake Build Language”
Jeff Friedl, “Mastering Regular Expressions”
32