the role of template engines in code generation terence parr university of san francisco...
TRANSCRIPT
The Role Of The Role Of Template Engines in Template Engines in
Code GenerationCode GenerationTerence ParrTerence Parr
University of San FranciscoUniversity of San Franciscoparrtparrt@[email protected]@[email protected]
IntroductionIntroduction Car aerodynamics mostly trumps style todayCar aerodynamics mostly trumps style today Similarly, the nature of generating text should Similarly, the nature of generating text should
dominate design decisions: use an output dominate design decisions: use an output grammargrammar
Don’t have output grammars, we have programs Don’t have output grammars, we have programs with print statements; template engines arose to with print statements; template engines arose to encourage separation of logic/display.encourage separation of logic/display.
Enforcing strict separation also leads to similar Enforcing strict separation also leads to similar grammar-like mechanismgrammar-like mechanism
Conclusion: if you’re generating text, you should Conclusion: if you’re generating text, you should be using something akin to StringTemplatebe using something akin to StringTemplate
OutlineOutline
Define and motivate model-view Define and motivate model-view separation; give enforcement rules, separation; give enforcement rules, show equivalence to CFGsshow equivalence to CFGs
Describe StringTemplate, provide Describe StringTemplate, provide exampleexample
Relate experience building ANTLR Relate experience building ANTLR 3.0 code generator with 3.0 code generator with StringTemplateStringTemplate
HTML GenerationHTML Generationout.println("<html>");out.println("<body>");out.println("<h1>Servlet test</h1>");String name = request.getParameter("name");out.println("Hello, "+name+".");out.println("</body>");out.println("</html>");
<html><body><h1>JSP test</h1>Hello, <%=request.getParameter("name")%>.</body></html>
Servlet:
JSP:
Example EntanglementsExample Entanglements $if(user==“parrt” && machine=“yoda”)$$if(user==“parrt” && machine=“yoda”)$ $price*.90$, $bloodPressure>130$$price*.90$, $bloodPressure>130$ $a=db.query(“select subject from email”)$$a=db.query(“select subject from email”)$ $model.pageRef(getURL())$$model.pageRef(getURL())$ $ClassLoader.loadClass(somethingEvil)$$ClassLoader.loadClass(somethingEvil)$ $names[ID]$$names[ID]$
st.setAttribute(“color”, “Red”);st.setAttribute(“color”, “Red”);
Motivation For Motivation For SeparationSeparation
EncapsulationEncapsulation ClarityClarity Division of laborDivision of labor Component reuseComponent reuse Single point-of-changeSingle point-of-change MaintenanceMaintenance Interchangeable views, retargetingInterchangeable views, retargeting SecuritySecurity
Existing EnginesExisting Engines ProblemProblem: engines don’t : engines don’t enforceenforce separation, separation,
they only they only encourageencourage separation (Murphy, separation (Murphy, IBM keybrd, etc…)IBM keybrd, etc…)
ReasonReason: engine builders and users fear that : engine builders and users fear that enforcement implies fatal weaknessenforcement implies fatal weakness
ResultResult: developers exploit loopholes, : developers exploit loopholes, encoding logic in templates, thus, entangling encoding logic in templates, thus, entangling model/viewmodel/view
We can enforce separation without We can enforce separation without emasculating the power of a template engine emasculating the power of a template engine empirical evidence and theoretical supportempirical evidence and theoretical support
Template DefinitionTemplate Definition
Unrestricted template:Unrestricted template:tt00ee00…t…tiieeiitti+1i+1…t…tnneemm
where where ttii is a literal and is a literal and eeii is is unrestricted computationally and unrestricted computationally and syntacticallysyntactically
Notes:Notes: unrestricted templates do not enforce unrestricted templates do not enforce
separationseparation XSLT is not a template engineXSLT is not a template engine
Rules of SeparationRules of Separation
1.1. the view cannot modify the modelthe view cannot modify the model
2.2. cannot perform computations upon cannot perform computations upon dependent data valuesdependent data values
3.3. cannot compare dependent data cannot compare dependent data valuesvalues
4.4. cannot make type assumptionscannot make type assumptions
5.5. data from model cannot contain data from model cannot contain display, layout informationdisplay, layout information
Restricted TemplatesRestricted Templates Restrict template to operate on read-only data Restrict template to operate on read-only data
values, attributes (single or multi-valued), to values, attributes (single or multi-valued), to prevent side-effectsprevent side-effects
eei i are attribute or template referencesare attribute or template references Even restricted templates can generate the Even restricted templates can generate the
context-free languagescontext-free languages By allowing conditional inclusion (predicates), By allowing conditional inclusion (predicates),
reaches into context-sensitive languagesreaches into context-sensitive languages XML DTDs are essentially CFGs, therefore, XML DTDs are essentially CFGs, therefore,
restricted template can generate syntax of any restricted template can generate syntax of any XML documentXML document
Equivalence to CFGsEquivalence to CFGs
Attributes = terminals, templates = Attributes = terminals, templates = rulesrules
Can show grammar’s derivation tree Can show grammar’s derivation tree for any sentence maps to a nested for any sentence maps to a nested template tree structuretemplate tree structure
prog : decl func ;decl : type ID ;func : type ID “()” “{“ body “}”…
prog ::= “<decl()> <func()>”decl ::= “<type> <ID> ;”func : << <type> <ID>() { <body()> }>>…
Grammar Template
StringTemplateStringTemplate Evolved from simple “document with holes” Evolved from simple “document with holes”
while building jGuru.com, after dumping JSPwhile building jGuru.com, after dumping JSP Side-effect free expressionsSide-effect free expressions No order of evaluationNo order of evaluation Recursion (recall output structures are nested)Recursion (recall output structures are nested) Dynamic scopingDynamic scoping Lazy-evaluationLazy-evaluation Template inheritance/polymorphismTemplate inheritance/polymorphism Simple enough for nonprogrammersSimple enough for nonprogrammers Strictly enforces separation of model/viewStrictly enforces separation of model/view No assignments, loops, …No assignments, loops, …
Canonical OperationsCanonical Operations
Attribute reference:Attribute reference:<type><type>
Template references (possibly recursive):Template references (possibly recursive):<statementList()><statementList()>
Apply template to multi-valued attribute:Apply template to multi-valued attribute:<decls:decl()><decls:decl()> oror<decls:{<it.type> <it.name>;}><decls:{<it.type> <it.name>;}>
Conditional include:Conditional include:<if(superClass)>extends <superClass><endif><if(superClass)>extends <superClass><endif>
Template GroupsTemplate Groups
Set of mutually-referential templates Set of mutually-referential templates with formal argumentswith formal arguments
group javaTemplates;
method(type,name,args,body) ::= <<public <type> <name>( <args:arg(); separator=“,”> ) { <body>}>>assign(lhs,expr) ::= “<lhs> = <expr>;”if(expr,stat) ::= “if (<expr>) <stat>”call(name,args) ::= “<name>( <args; separator=“,”> );”…
Template PolymorphismTemplate Polymorphism
Output: “Output: “y=1;y=1;” not “” not “x=1;x=1;” because ” because template instance’s group is template instance’s group is subGroupsubGroup
sub.setSuperGroup(sup);StringTemplate st = sub.getInstanceOf(”slist");System.out.println(st.toString());
Late bind
group sup;slist() ::= “<assign()>”assign() ::= “x=1;”
group sub;assign() ::= “y=1;”
Group determines symbol resolution
Example: Dump Java Example: Dump Java ClassClass
Expected output:Expected output:
class Dump { public int i; public java.lang.String name; public int[] data; public void main(class java.lang.String[] arg1); public void foo(int arg1, float[] arg2); public class java.lang.String bar();}
Dump Java Class Dump Java Class TemplatesTemplatesgroup Java;
class(name,fields,methods) ::= <<class <name> { <fields:field(); separator="\n"> <methods:method()>}>>
field() ::= "public <type(t=it.type)> <it.name>;"
method() ::= <<public <it.returnType> <it.name> (<it.parameterTypes:{<type(t=it)> arg<i>}; separator=", ">);>>
type(t) ::= <<<if(t.componentType)><t.componentType>[]<else><t.name><endif>>>
Dump Java Class Code Dump Java Class Code public class Dump { public int i; public String name; public int[] data; public static void main(String[] args) throws IOException { StringTemplateGroup group = new StringTemplateGroup(new FileReader("Java.stg"), AngleBracketTemplateLexer.class); Class c = Dump.class; Field[] fields = c.getFields(); Method[] methods = c.getDeclaredMethods(); StringTemplate classST = group.getInstanceOf("class"); classST.setAttribute("name", c.getName()); classST.setAttribute("fields", fields); classST.setAttribute("methods", methods); System.out.println(classST); } public void foo(int x, float[] y) {;} public String bar() {return "";}}
Dump XML InsteadDump XML Insteadgroup XML;
class(name,fields,methods) ::= <<<class> <name>$name$</name> $fields:field()$ $methods:method()$</class>>>
field() ::= <<<field> <type>$type(t=it.type)$</type><name>$it.name$</name></field>>>
…
<class> <name>Dump</name> <field> <type>int</type><name>i</name> </field> …</class>
Experience with ANTLR Experience with ANTLR 3.03.0
Tree walker (controller) collects data from Tree walker (controller) collects data from AST (model), pushes data into templates AST (model), pushes data into templates (view)(view)
Decouples order of computation from order of Decouples order of computation from order of display (this is huge)display (this is huge)
Enforced separation guarantees easy Enforced separation guarantees easy retargeting, no code duplication, …retargeting, no code duplication, … no code in templateno code in template no output strings in code generatorno output strings in code generator Previous code generator hopelessly entangledPrevious code generator hopelessly entangled
Group file format (output grammar) is great! Group file format (output grammar) is great! “Executable documentation”“Executable documentation”
Sample ANTLR 3.0 Sample ANTLR 3.0 TemplateTemplate
parser(name, tokens, rules, DFAs) ::= <<class <name> extends Parser { <tokens: {public static final int <it.name>=<it.type>;} > public <name>(TokenStream input) { super(input); }
<rules; separator="\n">
<DFAs>}>>
SummarySummary The nature of text generation and the The nature of text generation and the
enforcement of model-view separation enforcement of model-view separation dominate tool design-decisions:dominate tool design-decisions: tools should resemble output grammarstools should resemble output grammars
StringTemplate is a simple template StringTemplate is a simple template engine that evolved while building engine that evolved while building dynamic sites. It is proving exceptionally dynamic sites. It is proving exceptionally well suited to code generation tasks well suited to code generation tasks including ANTLR 3.0including ANTLR 3.0
open-source Java, BSD license (also a C# open-source Java, BSD license (also a C# port) port) http://www.stringtemplate.orghttp://www.stringtemplate.org