oodl runtime optimizations jonathan bachrach mit ai lab feb 2001
TRANSCRIPT
OODL Runtime Optimizations
Jonathan Bachrach
MIT AI Lab
Feb 2001
Runtime Techniques
• Assume can only write system code turbochargers– No sophisticated compiler available– Can only minimally perturb user code
Q: What are the Biggest Inefficiencies?
• Imagine trying to get Proto to run faster
Hint: Most Popular Operations
Running Example
(dg + ((x <num>) (y <num>) => <num>))
(dm + ((x <int>) (y <int>) => <int>) (%ib (%i+ (%iu x) (%iu y)))
(dm + ((x <flo>) (y <flo>) => <flo>) (%fb (%f+ (%fu x) (%fu y)))
(dm x2 ((x <num>) => <num>) (+ x x))(dm x2 ((x <int>) => <int>) (+ x x))
A: What are the Biggest Inefficiencies?
• Boxing• Method dispatch *• Type checks
• Slot access• Object creation
• * Today
Outline
• Overview
• Inline call caches
• Table
• Decision tree
• Variations
• Open Problems
Method Distributions
• Distribution can be measured– At generic– At call site
• Distribution can be– Monomorphic– Polymorphic– Megamorphic
• Distribution can be – peaked– uniform
Expense of Dispatch
• Problem: expensive if computed naively– Find applicable methods– Sort applicable methods– Call most applicable method– Three outcomes
• One most applicable method => ok
• No applicable methods => not understood error
• Many applicable methods => ambiguous error
Mapping View of Dispatch
• Dispatch can be thought of as a mapping from argument types to a method– (t1, t2, …, tn) => m
Solutions
• Caching
• Fast mapping
Table-based Approach
• N-dimensional tables– Keys are concrete classes of actual arguments– Values are methods to call– Must address size explosion– Talk a bit about this later
• Nested tables– Keys are concrete classes of actual arguments– Values are either other tables or methods to call
Table Example One
text(+ x y) textcache
gen +
#f
prog ram
Table Example Two
texttext(+ x y) textcache
gen +
prog ram
<int> <int> code texti+
i+
prog ram
x= 0 y= 0
Table Example Three
text
text(+ x y) textcache
+
prog ram
x= 0 .0 y= 0 .0
<int>
<int>
code i+
i+<flo>
<int>
textcode
f+
f+
prog ram
Table-based Critique
• Pros– Simple– Amenable to profile guided reordering
• Cons– Too many indirections– Very big
• demand build it• Sharing of subtables
– Only works for class types • can use multiple tables
Engine Node Dispatch
• Glenn Burke and myself at Harlequin, Inc.
circa 1996-– Partial Dispatch: Optimizing Dynamically-Dispatched Multimethod Calls
with Compile-Time Types and Runtime Feedback, 1998
• Shared decision tree built out of executable engine nodes
• Incrementally grows trees on demand upon miss
• Engine nodes are executed to perform some action typically tail calling another engine node eventually tail calling chosen method
• Appropriate engine nodes can be utilized to handle monomorphic, polymorphic, and megamorphic discrimination cases corresponding to single, linear, and table lookup
Engine Node Dispatch Picture
textcall
\+<i>,<i>m ethod
NAM
\+<f>,<f>m ethod
...discrim inator
...
<generic>
...
<f>
<i>
linear ep
<linear-engine>
...
< i>
m ono ep
<mono-engine>
...
<f>
m ono ep
<mono-engine>
...MEP
<method>
...MEP
<method>
...MEP
<method>
Define method \+ (x :: <i>, y :: <i>) … end;Define method \+ (x :: <f>, y :: <f>) … end;Seen (<i>, <i>) and (<f>, <f>) as inputs.
Engine Dispatch Critique
• Pros:
• Portable• Introspectable• Code Shareable
• Cons:
• Data and Code Indirections
• Sharing overhead• Hard to inline• Less partial eval opps
Lookup DAG
• Input is argument values• Output is method or error
• Lookup DAG is a decision tree with identical subtrees shared to save space
• Each interior node has a set of outgoing class-labeled edges and is labeled with an expression
• Each leaf node is labeled with a method which is either user specified, not-understood, or ambiguous.
Lookup DAG Picture•From Chambers and Chen OOPSLA-99
Lookup DAG Evaluation
• Formals start bound to actuals• Evaluation starts from root• To evaluate an interior node
– evaluate its expression yielding v and
– then search its edges for unique edge e whose label is the class of the result v and then edge's target node is evaluated recursively
• To evaluate a leaf node – return its method
Lookup DAG Evaluation Picture•From Chambers and Chen OOPSLA-99
Lookup DAG Constructionfunction BuildLookupDag (DF: canonical dispatch function): lookup DAG = create empty lookup DAG G create empty table Memo cs: set of Case := Cases(DF) G.root := buildSubDag(cs, Exprs(cs)) return G
function buildSubDag (cs: set of Case, es: set of Expr): set of Case = n: node if (cs, es)->n in Memo then return n if empty?(es) then n := create leaf node in G n.method := computeTarget(cs) else n := create interior node in G expr:Expr := pickExpr(es, cs) n.expr := expr for each class in StaticClasses(expr) do cs': set of Case := targetCases(cs, expr, class) es': set of Expr := (es - {expr}) ^ Exprs(cs') n': node := buildSubDag(cs', es') e: edge := create edge from n to n' in G e.class := class end for add (cs, es)->n to Memo return n
function computeTarget (cs: set of Case): Method = methods: set of Method := min<=(Methods(case)) if |methods| = 0 then return m-not-understood if |methods| > 1 then return m-ambiguous return single element m of methods
Single Dispatch Binary Search Tree
• Label classes with integers using inorder walk with goal to get subclasses to form a contiguous range
• Implement Class => Target Map as binary search tree balancing execution frequency information
Class Numbering
<any>
<a> <d>
<c><b> <e>
text0 1 2 3 4 5
Binary Search Tree Picture•From Chambers and Chen OOPSLA-99
Critique of Decision Tree
• Pros– Efficient to construct and execute– Can incorporate profile information to bias execution– Amenable to on demand construction– Amenable to partial evaluation and method inlining– Can easily incorporate static class information– Amenable to inlining into call-sites– Permits arbitrary predicates – Mixes linear, binary, and array lookups– Fast on modern CPU’s
• Cons– Requires code gen / compiler to produce best ones
Inline Call Caches
• Assumption: – method distribution is usually peaked and call-site
specific
• Each call-site has its own cache• Use call instruction as cache
– Calls last taken method– Method prologue checks for correct arguments– Calls slow lookup on miss which also patches call
instruction
• Deutsch and Schiffman, 1984
Inline Caching Example One
text(+ x y)
prog ram prog ram
look up
Inline Caching Two
text(+ x y)
prog ram prog ram
text
look up
x= 0 y= 0
(let ((tx (class-of x)) (ty (class-of y)) (unless (and (== tx <int>) (== ty <int>)) (tail-call lookup)))
... i+ code ...
i+
Inline Caching Three
text(+ x y)
prog ram prog ram
text
look up
x= 0 .0 y= 0 .0
(let ((tx (class-of x)) (ty (class-of y)) (unless (and (== tx <int>) (== ty <int>)) (tail-call lookup)))
... i+ code ...
i+
(let ((tx (class-of x)) (ty (class-of y)) (unless (and (== tx <flo>) (== ty <flo>)) (tail-call lookup)))
... f+ code ...
f+
Inline Caching Critique
• Pros– Fast dispatch sequence for hit– Usually high hit rate (90-95% for Smalltalk)
• Cons– Uses self-modifying code– Slow for misses– Depends on method distribution spike– Might be less beneficial for multimethods
Polymorphic Inline Caching
• Handles polymorphically peaked distribution
• Generate call-site specific dispatch stub
• Holzle et al., 1991
Polymorphic Inline CachingExample One
text(+ x y)
prog ram prog ram
look up
(lookup x y)
i+
f+
(i+ x y)
(f+ x y)
Polymorphic Inline CachingExample Two
text(+ x y)
prog ram prog ram
look up
(let ((tx (class-of x)) (ty (class-of y)) (if (and (== tx <int>) (== ty <int>)) (tail-call i+) (tail-call lookup)))
i+
f+
text
(i+ x y)
(f+ x y)
x= 0 y= 0
Polymorphic Inline CachingExample Three
text(+ x y)
prog ram prog ram
look up
(let ((tx (class-of x)) (ty (class-of y)) (if (and (== tx <int>) (== ty <int>)) (tail-call i+) (if (and (== tx <flo>) (== tx <flo>)) (tail-call i+) (tail-call lookup)))
i+
f+
(i+ x y)
(f+ x y)
x= 0 .0 y= 0 .0
Polymorphic Inline Cache Critique
• Pros– Faster for multiple peaked distributions
• Cons– Slow for uniform distribution– Requires runtime code generation– Doesn’t scale quite as well for multimethods
and predicate types
Other Multimethod Approaches
• Hash table indexed by N keys, – Kiczales and Rodriguez 1989
• Compressed N+1 dimensional dispatch table– Amiel et al. 1994– Pang et al. 1999
Variations
• Inline method bodies into leaves of decision tree
• Reorder decision tree based on method distributions
• Fold slot access into dispatch
Open Problems
• Feed static information into dynamic dispatch
• Smaller
• Faster
• More adaptive
Readings
• Deutsch and Schiffman 1984• Kiczales and Rodriguez 1989• Dussud 1989 • Moon and Cypher 19??• Amiel et al. 1994• Pang et al. 1999• Holzle and Ungar 1994• Chen and Turau 1994
• Peter Lee Advanced Language Implementation 1991
Acknowledgements
• This lecture includes some material from Craig Chambers’ OOPSLA course on OO language implementation.
Assignment 3 Hint
Create methods with the following construction:
(dm make-method ((n <int>) (types <lst>) (body <lst>) => <met>) (select n ((0) (fun () ...)) ((1) (fun ((a0 (elt types 0))) ...)) ((2) (fun ((a0 (elt types 0)) (a1 (elt types 1))) ...)) ...)
Assignment 4
• Write an associative dispatch cache
• Use linear lookup
• Include profile-guided reordering
• Don’t need to handle singleton dispatch