1

DiSTiL : A Transformation DiSTiL : A Transformation Library for Data StructuresLibrary for Data Structures

Yannis Smaragdakis

2

OverviewOverview

DiSTiL is a project to add GenVoca Components to Microsoft’s IP

Domain: container data structures

3

Overview (Continued)Overview (Continued)

Goal: increase productivity by programming in high-level data structure abstractions

cursor container

4

Overview (Continued)Overview (Continued)

Create complex container data structures by composing DiSTiL components

Design rule checking to ensure validity of component compositions

Automatic selection of data structures according to retrieval predicates

5

Implementation Implementation PlatformPlatform

IP Transformation System (Microsoft Research)

Handling of code with tree primitives

User can define new language primitives (intentions) and write code to transform them at compilation (reduction) time

6

DiSTiL Library (Realms)DiSTiL Library (Realms)

Data Structures (arrays, linked lists, red-black trees, hash-tables)

Storage (Persistent and Transient storage) Architectural (create code in functions) Special purpose layers (LRUTree) Various data-structure related layers

(garbage collection, bound checks) Hidden layers (out-of-bounds, predicate,

order)

7

DiSTiL Type ExpressionsDiSTiL Type Expressions

Typex construct: the composition of components (layers) that defines the target container data structure

Huge number of distinct container data structures can be generated from compositions of DiSTiL components

8

CursorsCursors

Cursors define retrieval predicates over containers

cursor(emp_cont, name < “An” && name > “Akers”, no_order)

Predicates used to generate efficient code for retrievals

The “smart” part of the library

9

A Composite Data A Composite Data StructureStructure

Jones4719711

Keen4789711

Lam4709711

Mann4799711

Guy4769711

Koch4749711

Land4739711

Smith4729711

Ajit4759711

typex Example = malloc[transient]

10

A Composite Data A Composite Data StructureStructure

Jones4719711

Keen4789711

Lam4709711

Mann4799711

Guy4769711

Koch4749711

Land4739711

Smith4729711

Ajit4759711

typex Example = tree[malloc[transient]]

key=name

11

typex Example = hash [ tree [ malloc [ transient]]]

A Composite Data A Composite Data StructureStructure

Jones4719711

Keen4789711

Lam4709711

Mann4799711

Guy4769711

Koch4749711

Land4739711

Smith4729711

Ajit4759711

key=name

key=phone

12

How to UseHow to Use

13

What to NoticeWhat to Notice

Programming in domain-specific abstractions makes code easier to understand makes code more reliable (simpler and

checked at a higher level) makes code easier to evolve (can

modify typex without modifying program)

should result in very good code

14

What to NoticeWhat to Notice

DiSTiL actually can increase programmer

productivity (simpler/shorter programs)

removes burden of coding & debugging data structures

can accommodate architectural styles

15

Example ApplicationExample Application

LRU memory policy simulation

0

1

2

0

1

1

0 0

3

16

What is different about What is different about DiSTil?DiSTil?

Different from static libraries (e.g. STL): meta-level optimizations

(retrieval structure selection, updates) more advanced static checking

(design-rule checking) level of programming significantly raised

(“what” vs “how”) more general, powerful approach larger selection of structures/mechanisms

17

What is different about What is different about DiSTil?DiSTil?

Even at the usual level of abstraction static libraries suffer from performance

loss not easy to integrate such a variety of

components example: templates and memory

allocation would have the added cost of a virtual

dispatch static consistency checking is a domain-

specific mechanism (type checking)

18

Internal organizationInternal organizationQuote Package Static Library

Components

Exported Symbols

Predicate engine

Transformationengine

User Application

GeneratedApplication

19

Next StepsNext Steps

Improve design-rule checking Clarify design, possibly develop

tools for writing generators Explore ideas (explicit, generation-

time scoping) examined in the Quote package

Broaden applications

20

Contributions of DiSTiLContributions of DiSTiL

Example of factored libraries (Biggerstaff 1994)

Primitive data structures factored in new ways (out-of-bounds checking, predicates on retrievals, inlined/functional implementation etc.)

Extensive background for software generator development in non-scoped environments (Quote package)

21

Contributions (cont’d)Contributions (cont’d)

Layered generator: analysis of layer interaction possible mechanisms to facilitate

construction determining pattern of information

flow among layers