![Page 1: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/1.jpg)
Data Representation SynthesisPLDI’2011, ESOP’12, PLDI’12
Peter Hawkins, Stanford UniversityAlex Aiken, Stanford University
Kathleen Fisher, TuftsMartin Rinard, MITMooly Sagiv, TAU
http://theory.stanford.edu/~hawkinsp/
![Page 2: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/2.jpg)
Research Interests
• Verifying properties of low level data structure manipulations
• Synthesizing low level data structure manipulations– Composing ADT operations
• Concurrent programming via smart libraries
![Page 3: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/3.jpg)
Motivation
• Sophisticated data structures exist• Increase abstraction level• Simplifies programming– Concurrency
• Integrated into modern PL• But hard to use– Composing several operations
![Page 4: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/4.jpg)
next next next next next
table[0]
[1][2][3][4][5]
null
index= 2 index = 0 index = 3 index = 1 index= 52 0 3 1 1
thttpd: Web ServerConn
Map
Representation Invariants:1. n: Map. v: Z. table[v] =n n->index=v
2. n: Map. n->rc =|{n' : Conn . n’->file_data = n}|
![Page 5: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/5.jpg)
static void add_map(Map *m){ int i = hash(m); … table[i] = m ; … m->index= i ; … m->rc++;
}
thttptd:mmc.c
Representation Invariants:1. n: Map. v:Z. table[v] =n index[n]=v
2. n:Map. rc[n] =|{n' : Conn . file_data[n’] = n}|
broken
restored
![Page 6: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/6.jpg)
attr = new HashMap();…
Attribute removeAttribute(String name){ Attribute val = null; synchronized(attr) { found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } } return val;}
TOMCAT Motivating ExampleTOMCAT 5.*
Invariant: removeAttribute(name) returns the removed value or null if the name does not exist
![Page 7: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/7.jpg)
attr = new ConcurrentHashMap();…
Attribute removeAttribute(String name){ Attribute val = null; /* synchronized(attr) { */ found = attr.containsKey(name) ; if (found) { val = attr.get(name); attr.remove(name); } /* } */ return val;}
TOMCAT Motivating ExampleTOMCAT 6.*
attr.put(“A”, o);
attr.remove(“A”);
o
Invariant: removeAttribute(name) returns the removed value or null if the name does not exist
![Page 8: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/8.jpg)
Multiple ADTs
Invariant: Every element that added to eden is either in eden or in longterm
public void put(K k, V v) { if (this.eden.size() >= size) { this.longterm.putAll(this.eden); this.eden.clear(); } this.eden.put(k, v);}
![Page 9: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/9.jpg)
filesystem=1s_list
s_files
filesystem=2s_list
s_files
filesystems
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_list
file_in_use
file_unused
Filesystem Example
![Page 10: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/10.jpg)
Filesystem Example
Access Patterns
• Find all mounted filesystems
• Find cached files on each filesystem
• Iterate over all used or unused cached files in Least-Recently-Used order
filesystem=1s_list
s_files
filesystem=2s_list
s_files
filesystems
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_list
file_in_use
file_unused
![Page 11: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/11.jpg)
11
Desired Properties of Data Representations
• Correctness
• Efficiency for access patterns
• Simple to reason about
• Easy to evolve
![Page 12: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/12.jpg)
Specifying Combination of Shared Data Structures
• Separation Logic• First order logic+ reachability• Monadic 2nd order logic on trees• Graph grammars• …
![Page 13: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/13.jpg)
fs file inuse
1 14 F
2 7 T
2 5 F
1 6 T
1 2 F
filesystems filesystem=1s_list
s_files
filesystem=2s_list
s_files
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_list
file_in_use
file_unused
{fs, file} {inuse}
fs file inuse
1 17 F
1 34 T
1 42 F
2 5 T
2 5 F
Filesystem Example
![Page 14: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/14.jpg)
Filesystem ExampleContainers: View 1
fs:1 fs:2
file:14file:6
file:2 file:7 file:5
filesystem=1s_list
s_files
filesystem=2s_list
s_files
filesystems
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_list
file_in_use
file_unused
![Page 15: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/15.jpg)
Filesystem ExampleContainers: View 2
filesystem=1s_list
s_files
filesystem=2s_list
s_files
filesystems
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_list
file_in_use
file_unused
fs: 2, file:7fs: 1,
file:14
fs: 2, file:5
inuse:T inuse:F
fs: 1, file:2
![Page 16: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/16.jpg)
Filesystem ExampleContainers: Both Views
fs:1 fs:2
file:14file:6
file:2 file:7 file:5
filesystem=1s_list
s_files
filesystem=2s_list
s_files
filesystems
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_list
file_in_use
file_unused
fs: 2, file:7fs: 1,
file14
fs: 2, file:5
inuse:T inuse:F
fs file inuse
{fs, file} { inuse}
![Page 17: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/17.jpg)
Decomposing Relations
• High-level shape descriptor• A rooted directed acyclic graph• The root represents the whole
relation• The sub-graphs rooted at each
node represent sub-relations• Edges are labeled with columns• Each edge maps a relation into a
sub-relation• Different types of edges for
different primitive data-structures
• Shared nodes correspond to shared sub-relations
fs file inuse
{fs, file} { inuse} inuse
fs, file
fs inuse
file
list
list
array
list
![Page 18: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/18.jpg)
Filesystem Examplefs file inuse1 14 F2 7 T2 5 F1 6 T1 2 F
inuse
fs, file
fs inuse
filefile inuse
14 F
6 T
2 F
file inuse
7 T
5 F
inuse
F
inuse
T
inuse
F
inuse
Tinuse
F
fs:1 fs:2
file:14 file:6 file:2 file:7 file:5
![Page 19: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/19.jpg)
Memory Decomposition(Left)
inuse
fs, file
fs inuse
file
inuse:F inuse:T inuse:F Inuse:T Inuse:F
fs:1 fs:2
file:14 file:6 file:2 file:7 file:5
![Page 20: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/20.jpg)
Filesystem Examplefs file inuse1 14 F2 7 T2 5 F1 6 T1 2 F
fs file2 71 6
fs file1 142 51 2
fs:2file:7
inuse
T
inuse
Tinuse
F
inuse
F
inuse
F
fs:1file:6
fs:1file:14
fs:2file:5
fs:1file:2
inuse:T inuse:F
inuse
fs, file
fs inuse
file
![Page 21: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/21.jpg)
Memory Decomposition(Right)
fs:2file:7
fs:1file:6
fs:1file:14
fs:2file:5
fs:1file:2
inuse:T inuse:F
inuse:F inuse:T inuse:F Inuse:T Inuse:F
inuse
fs, file
fs inuse
file
![Page 22: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/22.jpg)
Decomposition Instance
fs:2file:7
fs:1file:6
fs:1file:14
fs:2file:5
fs:1file:2
inuse:T inuse:F
inuse:F inuse:T inuse:F Inuse:T Inuse:F
fs file inuse
1 14 F
2 7 T
2 5 F
1 6 T
1 2 F
fs:1
file:14 file:6
file:2
file:7file:5
fs:2
fs file inuse
{fs, file} { inuse} inuse
fs, file
fs inuse
file
![Page 23: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/23.jpg)
Shape Descriptor
fs:2file:7
fs:1file:6
fs:1file:14
fs:2file:5
fs:1file:2
inuse:T inuse:F
inuse:F inuse:T inuse:F Inuse:T Inuse:F
fs file inuse
1 14 F
2 7 T
2 5 F
1 6 T
1 2 F
fs:1
file:14 file:6
file:2
file:7file:5
fs:2
fs file inuse
{fs, file} { inuse} inuse
fs, file
fs inuse
file
![Page 24: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/24.jpg)
Memory State
fs:2file:7
fs:1file:6
fs:1file:14
fs:2file:5
fs:1file:2
inuse:T inuse:F
inuse:F inuse:T inuse:F Inuse:T Inuse:F
fs file inuse
1 14 F
2 7 T
2 5 F
1 6 T
1 2 F
fs:1
file:14 file:6
file:2
file:7file:5
fs file inuse
{fs, file} { inuse} inuse
fs, file
fs inuse
filefs:2
list
list
array
list
s_list
f_fs_list f_fs_list f_fs_list
f_list f_list
f_list
![Page 25: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/25.jpg)
Memory State
fs file inuse1 14 F2 7 T2 5 F1 6 T1 2 F
fs file inuse
{fs, file} { inuse} inuse
fs, file
fs inuse
file
list
list
array
list
filesystems
filesystem=1s_list
s_files
filesystem=2s_list
s_files
file=14f_list
f_fs_list
file=6f_list
f_fs_list
file=2f_list
f_fs_list
file=7f_list
f_fs_list
file=5f_list
f_fs_listfile_in_use
file_unused
![Page 26: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/26.jpg)
Adequacy
A decomposition is adequate if it can represent every possible relation matching a relational specification
Adequacy
enforces sufficient conditions for adequacy
Not every decomposition is a good representation of a relation
![Page 27: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/27.jpg)
Adequacy of Decompositions
• All columns are represented• Nodes are consistent with functional
dependencies– Columns bound to paths leading to a common
node must functionally determine each other
![Page 28: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/28.jpg)
Respect Functional Dependencies
file
inuse
{file} {inuse}
![Page 29: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/29.jpg)
Respect Functional Dependencies
file,fs
inuse
{file, fs} {inuse}
![Page 30: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/30.jpg)
Adequacy and Sharing
fs, file
fs inuse
file
inuseColumns bound on a path to an object x must functionallydetermine columns bound on any other path to x
{fs, file}{inuse, fs, file}
![Page 31: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/31.jpg)
Adequacy and Sharing
fs
fs inuse
file
inuse
Columns bound on a path to an object x must functionallydetermine columns bound on any other path to x
{fs, file} {inuse, fs}
![Page 32: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/32.jpg)
Decompositions and Aliasing• A decomposition is an
upper bound on the set of possible aliases
• Example: there are exactly two paths to any instance of node w
![Page 33: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/33.jpg)
The RelC Compiler[PLDI’11]
RelationalSpecification
Programmer
Decomposition
Data structure designer
RelC C++
![Page 34: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/34.jpg)
The RelC Compilerfs fileinuse{fs, file} {inuse}
foreach <fs, file, inuse> filesystems s.t. fs= 5 do …
RelC C++
inuse
fs, file
fs
list inuse
filelist
array
list
Relational Specification
Graph decomposition
![Page 35: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/35.jpg)
Query Plans
foreach <fs, file, inuse> filesystems if inuse=T do …
fs, file
fs
list inuse
file
inuse
list
array
list
scan
scanCost proportional to the number of files
![Page 36: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/36.jpg)
Query Plans
foreach <fs, file, inuse> filesystems if inuse=T do …
fs, file
fs
list inuse
file
inuse
list
array
list
access
scan
Cost proportional to the number of files in use
![Page 37: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/37.jpg)
Removal and graph cutsremove <fs:1>
fs
list
file
inuse
list
array
list
fs file inuse
1 14 F
2 7 T
2 5 F
1 6 T
1 2 F
filesystems fs:1s_list
s_files
fs:2s_list
s_files
file:14f_list
f_fs_list
file:6f_list
f_fs_list
f_listf_fs_list
file:7f_list
f_fs_list
file:5f_list
f_fs_list
inuse:T
inuse:F
file:2
![Page 38: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/38.jpg)
Removal and graph cutsremove <fs:1>
fs file inuse
1 14 F
2 7 T
2 5 F
1 6 T
1 2 F
filesystems fs:2s_list
s_files
file:7f_list
f_fs_list
file:5f_list
f_fs_list
inuse:T
inuse:F
fs
list
file
inuse
list
array
list
![Page 39: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/39.jpg)
Abstraction Theorem
• If the programmer obeys the relational specification and the decomposition is adequate and if the individual containers are correct
• Then the generated low-level code maintains the relational abstraction
relation relationremove <fs:1>
memorystate
memorystate
low level code remove <fs:1>
![Page 40: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/40.jpg)
40
Filesystem ExampleConcurrent
^
•Lock-based concurrent code is difficult to write and difficult to reason about•Hard to choose the
correct granularity of locking•Hard to use
concurrent containers correctly
![Page 41: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/41.jpg)
41
Granularity of Locking
Fine-grained locking
Coarse-grained locking
More complex
Easier to reason aboutEasier to change
More scalable
Lower overheadLess complex
![Page 42: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/42.jpg)
42
Concurrent Containers• Concurrent containers are hard to write• Excellent concurrent containers exist (e.g.,
Java’s ConcurrentHashMap and ConcurrentSkipListMap)
• Composing concurrent containers is tricky– It is not usually safe to update containers in paralel– Lock association is tricky
• In practice programmers often* use concurrent containers incorrectly
* 44% of the time [Shacham et al., 2011]
![Page 43: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/43.jpg)
43
Concurrent Data Representation Synthesis[PLDI’12]
RelScala
Compiler
Concurrent Data Structures,Atomic Transactions
Relational Specification
Concurrent Decomposition
![Page 44: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/44.jpg)
44
Concurrent Decompositions
• Concurrent decompositions describe how to represent a relation using concurrent containers and locks
• Concurrent decompositions specify: – the choice of containers– the number and placement of
locks– which locks protect which data= ConcurrentHashMap
= TreeMap
![Page 45: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/45.jpg)
45
Concurrent Interface
Interface consisting of atomic operations:
![Page 46: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/46.jpg)
46
Goal: Serializability
Thread 1
Thread 2
Time
Every schedule of operations is equivalent to some serial schedule
![Page 47: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/47.jpg)
47
Goal: Serializability
Time
Thread 1
Thread 2
Every schedule of operations is equivalent to some serial schedule
![Page 48: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/48.jpg)
48
Two-Phase Locking
Two phase locking protocol:
• Well-locked: To perform a read or write, a thread must hold the corresponding lock
• Two-phase: All lock acquisitions must precede all lock releases
Attach a lock to each piece of data
Theorem [Eswaran et al., 1976]: Well-locked, two-phase transactions are serializable
![Page 49: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/49.jpg)
49
Two Phase Locking
Attach a lock to every edge
Problem 2: Too many locks
Decomposition Decomposition Instance
We’re done!
Problem 1: Can’t attach locks to container entries
![Page 50: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/50.jpg)
Lock Placements
1. Attach locks to nodes
Decomposition Decomposition Instance
![Page 51: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/51.jpg)
51
Coarse-Grained LockingDecomposition Decomposition Instance
![Page 52: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/52.jpg)
52
Finer-Grained LockingDecomposition Decomposition Instance
![Page 53: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/53.jpg)
53
Lock StripingDecomposition Decomposition Instance
![Page 54: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/54.jpg)
54
Lock Placements: Domination
Decomposition Decomposition Instance
Locks must dominate the edges they protect
![Page 55: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/55.jpg)
55
Lock Placements: Path-ClosureAll edges on a path between an edge and its lock must share the same lock
![Page 56: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/56.jpg)
56
Decompositions and Aliasing• A decomposition is an upper
bound on the set of possible aliases
• Example: there are exactly two paths to any instance of node w
• We can reason about the relationship between locks and heap cells soundly
![Page 57: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/57.jpg)
57
Lock Ordering
Prevent deadlock via a topological order on locks
![Page 58: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/58.jpg)
58
Queries and Deadlock
2. lookup(tv)
1. acquire(t)
3. acquire(v)
4. scan(vw)
Query plans must acquire the correct locks in the correct order
Example: find files on a particular filesystem
![Page 59: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/59.jpg)
Autotuner
• Given a fixed set of primitive types– list, circular list, doubly-linked list, array, map, …
• A workload• Exhaustively enumerate all the adequate
decompositions up to certain size• The compiler can automatically pick the best
representation for the workload
![Page 60: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/60.jpg)
Directed Graph Example (DFS)• Columns
src dst weight• Functional Dependencies
– {src, dst} {weight}
• Primitive data types– map, list
…
src
dst
weight
map
list
src
dst
dst
src
weight
map
listlist
mapdst
src
weightm
aplist
dst
weight
map srcmap
weight
dst
listsrclist
![Page 61: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/61.jpg)
61
Concurrent SynthesisFind optimal combination of
Decomposition Shape ContainerData Structures
ConcurrentHashMap
ConcurrentSkipListMap
CopyOnWriteArrayList
Array
HashMap
TreeMap
LinkedList
Lock Implementations
ReentrantReadWriteLock
ReentrantLock
Lock Striping Factors
Lock Placement
![Page 62: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/62.jpg)
62
Concurrent Graph Benchmark
• Start with an empty graph• Each thread performs 5 x 105 random
operations• Distribution of operations a-b-c-d (a% find
successors, b% find predecessors, c% insert edge, d% remove edge)
• Plot throughput with varying number of threads Based on Herlihy’s benchmark of concurrent maps
![Page 63: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/63.jpg)
Black = handwritten,isomorphic to blue
Key
= ConcurrentHashMap= HashMap
...
63
Results: 35-35-20-1035% find successor, 35% find predecessor, 20% insert edge, 10% remove edge
![Page 64: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/64.jpg)
(Some) Related Projects
• Relational synthesis: [Cohen & Campbell 1993], [Batory & Thomas 1996], [Smaragdakis & Batory 1997], [Batory et al. 2000] [Manevich, 2012] …
• Two-phase locking and Predicate Locking [Eswaran et al., 1976], Tree and DAG locking protocols [Attiya et al., 2010], Domination Locking [Golan-Gueta et al., 2011]
• Lock Inference for Atomic Sections: [McCloskey et al.,2006], [Hicks, 2006], [Emmi, 2007]
![Page 65: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/65.jpg)
Summary
• Decompositions describe how to represent a relation using concurrent containers
• Lock placements capture different locking strategies
• Synthesis explores the combined space of decompositions and lock placements to find the best possible concurrent data structure implementations
![Page 66: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/66.jpg)
The Big Idea
• Simpler code• Correctness by construction• Find better combinations of data structures
and locking
Benefits
Relational Specification
Low Level Concurrent Code
![Page 67: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/67.jpg)
Transactional Objects with Foresight
Guy GuetaG. Ramalingam
Mooly SagivEran Yahav
![Page 68: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/68.jpg)
Motivation
• Modern libraries provide atomic ADTs• Hidden synchronization complexity• Not composable– Sequences of operations are not necessarily
atomic
![Page 69: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/69.jpg)
Transactional Objects with Foresight
• The client declares the intended use of an API via temporal specifications (foresight)
• The library utilizes the specification– Synchronize between operations which do not
serialize with foresight• Foresight can be automatically inferred with
sequential static analysis
![Page 70: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/70.jpg)
Example: Positive CounterOperation A@mayUse(Inc);Inc()Inc()@mayUse()
Operation B@mayUse(Dec);Dec()Dec()@mayUse()
• Composite operations are not atomic• even when Inc/Dec are atomic
• Enforce atomicity using future information • Future information can be utilized for effective
synchronization
![Page 71: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/71.jpg)
Possible Executions (initial value=0)
![Page 72: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/72.jpg)
Main Results
• Define the notion of ADTs that utilize foresight information– Show that they are strict generalization of existing locking
mechanisms• Create a Java Map ADT which utilizes foresight • Static analysis for inferring conservative foresight
information• Performance is comparable to manual
synchronization• Can be used for real composite operations
![Page 73: Data Representation Synthesis PLDI’2011, ESOP’12, PLDI’12 Peter Hawkins, Stanford University Alex Aiken, Stanford University Kathleen Fisher, Tufts Martin](https://reader035.vdocuments.site/reader035/viewer/2022062805/5697c01c1a28abf838ccfc99/html5/thumbnails/73.jpg)
The RelC Compilerfs fileinuse{fs, file} {inuse}
foreach <fs, file, inuse> filesystems s.t. fs= 5 do …
RelC C++
inuse
fs, file
fs
list inuse
filelist
array
list
Relational Specification
Graph decomposition