Immutable Collections

Data Structure Encoding VM Support

Language and API DesignMichael J. Steindorfer

mail: michael@cwi.nltwitter: @loopingoptimism

“A collection represents a group of objects […]”

–java.util.Collection API

3

Data Type Semanticslists, sets, maps, multi-maps, bags, …

Orderingunordered (hashed), ordered

Update Semanticsmutable, transient, immutable

Processing Semanticssequential, concurrent, parallel

Encodingarray/hashtable, linked list, tree, trie, …

Contenthomogeneous, heterogeneous

…4

Data Type Semanticslists, sets, maps, multi-maps, bags, …

Orderingunordered (hashed), ordered

Update Semanticsmutable, transient, immutable

Processing Semanticssequential, concurrent, parallel

Encodingarray/hashtable, linked list, tree, trie, …

Contenthomogeneous, heterogeneous

…5

Why Immutability?

• More Optimizations (Constant)

• Simple (Programming Model)

• Robust (Concurrency)

6

“Classes should be immutable unless there’s a very good reason to make them mutable.”

–Effective Java, 2nd Edition (Joshua Bloch, 2008)Item 15: Minimize mutability

7

“[Collections] should be immutable unless there’s a very good reason to make them mutable.”

–Effective Java, 2nd Edition (Joshua Bloch, 2008)Item 15: Minimize mutability

8

“Immutable [collections] provide many advantages, and their only disadvantage is the potential for

performance problems under certain circumstances.”

–Effective Java, 2nd Edition (Joshua Bloch, 2008)Item 15: Minimize mutability

9

Immutable collections are challenging to optimize, but overall they constitute a more sensible default.

–Personal Opinion

10

Outline

1. Core Principles & Encodings

2. Minimize Memory Footprint

3. Efficiency & Expressivity

4. Performance Challenges

11

Core Principles

12

Set<Integer> updatedSet = hugeSet.newInstanceWith(34);

13

Set<Integer> __tmp = new HashSet<>(hugeSet.size + 1); __tmp.addAll(hugeSet);__tmp.add(34);

Set<Integer> updatedSet = Collections.unmodifiableSet(__tmp);

• • 32

•… • •

2 4098

… …

……

• • 32

• 34

Delta

14

15

Clojurehttp://clojure.org

Scalahttp://scala-lang.org

Rascalhttp://rascal-mpl.org

Hash Array Mapped Trie

16

0001011010

0000001110 01111

11000

… …

• • 32

•… • •

2 4098……

17

Prefix of Hash Code

tree node as array

{2, 32, 34, 898, 2942, 4098, 6014}

18

19

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

320 1 2 3 4 5 6

...... 31

034

1 2...

... 24...

...898

28...

... 31

20

......

40984 5 6 7 8

...... 31

029421

...... 4

60145 6 7

...... 31

Fig. 2: Examples for presentation.

class Node { int bitmap; Object[] content;}

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

Fig. 2: Examples for presentation.

What can you expect?

• Shallow and Wide Search Trees (Depth <= 7)

• Lookup/Insert/Delete in O(log32(n)) ≅ O(1)

• Union/Subset (Structural Operations)

Minimize Memory Footprintby Specializing Class Layouts

22

class Node { int bitmap; Object[] content;}

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

Fig. 2: Examples for presentation.

class ArrayList32 { /* 0 <= content.size <= 32 */ Object[] content;}

24

class ArrayList32 { Object[] content;}

25

class List1 implements ImmutableList { Object slot0;}

class List2 implements ImmutableList { Object slot0; Object slot1;}

class List3 implements ImmutableList { Object slot0; Object slot1; Object slot2;}

...

! Small Footprints ☹Megamorphic Call Sites ☹ Code Bloat

26

abstract class List2 implements ImmutableList { Object slot0; Object slot1; @Override public ImmutableList insertAtIndex( int index, Object item) { switch (index) { case 0: return new List3(item, slot0, slot1); case 1: return new List3(slot0, item, slot1); case 2: return new List3(slot0, slot1, item); default: throw new IllegalArgumentException(); } }}

27

! Small Footprints !Monomorphic Call Sites ! JIT-Friendly

28

Decoupling Data and Operations

29

abstract class List2 implements ImmutableList { Object slot0; Object slot1;}

// viewing consecutive fields as arrays createArrayView(List2.class, "slot0", “slot1”, Object.class);

31

ImmutableList insertAtIndex(int index, Object item) { ArrayView<Object> src = getArrayView(); ImmutableList newList = allocateList(src.length + 1); ArrayView<Object> dst = newList.getArrayView(); // copy 'src' and insert element at position 'index' arrayviewcopy(src, 0, dst, 0, index); dst.set(index, item); arrayviewcopy(src, index, dst, index + 1, src.length - index);

return newNode; }

Efficiency & Expressivity

33

class Node { int bitmap; Object[] content;}

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

Fig. 2: Examples for presentation.

More Performant …

35

Up to 6x faster …(speedup of iteration over Clojure / Scala)

36

Steindorfer, M. J., & Vinju, J. J. (2015). Optimizing Hash-Array Mapped Tries for Fast and Lean Immutable JVM Collections. OOPSLA’15.

… up to 28x faster …(speedup equality checking over Clojure / Scala)

37

Steindorfer, M. J., & Vinju, J. J. (2015). Optimizing Hash-Array Mapped Tries for Fast and Lean Immutable JVM Collections. OOPSLA’15.

… and up to 64% smaller.(memory footprint reduction over Clojure / Scala)

38

Steindorfer, M. J., & Vinju, J. J. (2015). Optimizing Hash-Array Mapped Tries for Fast and Lean Immutable JVM Collections. OOPSLA’15.

39

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

Fig. 2: Examples for presentation.

40

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

Fig. 2: Examples for presentation.

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

320 1 2 3 4 5 6

...... 31

034

1 2...

... 24...

...898

28...

... 31

20

......

40984 5 6 7 8

...... 31

029421

...... 4

60145 6 7

...... 31

Fig. 2: Examples for presentation.

class Node { int datamap; int nodemap; Object[] content;}

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

320 1 2 3 4 5 6

...... 31

034

1 2...

... 24...

...898

28...

... 31

20

......

40984 5 6 7 8

...... 31

029421

...... 4

60145 6 7

...... 31

Fig. 2: Examples for presentation.

Heterogeneous Data Structures for the Masses 5

320-Key

B0-Val 2-Key 2-Val

0-Key 0-Val

341-Key

C1-Val 24-Key 24-Val

89828-Key

D28-Val

20-Key

A0-Val

40984-Key

G4-Val

29421-Key

F1-Val

60145-Key

H5-Val

320-Key

B0-Val 2-Ref

341-Key

C1-Val

89828-Key

D28-Val 24-Ref 0-Ref

20-Key

A0-Val

40984-Key

G4-Val

29421-Key

F1-Val

60145-Key

H5-Val

B Examples for Truffle’15 Talk

C Examples for OOPSLA’15 Talk

42

Heterogeneous Data Structures for the Masses 5

320-Key

B0-Val 2-Key 2-Val

0-Key 0-Val

341-Key

C1-Val 24-Key 24-Val

89828-Key

D28-Val

20-Key

A0-Val

40984-Key

G4-Val

29421-Key

F1-Val

60145-Key

H5-Val

320-Key

B0-Val 2-Ref

341-Key

C1-Val

89828-Key

D28-Val 24-Ref 0-Ref

20-Key

A0-Val

40984-Key

G4-Val

29421-Key

F1-Val

60145-Key

H5-Val

B Examples for Truffle’15 Talk

C Examples for OOPSLA’15 Talk

43

More Expressive …

44

Numeric Data SetsStoring and Processing with Collections

data set ofint (32-bit)

SpecializedPrimitive

Collection

data set ofBigInteger (Object)

Generic Collection

Primitive Collection ???

data set ofint (32-bit) ✘

Generic Collection ???

data set ofBigInteger (Object)

data set ofInteger (Boxed)

data set ofint (32-bit)

data set ofBigInteger (Object)

Heterogeneous Collections

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

BIG

320 2

341 24

89828 0

BIG 20

40984

320 2

341

89828 24 0

BIG 20

40984

Fig. 2: HAMT-based sets with values in internal nodes versus values at the leavesonly.

51

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

BIG

320 2

341 24

89828 0

BIG 20

40984

320 2

341

89828 24 0

BIG 20

40984

Fig. 2: HAMT-based sets with values in internal nodes versus values at the leavesonly.

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

BIG

320 2

341 24

89828 0

BIG 20

40984

320 2

341

89828 24 0

BIG 20

40984

Fig. 2: HAMT-based sets with values in internal nodes versus values at the leavesonly.

52

Steindorfer, M. J., & Vinju, J. J. (2016). Fast and Lean Immutable Multi-Maps on the JVM based on Heterogeneous Hash-Array Mapped Tries. To Appear.

00 1 00 1 … 0 0 00

01 0 00 0 … 1 1 10

class Node {

Object[] content;}

int datamap; int nodemap; 2x 1-bit

00 1 … 001 0 … 1

BitVector bitmap;class Node {

Object[] content;}

1x n-bit

00 1 … 001 0 … 1

BitVector bitmap;class Node {

Object[] content;}

1x n-bitOrthogonal To:• Scala’s Union Types • Valhalla’s Primitive Generics

Performance Challenges

55

Trees (as fast) as Arrays?Improving Locality between Nodes.

56

• • 32

•… • •

2 4098

… …

……

57

• • 32 •… • • 2 4098……

58

• • 32

•… • •

2 4098

… …

……

• • 32

• 34

59

• • 32 •… • • 2 4098……

60

• • 32 • 34

Trees (as fast) as Arrays!Memory Management & Garbage Collection

61

Summary

15

Clojurehttp://clojure.org

Scalahttp://scala-lang.org

Rascalhttp://rascal-mpl.org

data set ofint (32-bit)

data set ofBigInteger (Object)

Heterogeneous Collections

class ArrayList32 { Object[] content;}

41

class List1 implements ImmutableList { Object slot0;}

class List2 implements ImmutableList { Object slot0; Object slot1;}

class List3 implements ImmutableList { Object slot0; Object slot1; Object slot2;}

...

40

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

Fig. 2: Examples for presentation.

6 Michael J. Steindorfer

320 2

034

1 24898

28

20

40984

29421

60145

320 2

341

89828 0 24

20

40984

29421

60145

320 2

341

89828 24 0

20

40984

29421

60145

320 1 2 3 4 5 6

...... 31

034

1 2...

... 24...

...898

28...

... 31

20

......

40984 5 6 7 8

...... 31

029421

...... 4

60145 6 7

...... 31

Fig. 2: Examples for presentation.

• • 32 •… • • 2 4098……

54

• • 32 • 34

Map<String, Long> wordCount = phrases.stream()

.flatMap (toWordStream)

.filter (word -> word.length() > 0)

.map (word -> new Tuple<>(word, 1L))

.collect (groupingBy(Tuple::getKey, counting()));

Map<String, Long> wordCount = phrases.stream()

.flatMap (toWordStream)

.filter (word -> word.length() > 0)

.map (word -> new Tuple<>(word, 1L))

.collect (groupingBy(Tuple::getKey, counting()));

Missing: Immutable Collections

usethesource/capsuleThe Capsule Hash Trie Collections Library

Immutable Collections

Data Structure Encoding VM Support

Language and API DesignMichael J. Steindorfer

mail: michael@cwi.nltwitter: @loopingoptimism