history independent data-structures. what is history independent data-structure ? sometimes data...

History Independent Data-Structures

What is History Independent

Data-Structure ?• Sometimes data structures

keep unnecessary information.– not accessible via the legitimate

interface of the data structure.– can be restored from the data-structure

layout.

The core problem: history of operations applied on the data-structure may be revealed.

History Independence - Motivation

• A privacy issue if an adversary gains control over the data-structure layout - Laptop was stolen.

• Sometimes you just send the data-structure over the web …

• Word documents• Search indexes inside a data-structure• List of Students/ grades etc.

ExampleData structure with three operations:• Insert(D, x)• Remove(D, x)• Print(D)

• Used for a wedding invitee list.

Naive Implementation – an array.• Insert – adds last entry.• Remove entry i – move entries i+1 to n

backwards• (wiser implementation - linked list on an array)

Layout implies the order.

For example, who was invited last !

Weak History Independence

[Naor, Teague]: A Data structure implementation is (weakly) History Independent if:

Any two sequences of operations S1 and S2 that yield the same content induce the same distribution on memory layout.

Security: Nothing gained from layout beyond the content.

Example – cont.

Making the previous data structure weakly history independent:

• Insert(x): (say, n elements in data-structure)– Choose uniformly at random r {1,2,

…,n+1} – Set A[n+1] A[r]; A[r] x

• Remove entry i: A[i] A[n]

The array is a uniformly chosen permutation on the elements

Weak History Independence Problems

No Information leaks if adversary gets layout once (e.g., the laptop was stolen).

But what if adversary may get layout several times ?

• Information on content modifications leaks.

• We want: no more information leakage.

Strong History Independence

Pair of sequences S1, S2

two lists of stop points in S1, S2

If content is the same in each pair of corresponding stop points Then: Joint distribution of memory layouts at stop points is identical in the two sequences.

[Naor-Teague]: A Data structure implementation is (Strongly) History Independent if:

Security: We cannot distinguish betweenany such two sequences.


S1 = ins(1), ins(2), ins(3), ins(4)

S2 = ins(2), ins(1), ins(5), ins(4), ins(3), del(5)

First stop Second stop

First stop Second stop

We should not be able to tell from the layouts which of the two sequences happened

Example – cont.

Recall example:• Insert(x) : (say, n elements in database)

– Choose uniformly at random r {1,2,…,n+1} – Set A[n+1] A[r]; A[r] x

• Remove entry i: A[i] A[n]

Is this implementation strongly history independent ?

No !

Example – cont.

1 2 3 4 5

Assume you get the layout of the array twice:

First time you see:

Second time you see: 5 2 3 4 1

What could not happen:

The empty sequence Remove(4), Insert(4)

Lots of other constraints…

Example – last

Making the data structure strongly history independent

We can keep the array aligned left and sorted.

Each content has only one possible layout.

Problem: The time complexity of Insert and Remove is Ω(n),

(“Usually” shift Ω(n) elements during insert or delete)

History of History Independence

[Micciancio97] • Weak history independent 2-3 trees

(motivated by the problem of private incremental cryptography [BGG95]).

[Naor-Teague01] • History-independent hash-table, union-find. • Weak history-independent memory allocation. • History independent Dynamic Perfect Hashing

History of History Independence

[Hartline et al. 02] • Strong history independence means

canonical layout. • Relaxation of strong history independence.• History independent memory resize.

[Buchbinder, Petrank 03] • Lower bounds on Strong History

independent data-structures. • History independent heaps.

What’s Next1. 2-3 Trees2. HashTables 3. Strong History Independence means Canonical

Representation.4. Lower Bounds on strong history independence.5. Lower Bounds on relaxed strong history

independence.6. Obtaining a weak history independent heap.

2-3 Trees and Cryptography

• You sign a word document. • Now you make a small change – do you need to resign the whole document?

• Maybe you want to deliver the document to a few people with only small changes.

• Do you need to compute several signatures?

You don’t want to work hard!!

2-3 Trees and Cryptography

Solution:• Partition the document into small blocks.• Sign each block.• Construct a 2-3 tree on the blocks. • Each internal node is a signature of its children

along with the size of its sub-tree.When delete/add/edit a block only O(log n) small

signatures are needed.

Problem: The structure of the 2-3 tree reveals some edit information!

2-3 Trees

Why does the standard implementation of 2-3 Trees not History Independent?

1 1 2 3 4

Insert: 1,2,3,4,5

1 2 1 2 3

1 2 3 4 5

2-3 Trees

1 2 3 4 5

S1 = Insert: 1,2,3,4,5

1 2 3 4 5

Insert 1

We may distinguish between the two sequences

2 3 4 5

Remove 1

S2 = Insert: 1,2,3,4,5, Remove 1, Insert 1

2-3 Trees – Solution

• The number of children of each internal node is 2 or 3 with equal probability.

(except the last one)

• CreateTree – O(n)• Find – O(log n)• Insert/Remove – O(log n) expected

time.

2-3 Trees – CreateTree

1 2 3 4 51 2 3 4 5

1 2 3 4 5

Prob = 1/2 Prob = 1/4

Prob = 1/4

3 nodes in level 2 Process should be

continued

2-3 Trees – CreateTree cont.

1 2 3 4 5

Prob = 1/8

1 2 3 4 5

1 2 3 4 5

Prob = 1/8

Prob = 1/4

2-3 Trees – Solution

• We want: Insert/Remove generate the same distribution as CreateTree.

History independent

Idea:• When inserting/removing a leaf:

– The previous leaves/nodes are Ok.– Fix the next leaves by new coin tosses.

2-3 Trees – inserting a new node

2 3 4 5 1 2 3 4 5

Prob = 1/2

2 3 4 5 1 2 3 4 5

Prob = 1/2

Continue on …

2-3 Trees – Insert/Remove

Complexity proof Ideas:• In each two successive iterations we

synchronize with previous grouping with constant probability.

• The number of nodes “touched” in each level is O(1).

• The total number of nodes “touched” in all levels is O(log n).

Hash-Tables

Standard implementation (open address):

• Choose hash functions: h1, h2, h3 …

Insert(x):

• If hi(x) is occupied – try hi+1(x) …

Delete(x): • Mark the cell as deleted

- No actual delete.

Hash-Tables - Problems

• The deleted items still appear!• If h1(x) = h1(y) then we can know whether x

or y where inserted firstthe one that was hashed by h2

Solution:• No deletions.• When hi(x) = hi(y): decide to rehash x or y

by some predetermined order between them.

The hash-table has canonical form Strong History independent

Strong History Independence =

Canonical Representation

Definition [content graph]: The content graph of data-structure:

Vertices: The possible contents.Edges: C1 C2 if operation OP and

parameters σ such that OP(C1, σ)= C2.

Definition [well behaved]: An abstract data-structure is well behaved if its content graph is strongly connected.

Strong History Independence =

Canonical Representation

Lemma: For any strongly history independent implementation of a well behaved data-structure:

layout L, operation Op, Op(L) yields only one possible layout.

Corollary: Any strongly history independent implementation of well-behaved data-structure is canonical.

Canonical Representation: Proof cont.

Corollary: Any strongly history independent implementation of well-behaved data-structure is canonical.

Proof sketch (assuming the lemma): Let S be a sequence of operations yielding

content C. • Each operation in S generates one layout.

By induction S yields one possible layout.• By strong history independence any other

sequence yielding C creates the same layout.

Canonical Representation Proof of Lemma

Lemma: For any strongly history independent implementation of a well behaved data-structure:

layout L, operation Op, Op(L) yields only one possible layout.

• Assuming well-behaved, any operation Op has a sequence OP-1 that “reverses” Op.

• Assuming strong history independence we may set any two sequences with stop points.


Proof sketch: Fix any layout L, fix any operation Op. We need to show that Op(L) yields a single specific layout L’.

Let S be any sequence of operation yielding L with probability > 0.

Consider the following sequences with the following ‘stop’ points:

S1 = S S2 = S ◦ Op ◦ OP-1

12 1 2

The two stop points are the same in S1.

The same layout must also appear in S2.


S1 = S S2 = S ◦ Op ◦ OP-1

12 1 2

• Suppose L appears after S. • L must appear again at the end of S2. Otherwise,

we could distinguish between the two sequences. • For any Li =Op(L), Op-1 must transform Li to L with

probability 1.

L L2

L1

Lk

Op

L

Op-1

Op

Op

Op-1

Op-1

Canonical Representation: Proof Now let’s extend the sequence and modify stop points:

S3 = S ◦ Op

12 1 2

S4 = S ◦ Op ◦ Op-1 ◦ Op

• Suppose some Li=Op(L) appears after S ◦ Op.

Li must appear also at the end of S4.Otherwise, we could distinguish between the two sequences.

• After Op-1 the layout is again L.• The operation of Op depends only on L.• Op cannot “know” which Li to create.

There is only one Li = Op(L)

L L2

L1

Lk

Op

L

Op-1

Op

Op

Op-1

Op-1

Lower Bounds: an example

Lemma: D: Data-structure whose content is the set of

keys stored inside it. I: Implementation of D that is :

comparison-based and canonical. The operation Insert(D, x) requires time Ω(n).

This lemma applies for example to:Heaps, Dictionaries, Search trees.

Why is Comparison Based implementation important?

• It is “natural”: – Standard implementations for most data

structure operations are like that.– Therefore, we should know not to design

this way when seeking strong history independence

• Library functions are easy to use: – Only implement the comparison

operation on data structure elements.

Lower Bounds – cont.

Proof sketch:• comparison-based: keys are treated as

‘black boxes’ according to the comparison order.

The algorithm treats any n keys only according to their total order.

The canonical layout of any n different keys is the same no matter what their real values are.

• d1, d2, … dn - memory addresses of n keys in the layout according to their total order.

• d’1, d’2, … d’n+1 - memory addresses of n+1 keys in the layout according to their total order.

Lower Bounds – cont.Δ: The number of indices for which di d’i

Consider the content C = {k2, k3, … , kn+1} k2< k3< … < kn+1:

Case 1 - Δ > n/2 - consider insert(C, kn+2): • Puts kn+2 in address d’n+1.

• Moves each ki (2 i n+1) from di-1 to d’i-1. The operation moves at least n/2 keys.

Case 2 - Δ n/2 - consider insert(C, k1):• Puts k1 in d’1 • Moves each ki (2 i n+1) from di-1 to d’i.

The operation moves at least n/2 keys.

More Lower Bounds

By similar methods we can show:

• Remove-key requires time Ω(n).• For a Heap:

– Increase-key requires time Ω(n).– Build-Heap Operation requires time Ω(n log

n).

• For a queue: either Insert-first or Remove-Last requires time Ω(n).

Relaxed strong history independence

Strong history independence implies very strong lower bounds.

How can we relax the definition allowing more efficient data structures ?

One possible way [HHMPR02 ]:• Allowing the adversary to distinguish

between the empty sequence and other sequences.

Does this definition implies canonical memory layout ?

Relaxed strong history independence (cont.)

The relaxed definition does not implies canonical memory layout.

Possible implementation of previous data structure:

In each operation - choose a new independent uniformly chosen permutation of the elements.

1. Not canonical …2. Relaxed strong history independent.3. Each operation - O(n)

Relaxed strong history independence

• Is this relaxation enough ? (for efficient implementations)

No

• We may prove almost the same lower bounds using different property of these data structures.

The Binary Heap

Binary heap - a simple implementation of a priority queue.

• The keys are stored in an almost full binary tree.

• Heap property - For each node i: V(parent(i)) V(i)

• Assume that all values in theheap are unique.

10

7 9

36 4 8

1 5 2

The Binary Heap: Heapify

Heapify - used to preserve the heap property. • Input: a root and two proper sub-heaps of

height h-1. • Output: a proper heap of height h.

The node always chooses to sift down to the direction of the larger value.

2

10 9

36 7 8

1 5 4

Heapify Operation

2

10 9

36 7 8

1 5 4

10

7 9

36 4 8

1 5 2

Reversing Heapify

heapify-1: “reversing” heapify:

Heapify-1(H: Heap, i: position)

• Root vi

• All the path from the root to node i are shifted down.

10

7 9

36 4 8

1 5 2

The parameter i is a position in the heap H

Heapify-1 Operation

10

7 9

36 4 8

1 5 2

2

10 9

36 7 8

1 5 4

Heapify(Heapify-1(H, i)) = H

Property: If all the keys in the heap are unique then for any i:

The Binary Heap: Build-heap in O(n)

Building a heap - applying heapify on any sub-tree in the heap in a bottom up manner.

nh

hh

nh

hh

nOh

nOhOn log

11

log

11

)()2

()(2

Time Complexity10

7 9

36 4 8

1 5 2

Reversing Build-heap

Build-Heap-1(H: heap) : Tree• If size(H) = 1 then return (H);• Choose a node i uniformly at random among

the nodes in the heap H;• H Heapify-1(H, i);

• Return TREE(root(H), build-heap-1(HL),

build-heap-1(HR));

For any random choice:

Build-heap(Build-heap-1(H)) = H

Works in a Top-Bottom manner

Uniformly Chosen Heaps

• Build-heap is a Many-To-One procedure.• Build-heap-1 is a One-To-Many procedure

depending on the random choices.

Support(H) : The set of permutations (trees) such that build-heap(T) = H

Facts (without proof):1.For each heap H the size of Support(H) is

the same.2.Build-heap-1 returns one of these heaps

uniformly.

How to Obtain a Weak History Independent

Heap

Main idea: keeping a uniformly random heap at all time.

We want:1.Build-heap: Return one of the possible

heaps uniformly.

2.Other operations: preserve this property.

An Easy Implementation: Build-Heap

Apply random permutation on the input elementsand then use the standard build-heap.Analysis:Each heap has the same size of Support group each heap has the same probability.

More intuition: Applying random permutation on the elements erases all data about the order of the elements. There is no information on the history.

Another Easy Implementation: Increase-key

Standard Increase-key - changes the value

of element and sift it up until it gets to the correct place.

9

7 8

36 10 4

1 5 2

10

9 8

36 7 4

1 5 2

Increase-key – cont.

The standard increase-key is good for us.

1.The increase-key operation is reversible:• decreasing the value of the key back will

return the key to its previous location.2.The number of heaps with n different keys is

the same no matter of the actual values of keys.

The increase-key function is 1-1.

If we had uniformly chosen heap then afterincrease-key it stays uniformly chosen heap.

Not So Easy: Extract-max and Insert

Extract-max(H)• Replace the value at the root with the value of

the last leaf.• Let the value sift down to the right position.

The standard operation of extract-max:

Is this good for us ?

No !

Standard Extract-max is Not GoodThree possible heaps with 4 elements:

One heap has probability 1/3 while the other has probability of 2/3 !

4

3 2

1

1/3

4

3 2

1

1/3

4

2 3

1

1/3

4

3 1

2

1/3

3

1 2

3

2 1

3

2 1

Naive Implementation: Extract-max

Extract-max(H)1. T = build-heap-1(H)2. Remove the last node v in the tree (T’).3. H’ = build-heap(T’)4. If we already removed the maximal value

return H’ Otherwise:5. Replace the root with v and let v sift down

to its correct position.

build-heap-1 and build-heap works in O(n)

… but this implementation is history independent.

Analysis: Extract-max

Extract-max(H)1.T = build-heap-1(H)2.Remove the last node v in the tree (T’).

3.H’ = build-heap(T’)

H’ is a random uniform heap on the n original keys of the heap excluding a random key v.

• T is a random uniform permutation on the n+1 keys of the heap.

• T’ is a random uniform permutation on n keys of the heap excluding the random key v.

Analysis: Extract-max

4.If we already removed the maximal value return H’ Otherwise:

5.Replace the root with v and let v sift downto its correct position.

• If we already removed the maximal value we are done.

• Otherwise: This is just applying increase/decrease-key on the value at the root. (this is a 1-1 function …)

Improving Complexity: Extract-max

First 3 steps of Extract-max(H)1.T = build-heap-1(H)2.Remove the last node v in the tree.3.H’ = build-heap(T’)

Main problem - steps 1 to 3 that takes O(n).

Simple observation reduces the complexity of these steps to O(log2(n)) instead of O(n)

Reducing the Complexity to O(log2(n))

Observation: Most of the operations of build-heap-1 are redundant. they are always canceled by the operation of build-heap. Only the operations applied on nodes lying on the path from the root to the last leaf are really needed.

10

7 9

36 4 8

1 5 2

Reducing the Complexity to O(log2(n))

10

7 9

36 4 8

1 5 2

Complexity analysis: • Each heapify-1 and heapify operation takes at

most O(log n). • There are O(log n) such operations.

Reducing the Complexity:O(log(n)) Expected Time

Extract-max(H)1. T = build-heap-1(H)2. Remove the last node v in the tree (T’).3. H’ = build-heap(T’)4. If we already removed the maximal value

return H’ Otherwise:5. Replace the root with v and let v sift down

to its correct position.We actually remove the last value of a uniformly chosen permutation and

build back the heap


This is the most complex part

Main ideas:• We can show that there are actually O(1)

operations of heapify-1 and heapify that make a differnce (in average over the random choices made by the algorithm in each step).

• We can detect these operations and apply only them.


Main lemma:When applying build-heap on a uniformly

chosen permutation:The height of the last value in the

permutation is O(1).

Proof idea:• Backward analysis on build-heap-1 instead

of build-heap.

The Insert Operation

• The standard implementation of insert is not good for us.

• Good implementation must use randomization in order to be efficient (otherwise it should be canonical …)

• Making insert history independent is also not easy.

• The general method is similar to Extract-max.

Naive Implementation: Insert

Insert(H, v)1.Choose uniformly a random number

1≤i≤n+12.Let vi be the value in the heap in that

place.3. If i=n+1 skip to step 54.H’ Increase-key(H, i, v)• H’ is a uniformly chosen heap without the value vi

• The value vi not in the heap is a randomly chosen value.

Naive Implementation: Insert

Insert(H, v)5.T = build-heap-1(H’)6.T’ T “+” Add the value vi to the n+1

position7. H = build-heap(T’)8.Return (H)• T is a uniformly chosen permutation

without the value vi.• T’ is a uniformly chosen permutation

with the value vi.• H is a uniformly chosen heap with the

value v.

Insert Operation – Reducing complexity

• The general ideas are similar to Extract-max.

• Reducing the complexity to O(log2n) by running heapify-1 and heapify only on the path to the newly added node.

• The most difficult part is again reducing the complexity from O(log2n) to O(log n) expected time

** notice that we insert a random key into a random heap.

Conclusions

1.Demanding strong history independence usually requires a high efficiency penalty in the comparison based model.

2.Weak history independent heap in the

comparison-based model without penalty,

Complexity:• build-heap - O(n) worst case. • increase-key - O(log n) worst case.• extract-max, insert- O(log n) expected

time, O(log2n) worst case.

Bounds Summary

Operation Weak History Independence


heap: insert O(log n) Ω(n)

heap: increase-key O(log n) Ω(n)

heap: extract-max O(log n) No lower bound

heap: build-heap O(n) Ω(n log n)

queue: max{ insert-first, remove-last}

O(1) Ω(n)

Memory allocation

• Assume we allocate fixed size records.• We would like: After an arbitrary number

of allocate/delete the memory “dump” do not reveal information about the allocations.

Main idea:When allocate a cell k: • Choose random number 1≤i≤k• Put the cell in the ith place. Copy the ith

cell to the kth position. Require to change all incoming pointers into the two cells.

Memory allocation

• Require to change all incoming pointers into the two cells.

• Can be done using doubly linked pointers.

• We can make any pointer based with bounded in degree, fixed size record data structure history independent. If its “shape” is history independent

• Example: 2-3 trees we saw.

Memory allocation: non-fixed size

Main idea:• We partition the allocation into sizes of [2i,

2i+1).• The larger allocations are more left according

to their order.• Each group is uniformly ordered as fixed size.

When we allocate: • We round up the size.• We make place moving records in

“smaller” groups.• Allocation of size ‘s’ in time O(s log s).

Open Questions

1.Can we show separation between weak and strong History independence in the non-comparison model ?

2.History independent implementation of other, more complex, data structures.

3.Strong History independent implementations that do not require canonical representation – Union find.

Thank you

history independent data-structures. what is history independent data-structure ? sometimes data...

Documents

example data structure

data structure implementation

datastructure layout

history of operations

data structures

strong history independence

array layout

weak history independence