data structures and algorithms course slides: hashing algis

Data Structuresand

AlgorithmsCourse slides: Hashing

www.mif.vu.lt/~algis

2

Data Structures for Sets

Many applications deal with sets.

Compilers have symbol tables (set of vars, classes) Dictionary is a set of words. Routers have sets of forwarding rules. Web servers have set of clients, etc.

A set is a collection of members

No repetition of members Members themselves can be sets

3

Data Structures for Sets

Examples

Set of first 5 natural numbers: {1,2,3,4,5} {x | x is a positive integer and x < 100} {x | x is a CA driver with > 10 years of driving

experience and 0 accidents in the last 3 years}

4

Set Operations

Unary operation: min, max, sort, makenull, …

Binary operations

Member Set

Member Order (=, <, >)

Find, insert, delete, split, …

Set Find, insert, delete, split, …

Union, intersection, difference, equal, …

5

Observations

Set + Operations define an ADT.

A set + insert, delete, find

A set + ordering

Multiple sets + union, insert, delete

Multiple sets + merge

Etc.

Depending on type of members and choice of operations, different implementations can have different asymptotic complexity.

6

Dictionary ADTs

Maintain a set of items with distinct keys with:

find (k): find item with key k insert (x): insert item x into the dictionary remove (k): delete item with key k

Where do we use them:

Symbol tables for compiler Customer records (access by name) Games (positions, configurations) Spell checkers Peer to Peer systems (access songs by name), etc.

7

Naïve Implementations

The simplest possible scheme to implement a dictionary is “log file” or “audit trail”.

Maintain the elements in a linked list, with insertions occuring at the head.

The search and delete operations require searching the entire list in the worst-case.

Insertion is O(1), but find and delete are O(n).

A sorted array does not help, even with ordered keys. The search becomes fast, but insert/delete take O(n).

8

Hash Tables: Intuition

Hashing is function that maps each key to a location in memory.

A key’s location does not depend on other elements, and does not change after insertion.

unlike a sorted list A good hash function should be easy to compute.

With such a hash function, the dictionary operations can be implemented in O(1) time

9

Hash Tables: Intuition

Let us denote the set of all possible key values (i.e., the universe of keys) used in a dictionary application by U. Suppose an application requires a dictionary in which elements are assigned keys from the set of small natural numbers. That is, U ⊂ Z+ and U is relatively small. ⏐ ⏐

If no two elements have the same key, then this dictionary can be implemented by storing its elements in the array T[0, ... , U ⏐ ⏐ - 1]. This implementation is referred to as a direct-access table since each of the requisite DICTIONARY ADT operations - Search, Insert, and Delete - can always be performed in Θ(1) time by using a given key value to index directly into T, as shown:

10

Hash Tables: Intuition

The obvious shortcoming associated with direct-access tables is that the set U rarely has such "nice" properties. In practice, U can be quite large. This will lead to wasted memory if the ⏐ ⏐number of elements actually stored in the table is small relative to U . ⏐ ⏐

Furthermore, it may be difficult to ensure that all keys are unique. Finally, a specific application may require that the key values be real numbers, or some symbols which cannot be used directly to index into the table.

An effective alternative to direct-access tables are hash tables. A hash table is a sequentially mapped data structure that is similar to a direct-access table in that both attempt to make use of the random- access capability afforded by sequential mapping.

11

Hash Tables: Intuition

Hash Tables

All search structures so far

Relied on a comparison operation

Performance O(n) or O( log n)

Assume I have a function

f ( key ) ® integer ie one that maps a key to an integer

What performance might I expect now?

Hash Tables - Structure

Simplest case:

Assume items have integer keys in the range 1 .. m Use the value of the key itself

to select a slot in a direct access table in which to store the item

To search for an item with key, k,just look in slot k If there’s an item there,

you’ve found it

If the tag is 0, it’s missing.

Constant time, O(1)

14

Hashing : the basic idea Map key values to hash table addresses

keys -> hash table address

This applies to find, insert, and remove

Usually: integers -> {0, 1, 2, …, Hsize-1}Typical example: f(n) = n mod Hsize

Non-numeric keys converted to numbers

For example, strings converted to numbers as Sum of ASCII values

First three characters

15

Hashing : the basic idea9

10

20

39

4

14

8

Perm # (mod 9)

Student Records

Hash Tables - Choosing the Hash Function

Uniform hashing

Ideal hash function P(k) = probability that a key, k, occurs

If there are m slots in our hash table,

a uniform hashing function, h(k), would ensure:

or, in plain English,

the number of keys that map to each slot is equal

S P(k) =k | h(k) = 0

S P(k) = ....k | h(k) = 1

S P(k) =k | h(k) = m-1

1m

Read as sum over all k such that h(k) = 0

Hash Tables - A Uniform Hash Function

If the keys are integers randomly distributed in [ 0 , r ), then

is a uniform hash function

Most hashing functions can be made to map the keys to [ 0 , r ) for some r, eg adding the ASCII codes for characters mod 255 will give values in [ 0, 256 ) or [ 0, 255 ]

Replace + by xor

same range without the mod operation

Read as 0 £ k < r h(k) = mk

r

Hash Tables - Reducing the range to [ 0, m )

We’ve mapped the keys to a range of integers 0 £ k < r

Now we must reduce this range to [ 0, m )

where m is a reasonable size for the hash table

Strategies

Division - use a mod function Multiplication Universal hashing

Hash Tables - Reducing the range to [ 0, m )

Division

Use a mod function

h(k) = k mod m

Choice of m? Powers of 2 are generally not good!

h(k) = k mod 2n selects last n bits of k All combinations are not generally equally likely

Prime numbers close to 2n seem to be good choices eg want ~4000 entry table, choose m = 4093

0110010111000011010

k mod 28 selects these bits

Hash Tables - Reducing the range to [ 0, m )

Multiplication method

Multiply the key by constant, A, 0 < A < 1 Extract the fractional part of the product ( kA - ëkA û )

Multiply this by m h(k) = ëm * ( kA - ëkA û )û

Now m is not critical and a power of 2 can be chosen So this procedure is fast on a typical digital computer Set m = 2p

Multiply k (w bits) by ëA•2w û ç 2w bit product Extract p most significant bits of lower half A = ½(Ö5 -1) seems to be a good choice (see Knuth)

Hash Tables - Reducing the range to [ 0, m )

Universal Hashing

A determined “adversary” can always find a set of data that will defeat any hash function

Hash all keys to same slot ç O(n) search

Select the hash function randomly (at run time)from a set of hash functions

Reduced probability of poor performance

Set of functions, H, which map keys to [ 0, m )

H, is universal, if for each pair of keys, x and y,the number of functions, h Ì H,for which h(x) = h(y) is |H |/m

Hash Tables - Reducing the range to ( 0, m ]

Universal Hashing

A determined “adversary” can always find a set of data that will defeat any hash function

Hash all keys to same slot ç O(n) search

Select the hash function randomly (at run time)from a set of hash functions

---------

Functions are selected at run time Each run can give different results

Even with the same data

Good average performance obtainable

Hash Tables - Reducing the range to ( 0, m ]

Universal Hashing

Can we design a set of universal hash functions?

Quite easily

Key, x = x0, x1, x2, ...., xr

Choose a = <a0, a1, a2, ...., ar>a is a sequence of elements chosen randomly from { 0, m-1 }

ha(x) = S aixi mod m

There are mr+1 sequences a,so there are mr+1 functions, ha(x)

Theorem

The ha form a set of universal hash functions

Proof:See Cormen

Hash Tables - Constraints

Constraints

Keys must be unique

Keys must lie in a small range

For storage efficiency,keys must be dense in the range

If they’re sparse (lots of gaps between values),a lot of space is used to obtain speed Space for speed trade-off

Hash Tables - Relaxing the constraints

Keys must be unique

Construct a linked list of duplicates “attached” to each slot

If a search can be satisfiedby any item with key, k,performance is still O(1)

but

If the item has some other distinguishing featurewhich must be matched,we get O(nmax)

where nmax is the largest number of duplicates - or length of the longest chain

Hash Tables - Relaxing the constraints

Keys are integers

Need a hash functionh( key ) ® integer

ie one that maps a key to an integer

Applying this function to thekey produces an address

If h maps each key to a uniqueinteger in the range 0 .. m-1then search is O(1)

Hash Tables - Hash functions

Form of the hash function

Example - using an n-character key

int hash( char *s, int n ) { int sum = 0; while( n-- ) sum = sum + *s++; return sum % 256; }returns a value in 0 .. 255

xor function is also commonly used sum = sum ^ *s++;

But any function that generates integers in 0..m-1 for some suitable (not too large) m will do

As long as the hash function itself is O(1) !

Hash Tables - Collisions

Hash function

With this hash function

int hash( char *s, int n ) { int sum = 0; while( n-- ) sum = sum + *s++; return sum % 256; }

hash( “AB”, 2 ) andhash( “BA”, 2 )return the same value!

This is called a collision

A variety of techniques are used for resolving collisions

Hash Tables - Collision handling

Collisions

Occur when the hash function maps two different keys to the same address

The table must be able to recognise and resolve this

Recognise Store the actual key with the item in the hash table

Compute the address k = h( key )

Check for a hit if ( table[k].key == key ) then hit

else try next entry

Resolution Variety of techniques

Hash Tables - Linked lists

Collisions - Resolution

Linked list attached to each primary table slot

h(i) == h(i1)

h(k) == h(k1) == h(k2)

Searching for i1 Calculate h(i1)

Item in table, i,doesn’t match

Follow linked list to i1

If NULL found, key isn’t in table

Hash Tables - Overflow area

Overflow area Linked list constructed

in special area of tablecalled overflow area

h(k) == h(j)

k stored first

Adding j Calculate h(j)

Find k

Get first slot in overflow area

Put j in it

k’s pointer points to this slot

Searching - same as linked list

h’(x) - second hash function

Hash Tables - Re-hashing Use a second hash function Many variations

General term: re-hashing

h(k) == h(j)

k stored first

Adding j Calculate h(j)

Find k

Repeat until we find an empty slot Calculate h’(j)

Put j in it

Searching - Use h(x), then h’(x)

Hash Tables - Re-hash functions

The re-hash function Many variations

Linear probing h’(x) is +1

Go to the next slotuntil you find one empty

Can lead to bad clustering

Re-hash keys fill in gapsbetween other keys and exacerbatethe collision problem

Hash Tables - Re-hash functions

The re-hash function Many variations

Quadratic probing h’(x) is c i2 on the ith probe

Avoids primary clustering

Secondary clustering occurs All keys which collide on h(x) follow the same sequence

First

a = h(j) = h(k)

Then a + c, a + 4c, a + 9c, ....

Secondary clustering generally less of a problem

Hash Tables - Collision Resolution Summary

Chaining

+ Unlimited number of elements

+ Unlimited number of collisions

- Overhead of multiple linked lists

Re-hashing

+ Fast re-hashing

+ Fast access through use of main table space

- Maximum number of elements must be known

- Multiple collisions become probable

Overflow area

+ Fast access

+ Collisions don't use primary table space

- Two parameters which govern performance need to be estimated

Hash Tables - Collision Resolution Summary

Re-hashing

+ Fast re-hashing

+ Fast access through use of main table space

- Maximum number of elements must be known

- Multiple collisions become probable

Overflow area

+ Fast access

+ Collisions don't use primary table space

- Two parameters which govern performance need to be estimated

Hash Tables - Summary so far ...

Potential O(1) search time

If a suitable function h(key) ® integer can be found Space for speed trade-off

“Full” hash tables don’t work (more later!) Collisions

Inevitable Hash function reduces amount of information in key

Various resolution strategies Linked lists Overflow areas Re-hash functions

Linear probing h’ is +1 Quadratic probing h’ is +ci2

Any other hash function! or even sequence of functions!

38

Hashing:

Choose a hash function h; it also determines the hash table size.

Given an item x with key k, put x at location h(k).

To find if x is in the set, check location h(k).

What to do if more than one keys hash to the same value. This is called collision.

We will discuss two methods to handle collision:

Separate chaining

Open addressing

39

Maintain a list of all elements that hash to the same value

Search -- using the hash function to determine which list to traverse

Insert/deletion–once the “bucket” is found through Hash, insert and delete are list operations

Separate chaining

class HashTable {…… private:

unsigned int Hsize;

List<E,K> *TheList;

……

find(k,e)HashVal = Hash(k,Hsize);if

(TheList[HashVal].Search(k,e))then return true;else return false;

14

42

29

20

1

36

5623

16

24

31

177

0123456789

10

40

Insertion: insert 53

14

42

29

20

1

36

5623

16

24

31

177

0123456789

10

53 = 4 x 11 + 953 mod 11 = 9

14

42

29

20

1

36

5623

16

24

53

177

0123456789

1031

41

Analysis of Hashing with Chaining

Worst case

All keys hash into the same bucket

a single linked list.

insert, delete, find take O(n) time.

Average case

Keys are uniformly distributed into buckets

O(1+N/B): N is the number of elements in a hash table, B is the number of buckets.

If N = O(B), then O(1) time per operation.

N/B is called the load factor of the hash table.

42

Open addressing

If collision happens, alternative cells are tried until an empty cell is found.

Linear probing :Try next available position

0123456789

10

42

9

14

1

16

24

31

287

43

Linear Probing (insert 12)

0123456789

10

42

9

14

1

16

24

31

287

12 = 1 x 11 + 112 mod 11 = 1

0123456789

10

42

9

14

1

16

24

31

287

12

44

Search with linear probing (Search 15)

15 = 1 x 11 + 415 mod 11 = 4

0123456789

10

42

9

14

1

16

24

31

287

12

NOT FOUND !

45

Deletion in Hashing with Linear Probing

Since empty buckets are used to terminate search, standard deletion does not work.

One simple idea is to not delete, but mark.

Insert: put item in first empty or marked bucket.

Search: Continue past marked buckets.

Delete: just mark the bucket as deleted.

Advantage: Easy and correct.

Disadvantage: table can become full with dead items.

46

Deletion with linear probing: LAZY (Delete 9)

9 = 0 x 11 + 99 mod 11 = 9

0123456789

10

42

9

14

1

16

24

31

287

12

FOUND !

0123456789

10

42

D

14

1

16

24

31

287

12

47

remove(j) { i = j;

empty[i] = true;i = (i + 1) % D; // candidate for swappingwhile ((not empty[i]) and i!=j) {

r = Hash(ht[i]); // where should it go without collision? // can we still find it based on the rehashing strategy?

if not ((j<r<=i) or (i<j<r) or (r<=i<j))

then break; // yes find it from rehashing, swap

i = (i + 1) % D; // no, cannot find it from rehashing

}if (i!=j and not empty[i])then {

ht[j] = ht[i];remove(i);

}}

Eager Deletion: fill holes

Remove and find replacement:Fill in the hole for later searches

48

Eager Deletion Analysis (cont.)

If not full After deletion, there will be at least two holes

Elements that are affected by the new hole are Initial hashed location is cyclically before the new

hole

Location after linear probing is in between the new hole and the next hole in the search order

Elements are movable to fill the hole

Next hole in the search orderNew hole

Initialhashed location

Location after linear probing

Next hole in the search order

Initialhashed location

49

Eager Deletion Analysis (cont.) The important thing is to make sure that if a

replacement (i) is swapped into deleted (j), we can still find that element. How can we not find it?

If the original hashed position (r) is circularly in between deleted and the replacementj r i

j ri

jr i

i rWill not find i past the empty green slot!

j i r i r

Will find i

50

Quadratic Probing

Solves the clustering problem in Linear Probing

Check H(x)

If collision occurs check H(x) + 1

If collision occurs check H(x) + 4

If collision occurs check H(x) + 9

If collision occurs check H(x) + 16

...

H(x) + i2

51

Quadratic Probing (insert 12)

0123456789

10

42

9

14

1

16

24

31

287

12 = 1 x 11 + 112 mod 11 = 1

0123456789

10

42

9

14

1

16

24

31

287

12

52

Double Hashing

When collision occurs use a second hash function

Hash2 (x) = R – (x mod R) R: greatest prime number smaller than table-size

Inserting 12

H2(x) = 7 – (x mod 7) = 7 – (12 mod 7) = 2 Check H(x) If collision occurs check H(x) + 2 If collision occurs check H(x) + 4 If collision occurs check H(x) + 6 If collision occurs check H(x) + 8 H(x) + i * H2(x)

53

Double Hashing (insert 12)

0123456789

10

42

9

14

1

16

24

31

287

12 = 1 x 11 + 112 mod 11 = 1

7 –12 mod 7 = 2

0123456789

10

42

9

14

1

16

24

31

287

12

54

Rehashing

If table gets too full, operations will take too long.

Build another table, twice as big (and prime).

Next prime number after 11 x 2 is 23

Insert every element again to this table

Rehash after a percentage of the table becomes full (70% for example)

55

Good and Bad Hashing Functions

Hash using the wrong key

Age of a student

Hash using limited information

First letter of last names (a lot of A’s, few Z’s)

Hash functions choices :

keys evenly distributed in the hash table

Even distribution guaranteed by “randomness”

No expectation of outcomes

Cannot design input patterns to defeat randomness

56

Examples of Hashing Function

B=100, N=100, keys = A0, A1, …, A99

Hashing(A12) = (Ascii(A)+Ascii(1)+Ascii(2)) / B

H(A18)=H(A27)=H(A36)=H(A45) …

Theoretically, N(1+N/B)= 200

In reality, 395 steps are needed because of collision

How to fix it?

Hashing(A12) = (Ascii(A)*22+Ascii(1)*2+Ascci(2))/B

H(A12)!=H(A21)

Examples: numerical keys

Use X2 and take middle numbers

Collision Frequency

Birthdays or the von Mises paradox

There are 365 days in a normal year Birthdays on the same day unlikely?

How many people do I need before “it’s an even bet”(ie the probability is > 50%)that two have the same birthday?

View the days of the year as the slots in a hash table

the “birthday function” as mapping people to slots

Answering von Mises’ question answers the question about the probability of collisions in a hash table

Distinct Birthdays

Let Q(n) = probability that n people have distinct birthdays

Q(1) = 1

With two people, the 2nd has only 364 “free” birthdays

The 3rd has only 363, and so on:

Q(2) = Q(1) * 364

365

Q(n) = Q(1) * 364

365

364

365

365-n+1

365* * … *

Coincident Birthdays

Probability of having two identical birthdays

P(n) = 1 - Q(n)

P(23) = 0.507

With 23 entries,table is only23/365 = 6.3%full!

0.0000.1000.2000.3000.4000.5000.600

0.7000.8000.9001.000

0 20 40 60 80

Hash Tables - Load factor

Collisions are very probable!

Table load factor must be kept low

Detailed analyses of the average chain length(or number of comparisons/search) are available

Separate chaining

linked lists attached to each slot gives best performance

but uses more space!

n

m

n = number of items

m = number of slots

Hash Tables - General Design

Choose the table size

Large tables reduce the probability of collisions!

Table size, m

n items

Collision probability a = n / m

Choose a table organisation

Does the collection keep growing? Linked lists (....... but consider a tree!)

Size relatively static? Overflow area or

Re-hash

Choose a hash function

....

Hash Tables - General Design

Choose a hash function

A simple (and fast) one may well be fine ...

Read your text for some ideas!

Check the hash function against your data

Fixed data Try various h, m

until the maximum collision chain is acceptable

Known performance

Changing data Choose some representative data

Try various h, m until collision chain is OK

Usually predictable performance

Hash Tables - Review

If you can meet the constraints

Hash Tables will generally give good performance

O(1) search

Like radix sort, they rely on calculating an address from a key

But, unlike radix sort, relatively easy to get good performance with a little experimentation

\ not advisable for unknown data

collection size relatively static

memory management is actually simpler All memory is pre-allocated!

64

Collision Functions

Hi(x)= (H(x)+i) mod B

Linear pobing

Hi(x)= (H(x)+ci) mod B (c>1)

Linear probing with step-size = c

Hi(x)= (H(x)+i2) mod B

Quadratic probing

Hi(x)= (H(x)+ i * H2(x)) mod B

65

Analysis of Open Hashing

Effort of one Insert?

Intuitively – that depends on how full the hash is Effort of an average Insert?

Effort to fill the Bucket to a certain capacity?

Intuitively – accumulated efforts in inserts Effort to search an item (both successful and

unsuccessful)?

Effort to delete an item (both successful and unsuccessful)?

Same effort for successful search and delete? Same effort for unsuccessful search and delete?

Picking a Hash Function In practice, hash functions often use the bit representation

of values.

e.g. compute the binary representation of the characters in the search key and return the sum modulus the number of buckets. e.g. distribute first names over 59 buckets. Use ASCII values of the

letters e.g. John (74 + 111 + 104 + 110 = 399), 399 modulo 59 = bucket 45.

In practice schemes are more complicated than the above. See the big white algorithm book for more details (Cormen, Leiserson, Rivest).

For a hash table the idea is that there will be n entries.

n = the number of actual values. the hope is that each of the n values maps to a different index (no

collisions).

For DB hash indexing the index is divided into buckets.

each bucket maps to a disk page (which maps to a disk block). The hash function maps index entries to buckets with the intent that

no bucket overflows.

Hash Function Problems

Poor distribution of values to buckets.

This can happen because the hash function is either not random or not uniform.

Solution: Change the hash function.

Skewed data distribution.

The distribution of search key values is skewed. That is, there is not a uniform distribution of values – many incidences of

some values and few of others.

Solution: There is no way this can be addressed by changing the hash function. If this is a problem then a hash index may not be a good choice.

Collisions

Overflow may be caused by inserts of data entries with the same hash value (but different search key values).

Solution: Static hashing does not address this.

Extendible hashing and linear hashing deal with collisions to some extent.

Static Hashing Hash function

A uniform and random hash function h evenly maps search key values to a primary page.

With N buckets the hash function maps a search key value to buckets 0 to N-1.

Primary pages

The number of buckets (pages) is pre-determined. Each bucket is a disk page.

Overflow pages

As the DB grows primary pages become full. Additional data is placed in overflow pages chained to the primary pages. Finding a value involves searching these pages.

Performance

Generally a search requires one disk access. Insert or delete require two disk accesses. The number of primary pages is fixed.

The structure becomes inefficient as the file grows (because of the overflow pages).

It wastes space if the file shrinks.

Re-hashing removes overflow until the DB grows again. This process is time-consuming and the index is inaccessible during it.

Extendible Hashing The efficiency of static hashing declines as the file grows.

Overflow pages increase search time. One solution would be to use a range of hash functions based on a

bit value and double the number of buckets (and the function range) whenever an overflow page is needed. Such a reorganization is expensive.

Is it possible to make local changes?

Use a directory of pointers to buckets.

Double the directory size when required. Only split the pages that have overflowed. The directory need only consist of an array of pointers to pages so

is relatively compact. The array index represents the value computed by the hash

function. At any time the array size is determined by how many bits of the

hash result are being used. Usually the last d (least significant) bits are used.

Basic Structure

The array index is the last two bits of the hash value. Note that the values in the cells represents the hash value (not the

search key value). Assume that three records fit on a page (so each bucket is a disc page). There are only four pages of data and none of the pages is full – only two

bits are required as an index.

64 16

00 1 17 5

01

10

11 6

31 15

Inserting Values

Insert 14 and 9 into the index shown in the previous example. 14 fits but inserting 9 causes the page to overflow. Double the directory and split the overflowing bucket (01) into two.

Distribute the entries based on the last three digits of the hash value.

Directory pointers to the existing buckets are added for the other three new three digit hash values.

64 16

00 1 17 5

01 9

10

11 6 14

31 15

insert

Inserting Values Example64 16

1 17 9

000

001

010 6 14

011

100

101 31 15

110

111

5

new bucket Note how the directory has doubled in size but only one new

bucket has been created. The directory is small (each entry consists of a value and a

pointer). New index pages (buckets) are kept to a minimum.

Keeping Track of Bits

After the directory is doubled some pages will be referenced by two directory entries.

If the referenced pages become full, subsequent insertions will require a new page to be allocated but will not require doubling the directory.

If pages that are referenced by only one directory entry overflow the directory will have to be doubled again.

Contrast inserting 4 and 12 (x00) into the example with inserting 25 (x01).

How do we keep track of whether or not an insert requires that the directory is doubled?

Record the global depth of the hashed file. The number of bits required for the directory.

Also record the local depth of each page. The number of bits needed to reference a particular page.

Local and Global Depth2

64 16

3

1 17 9

000

001 2

010 6 14

011

100 2

101 31 15

110

111 3

5

3

local depthglobal depth

Summary

Create initial hash file and directory. Each directory entry points to a different bucket. All local depths are equal to the global depth.

Insert values If no overflow occurs insert entry and finish. If the bucket overflows compare the local depth of the

bucket with the global depth. If the local depth is less than the global depth create a new

bucket and distribute the entries with no change to the directory. Increment the local depth of the split buckets.

If the local depth is the same as the global depth double the directory, and split the bucket. Increment the local depth of the split buckets and the global depth of the directory.

Delete values If the deletion empties the bucket it can be merged and

the local depth decremented. If the local depth of all buckets is less than the global

depth the directory can be halved. In practice this is often not done.

Performance Performance is identical to static hashing if the directory fits in

memory. If the directory does not fit in memory an additional disk access is required.

It is likely that the directory will fit in memory.

Collisions and Performance Collisions at low global depth are dealt with by doubling the directory, and

the range of the hash function, and making local index changes.

Many collisions will result in producing a large directory (that might not fit in memory).

Overflow pages If many entries have the same hash value across the entire range of bits

overflow pages have to be allocated, reducing efficiency.

This is because splitting a bucket (and doubling the directory) would cause another split if all the entries map the same bucket after splitting.

This can occur if the hash function is poor.

Collisions will occur if the distribution of values is skewed.

Linear Hashing

Another dynamic system.

Like extendible hashing insertions and deletions are efficiently accommodated.

Unlike extendible hashing a directory is not required. Collisions may result in chains of overflow pages.

Linear hashing, like extendible hashing uses a family of hash functions.

Each function’s range is twice that of its predecessor. Pages are split when overflows occur – but not necessarily

the page with the overflow. Splitting occurs in turn, in a round robin fashion. When all the pages at one level (the current hash function)

have been split a new level is applied. Splitting occurs gradually Primary pages are allocated consecutively.

Levels of Linear Hashing Initial Stage.

The initial level distributes entries into N0 buckets. Call the hash function to perform this h0.

Splitting buckets. If a bucket overflows its primary page is chained to an overflow page

(increasing the bucket’s size). Also when a bucket overflows some bucket is split.

The first bucket to be split is the first bucket in the file (not necessarily the bucket that overflows).

The next bucket to be split is the second bucket in the file … and so on until the Nth. has been split.

When buckets are split their entries (including those in overflow pages) are distributed using h1.

To access split buckets the next level hash function (h1) is applied. h1 maps entries to 2N0 (or N1)buckets.

Level progression. Once all Ni buckets of the current level (i) are split the hash function hi is

replaced by hi+1. The splitting process starts again at the first bucket and hi+2 is applied to find

entries in split buckets.

Linear Hashing Example

The example above shows the index level equal to 0 where N0 equals 4 (three entries fit on a page).

h0 maps index entries to one of four buckets.

Given the initial page of the file the appropriate primary page can be determined by using an offset. i.e. initial page + h0(search key value)

In the above example only h0 is used and no buckets have been split.

Now consider what happens when 9 is inserted (which will not fit in the second bucket).

Note that next indicates which bucket is to split next.

next 64 36

1 17 5

6

31 15

Linear Hashing Example 2

The page indicated by next is split (the first one).

Next is incremented.

An overflow page is chained to the primary page to contain the inserted value.

If h0 maps a value from zero to next – 1 (just the first page in this case) h1 must be used to where to insert the new entry.

Note how the new page falls naturally into the sequence as the fifth page.

h1 next 64

h0 next 1 17 5 9

h0 6

h0 31 15

h1 36

Linear Hashing Example 3

Assume inserts of 8, 7, 18, 14, 111, 32, 162, 10, 13, 233

After the 2nd. split the base level is 1 (N1 = 8), use h1.

Subsequent splits will use h2 for inserts between the first bucket and next-1.

2 1

h1 h1 next3 64 8 32 16

h1 h1 1 17 9

h1 h0 next1 10 186 18 14

h0 h0 next2 1131 15 7 11

h1 h1 36

h1 h1 5 13

h1 - 6 14

- - 31 15 7 23

Comparing Linear and Extendible Hashing

Differences

Because buckets are split in turn linear hashing does not need a directory.

Extendible hashing may lead to better use of space because the overflowing bucket is always the one that is split.

In particular, linear hashing does not deal elegantly with collisions in that long overflow chains may develop.

Collisions with extendible hashing lead to a large directory.

Similarities

Doubling the directory and moving to the next level of hash function have the same effect: the range of buckets is doubled.

The hash functions used by the two schemes may be the same. i.e. they can both use a bit translation of the search key value modulus the number of buckets required.

83

Issues:

What do we lose?

Operations that require ordering are inefficient

FindMax: O(n) O(log n) Balanced binary tree

FindMin: O(n) O(log n) Balanced binary tree

PrintSorted: O(n log n) O(n) Balanced binary tree

What do we gain?

Insert: O(1) O(log n) Balanced binary tree

Delete: O(1) O(log n) Balanced binary tree

Find: O(1) O(log n) Balanced binary tree

How to handle Collision?

Separate chaining

Open addressing

Interface Main methods:

Void Put(Object)

Object Get(Object) … returns null if not i

… Remove(Object)

Goal: methods are O(1)! (ususally)

Implementation details

HashTable: the storage bin

hashfunction(object): tells where object should go

collision resolution strategy: what to do when two objects “hash” to same location.

In Java, all objects have default int hashcode(), but better to define your own. Except for strings.

String hashing in Java is good.

HashFunctions

Goal: map objects into table so distribution is uniform

Tricky to do.

Examples for string s

product ascii codes, then mod tablesize nearly always even, so bad

sum ascii codes, then mod tablesize may be too small

shift bits in ascii code java allows this with << and >>

Java does a good job with Strings

Example Problem

Suppose we are storing numeric id’s of customers, maybe 100,000

We want to check if a person is delinquent, usually less than 400.

Use an array of size 1000, the delinquents.

Put id in at id mod tableSize.

Clearly fast for getting, removing

But what happens if entries collide?

Separate Chaining

Array of linked lists

The hash function determines which list to search

May or may keep individual lists in sorted order

Problems:

needs a very good hash function, which may not exist

worse case: O(n)

extra-space for links

Another approach: Open Addressing

everything goes into the array, somehow

several approaches: linear, quadratic, double, rehashing

Linear Probing

Store information (or prts to objects) in array

Linear Probing

When inserting an object, if location filled, find first unfilled position. I.e look at hi(x)+f(i) where f(i)= i;

When getting an object, start at hash addresses, and do linear search till find object or a hole.

primary clustering blocks of filled cells occur

Harder to insert than find existing element

Load factor =lf = percent of array filled

Expected probes for insertion: 1/2(1+1/(1-lf)^2))

successful search: 1/2(1+1/(1-lf))

Expected number of probes

Load factor failure success

.1 1.11 1.06

.2 1.28 1.13

.3 1.52 1.21

.4 1.89 1.33

.5 2.5 1.50

.6 3.6 1.75

.7 6.0 2.17

.8 13.0 3.0

.9 50.5 5.5

Quadratic Probing

Idea: f(i) = i^2 (or some other quadratic function)

Problem: If table is more than 1/2 full, no quarantee of finding any space!

Theorem: if table is less than 1/2 full, and table size is prime, then an element can be inserted.

Good: Quadratic probing eliminates primary clustering

Quadratic probing has secondary clustering (minor)

if hash to same addresses, then probe sequence will be the same

Proof of theorem

Theorem: The first P/2 probes are distinct.

Suppose not.

Then there are i and j <P/2 that hash to same place

So h(x)+i^2 = h(y)+j^2 and h(x) = h(y).

So i^2 = j^2 mod P

(i+j)*(i-j) = 0 mod P

Since P is prime and i and j are less than P/2

then i+j and i-j are less than P and P factors.

Contradiction

Double Hashing

Goal: spreading out the probe sequence

f(i) = i*hash2(x), where hash2 is another hash function

Dangerous: can be very bad.

Also may not eliminate any problems

In best case, it’s great

Rehashing

All methods degrade when table becomes too full

Simpliest solution:

create new table, twice as large

rehash everything

O(N), so not happy if often

With quadratic probing, rehash when table 1/2 full

Extendible Hashing: Uses secondary storage

Suppose data does not fit in main memory Goal: Reduce number of disks accesses. Suppose N records to store and M records fit in a

disk block Result: 2 disk accesses for find (~4 for insert) Let D be max number of bits so 2^D < M. This is for root or directory (a disk block) Algo:

hash on first D bits, yields ptr to disk block Expected number of leaves: (N/M) log 2 Expected directory size: O(N^(1+1/M) / M) Theoretically difficult, more details for

implementation

Applications

Compilers: keep track of variables and scope

Graph Theory: associate id with name (general)

Game Playing: E.G. in chess, keep track of positions already considered and evaluated (which may be expensive)

Spelling Checker: At least to check that word is right.

But how to suggest correct word

Lexicon/book indices

HashSets vs HashMaps

HashSets store objects supports adding and removing in constant time

HashMaps store a pair (key,object) this is an implementation of a Map

HashMaps are more useful and standard

HashMaps main methods are: put(Object key, Object value)

get(Object key)

remove(Object key)

All done in expected O(1) time.

Lexicon Example

Inputs: text file (N) + content word file (the keys) (M)

Ouput: content words in order, with page numbers

Algo:

Define entry = (content word, linked list of integers)

Initially, list is empty for each word.

Step 1: Read content word file and Make HashMap of content word, empty list

Step 2: Read text file and check if work in HashMap;

if in, add to page number, else continue.

Step 3: Use the iterator method to now walk thru the HashMap and put it into a sortable container.

Lexicon Example

Complexity:

step 1: O(M), M number of content words

step 2: O(N), N word file size

step 3: O(M log M) max.

So O(max(N, M log M))

Dumb Algorithm

Sort content words O(Mlog M) (balanced tree)

Look up each word in Content Word tree and update

O(N*logM)

Total complexity: O(N log M)

N = 500*2000 =1,000,000 and M = 1000

Smart algo: 1,000,000; dumb algo: 1,000,000*10.

Memoization

Recursive Fibonacci:

fib(n) = if (n<2) return 1

else return fib(n-1)+fib(n-2)

Use hashing to store intermediate results

Hashtable ht;

fib(n) = Entry e = (Entry)ht.get(n);

if (e != null) return e.answer;

else if (n<2) return 1;

else ans = fib(n-1)+fib(n-2);

ht.put(n,ans);

return ans;

Appendix: Hashingfor

Databases

Contents

Static Hashing File Organization Properties of the Hash Function Bucket Overflow Indices

Dynamic Hashing Underlying Data Structure Querying and Updating

Comparisons Other types of hashing Ordered Indexing vs. Hashing

Static Hashing

Hashing provides a means for accessing data without the use of an index structure.

Data is addressed on disk by computing a function on a search key instead.

File organization

A bucket in a hash file is unit of storage (typically a disk block) that can hold one or more records.

The hash function, h, is a function from the set of all search-keys, K, to the set of all bucket addresses, B.

Insertion, deletion, and lookup are done in constant time.

Querying and Updates

To insert a record into the structure compute the hash value h(Ki), and place the record in the bucket address returned.

For lookup operations, compute the hash value as above and search each record in the bucket for the specific record.

To delete simply lookup and remove.

Properties of the Hash Function

The distribution should be uniform.

An ideal hash function should assign the same number of records in each bucket.

The distribution should be random.

Regardless of the actual search-keys, the each bucket has the same number of records on average

Hash values should not depend on any ordering or the search-keys

Bucket Overflow

How does bucket overflow occur?

Not enough buckets to handle data

A few buckets have considerably more records then others. This is referred to as skew. Multiple records have the same hash value

Non-uniform hash function distribution.

Solutions

Provide more buckets then are needed.

Overflow chaining

If a bucket is full, link another bucket to it. Repeat as necessary.

The system must then check overflow buckets for querying and updates. This is known as closed hashing.

Alternatives

Open hashing

The number of buckets is fixed

Overflow is handled by using the next bucket in cyclic order that has space. This is known as linear probing.

Compute more hash functions.

Note: Closed hashing is preferred in database systems.

Indices

A hash index organizes the search keys, with their pointers, into a hash file.

Hash indices never primary even though they provide direct access.

Example of Hash Index

Dynamic Hashing

More effective then static hashing when the database grows or shrinks

Extendable hashing splits and coalesces buckets appropriately with the database size.

i.e. buckets are added and deleted on demand.

The Hash Function

Typically produces a large number of values, uniformly and randomly.

Only part of the value is used depending on the size of the database.

Data Structure

Hash indices are typically a prefix of the entire hash value.

More then one consecutive index can point to the same bucket.

The indices have the same hash prefix which can be shorter then the length of the index.

General Extendable Hash Structure

In this structure, i2 = i3 = i, whereas i1 = i – 1

Queries and Updates

Lookup

Take the first i bits of the hash value.

Following the corresponding entry in the bucket address table.

Look in the bucket.

Queries and Updates (Cont’d)

Insertion

Follow lookup procedure

If the bucket has space, add the record.

If not…

Insertion (Cont’d)

Case 1: i = ij

Use an additional bit in the hash value This doubles the size of the bucket address table.

Makes two entries in the table point to the full bucket.

Allocate a new bucket, z. Set ij and iz to i

Point the second entry to the new bucket

Rehash the old bucket

Repeat insertion attempt

Insertion (cont’d)

Case 2: i > ij

Allocate a new bucket, z

Add 1 to ij, set ij and iz to this new value

Put half of the entries in the first bucket and half in the other

Rehash records in bucket j

Reattempt insertion

Insertion (Finally)

If all the records in the bucket have the same search value, simply use overflow buckets as seen in static hashing.

Use of Extendable Hash Structure: Example

Initial Hash structure, bucket size = 2

Example (Cont.)

Hash structure after insertion of one Brighton and two Downtown records

Example (Cont.)Hash structure after insertion of Mianus record

Example (Cont.)

Hash structure after insertion of three Perryridge records

Example (Cont.)

Hash structure after insertion of Redwood and Round Hill records

Comparison to Other Hashing Methods

Advantage: performance does not decrease as the database size increases

Space is conserved by adding and removing as necessary

Disadvantage: additional level of indirection for operations

Complex implementation

Ordered Indexing vs. Hashing

Hashing is less efficient if queries to the database include ranges as opposed to specific values.

In cases where ranges are infrequent hashing provides faster insertion, deletion, and lookup then ordered indexing.