Data Structures and

Algorithms

Course’s slides: Introduction, Basic data types

www.mif.vu.lt/~algis

Motto

ò  Data Structures and Algorithms

ò  The key to your professional reputation

ò  A much more dramatic effect can be made on the performance of a program by changing to a better algorithm than by

ò  hacking

ò  converting to assembler

ò  Buy a text for long-term reference!

ò  Professional software engineers have algorithms text(s) on their shelves

ò  Hackers have user manuals for the latest software package

Content

ò  1. Introduction, computing model, von Neuman principles, data, abstract data types, data structures, basic data types

ò  2. Sorting, internal sorting, quicksort

ò  3. Merge sort, von Neuman sorting, external sorting

ò  4. Abstract data types, stack, queue, programming of stack and queue

ò  5. Heap, priority queue, priority queue by heap structure, lists, list programming, dynamic sets ADT

ò  6. Hierarchical structures, binary search trees, tree allocation in memory

ò  7. AVL trees, 2-3-4 trees, red-black trees

Content

ò  8. B-trees and other similar trees, Huffman algorithm for data compression

ò  9. Hashing idea, hashing functions and tables, hashing procedures and algorithms, extendable hashing

ò  10. Radix search algorithms, radix trees, radix algorithms, radix search

ò  11. Patricia trees, suffix tree

ò  12. Text search

ò  13. Analysis of algorithms

Introduction

ò  Informally, algorithm means is a well-defined computational procedure that takes value (set of values) as input and produces some value (set of values) as output.

ò  An algorithm is thus a sequence of computational steps that transform the input into the output.

ò  Algorithm is also viewed as a tool for solving a well-specified problem, involving computers.

ò  There exist many points of view to algorithms. A good example of this is a famous Euclid’s algorithm:

ò  for two integers x, y calculate the greatest common divisor gcd (x, y). Direct implementation of the algorithm looks like:

Introduction

program euclid (input, output); var x,y: integer; function gcd (u,v: integer): integer;

var t: integer; begin repeat

if u<v then begin t := u; u := v; v := t end; u := u-v; until u = 0; gcd := v end;

begin while not eof do

begin readln (x, y); if (x>0) and (y>0) then writeln (x, y, gcd (x, y)) end;

end.

Introduction

ò  This algorithm has some exceptional features

ò  it is applicable only to numbers;

ò  it has to be changed every time when something of the environment changes, say if numbers are very long and does not fit into a size of variable (numbers like 1000!).

ò  For algorithms of applications in the focus of this course, like databases, information systems, etc., they are usually understood in a slightly different way:

ò  is repeated many times;

ò  in a various circumstancies;

ò  with different types of data.

Introduction ò  404800479988610197196058631666872994808558901323↵ 829669944590997424504087073759918823627727188732↵

519779505950995276120874975462497043601418278094↵ 646496291056393887437886487337119181045825783647↵ 849977012476632889835955735432513185323958463075↵ 557409114262417474349347553428646576611667797396↵ 668820291207379143853719588249808126867838374559↵ 731746136085379534524221586593201928090878297308↵ 431392844403281231558611036976801357304216168747↵ 609675871348312025478589320767169132448426236131↵ 412508780208000261683151027341827977704784635868↵ 170164365024153691398281264810213092761244896359↵ 928705114964975419909342221566832572080821333186↵ 116811553615836546984046708975602900950537616475↵ 847728421889679646244945160765353408198901385442↵ 487984959953319101723355556602139450399736280750↵ 137837615307127761926849034352625200015888535147↵ 331611702103968175921510907788019393178114194545↵ 257223865541461062892187960223838971476088506276↵ 862967146674697562911234082439208160153780889893↵ 964518263243671616762179168909779911903754031274↵ 622289988005195444414282012187361745992642956581↵ 746628302955570299024324153181617210465832036786↵ 906117260158783520751516284225540265170483304226↵ 143974286933061690897968482590125458327168226458↵ 066526769958652682272807075781391858178889652208↵ 164348344825993266043367660176999612831860788386↵ 150279465955131156552036093988180612138558600301↵ 435694527224206344631797460594682573103790084024↵ 432438465657245014402821885252470935190620929023↵ 136493273497565513958720559654228749774011413346↵ 962715422845862377387538230483865688976461927383↵ 814900140767310446640259899490222221765904339901↵ 886018566526485061799702356193897017860040811889↵ 729918311021171229845901641921068884387121855646↵ 124960798722908519296819372388642614839657382291↵ 123125024186649353143970137428531926649875337218↵ 940694281434118520158014123344828015051399694290↵ 153483077644569099073152433278288269864602789864↵ 321139083506217095002597389863554277196742822248↵ 757586765752344220207573630569498825087968928162↵ 753848863396909959826280956121450994871701244516↵ 461260379029309120889086942028510640182154399457↵ 156805941872748998094254742173582401063677404595↵ 741785160829230135358081840096996372524230560855↵ 903700624271243416909004153690105933983835777939↵ 410970027753472000000000000000000000000000000000↵ 000000000000000000000000000000000000000000000000↵ 000000000000000000000000000000000000000000000000↵ 000000000000000000000000000000000000000000000000↵ 000000000000000000000000000000000000000000000000↵ 000000000000000000000000

Von Neuman computing model

1943: ENIAC

•  Presper Eckert and John Mauchly -- first general electronic computer. Hard-wired program -- settings of dials and switches.

•  or John V. Atanasoff in 1939

1944: Beginnings of EDVAC

•  among other improvements, includes program stored in memory

1945: John von Neumann - wrote a report on the stored program concept, known as the First Draft of a Report on EDVAC , the “von Neumann machine” (or model):

•  a memory, containing instructions and data

•  a processing unit, for performing arithmetic and logical operations

•  a control unit, for interpreting instructions

Von Neuman computing model

valdymas skaičiavimas

registrai

RAM

3

5

8

PC

Von Neuman computing model

Memory Types: RAM

ò  RAM is typically volatile memory (meaning it doesn’t retain voltage settings once power is removed)

ò  RAM is an array of cells, each with a unique address

ò  A cell is the minimum unit of access. Originally, this was 8 bits taken together as a byte. In today’s computer, word-sized cells (16 bits, grouped in 4) are more typical.

ò  RAM gets its name from its access performance. In RAM memory, theoretically, it would take the same amount of time to access any memory cell, regardless of its location with the memory bank (“random” access).

Memory

University of Texas at Austin CS310H - Computer Organization Spring 2010 Don Fussell 4

Memory

2k x m array of stored bits

Address

unique (k-bit) identifier of location

Contents

m-bit value stored in location

Basic Operations:

LOAD

read a value from a memory location

STORE

write a value to a memory location

•••

0000

0001

0010

0011

0100

0101

0110

1101

1110

1111

00101101

10100010

Interface to memory

University of Texas at Austin CS310H - Computer Organization Spring 2010 Don Fussell 5

Interface to Memory

How does processing unit get data to/from memory?

MAR: Memory Address Register

MDR: Memory Data Register

To LOAD a location (A):

1. Write the address (A) into the MAR.

2. Send a “read” signal to the memory.

3. Read the data from MDR.

1. To STORE a value (X) to a location (A):

1. Write the data (X) to the MDR.

2. Write the address (A) into the MAR.

3. Send a “write” signal to the memory.

MEMORY

MAR MDR

Processing unit

University of Texas at Austin CS310H - Computer Organization Spring 2010 Don Fussell 6

Processing Unit

Functional Units

ALU = Arithmetic and Logic Unit

could have many functional units.

some of them special-purpose

(multiply, square root, …)

LC-3 performs ADD, AND, NOT

Registers

Small, temporary storage

Operands and results of functional units

LC-3 has eight registers (R0, …, R7), each 16 bits wide

Word Size

number of bits normally processed by ALU in one instruction

also width of registers

LC-3 is 16 bits

PROCESSING UNIT

ALU TEMP

The ALU

ò  The third component in the von Neumann architecture is called the Arithmetic Logic Unit.

ò  This is the subcomponent that performs the arithmetic and logic operations for which we have been building parts.

ò  The ALU is the “brain” of the computer.

Instruction processing

University of Texas at Austin CS310H - Computer Organization Spring 2010 Don Fussell 9

Instruction Processing

Decode instruction

Evaluate address

Fetch operands from memory

Execute operation

Store result

Fetch instruction from memory

Boolean data Data values: {false, true} In C/C++: false = 0, true = 1 (or nonzero) Could store 1 value per bit, but usually use a byte (or word)

M

M Operations: and && or || not !

&& 0 1

0 0 0

1 0 1

| | 0 1

0 0 1

1 1 1

x !x 0 1 1 0

Character Data Store numeric codes (ASCII, EBCDIC, Unicode) 1 byte for ASCII and EBCDIC, 2 bytes for Unicode

Basic operation: comparison of chars to determine if ==, <, >, etc. uses their numeric codes (i.e. uses their ordinal values)

Integer Data Nonegative (unsigned) integer:

type unsigned (and variations) in C++ Store its base-two representation in a fixed number w of bits

(e.g., w = 16 or w = 32)

88 = 00000000010110002

Signed integer: type int (and variations) in C++ Store in a fixed number w of bits using one of the following

representations:

Sign-magnitude representation

Save one bit (usually most significant) for sign (0 = +, 1 = – )

Use base-two representation in the other bits.

88 → _000000001011000

0↑ sign bit

1. Cumbersome for arithmetic computations

2. 2 0’s in this scheme

3.  Incrementing by one results in subtraction of one, not addition!

–88 → _000000001011000 ↓

1

Both 0 and -0

Complement representation

For negative n (–n): (1) Find w-bit base-2 representation of n (2) Complement each bit. (3) Add 1

Example: –88 1. 88 as a 16-bit base-two number

0000000001011000

Same as subtracting the number from 0!

Same as sign mag. For non-negative n:

Use ordinary base-two representation with leading (sign) bit 0

2. Complement this bit string 3. Add 1

1111111110100111 1111111110101000

(see p. 38)

5 + 7: 0000000000000101 +0000000000000111

5 + –6: 0000000000000101 +1111111111111010

These work for both + and – integers

0000000000001100

111←⎯⎯ carry bits

1111111111111111

+ 0 1

0 0 1

1 1 10

x 0 1

0 0 0

1 0 1

Good for arithmetic computation

Problems with Integer Representation

ò  Limited Capacity -- a finite number of bitsò  Overflow and Underflow:

Overflow- addition or multiplication can exceed largest value permitted by

storage schemeUnderflow

- subtraction or multiplication can exceed smallest value permitted by storage scheme

ò  Not a perfect representation of (mathematical) integersò  can only store a finite (sub)range of them

How is Real Data represented? Types float and double (and variations) in C++Single precision (IEEE Floating-Point Format )

1. Write binary representation in floating-point form:b1.b2b3 . . . × 2k with each bi a bit and b1 = 1 (unless number is 0)

↑mantissa exponent

or fractional part

Example: 22.625 = (see p.41) Floating point form:

10110.1012

1.01101012 × 24

+ 127

double: Exp: 11 bits, bias 1023 Mant: 52 bits

p. 756

Round-off Errors

base

Basic data types for C

C programming language:

char - smallest addressable unit of the machine that can contain basic character set. It is an integer type, actual type can be either signed or unsigned depending on the implementation.

signed char - same size as char, but guaranteed to be signed.

unsigned char - same size as char, but guaranteed to be unsigned.

short short int signed short signed short int - short signed integer type, at least 16 bits in size.

Basic data types for C unsigned unsigned int - same as int, but unsigned.

long long int signed long signed long int - long signed integer type, at least 32 bits in size.

unsigned long unsigned long int - same as long, but unsigned.

long long long long int signed long long signed long long int - long long signed integer type, at least 64 bits in size

Basic data types for C

unsigned long long int - same as long long, but unsigned.

float - single precision floating-point type.

double - double precision floating-point type, actual properties unspecified (except minimum limits), however on most systems this is the IEEE 754 double-precision binary floating-point format

long double - extended precision floating-point type, actual properties unspecified.

Boolean type

Structures:

•  struct birthday { char name[20]; int day; int month; int year; };

Basic data types for C

Array - array of N elements of type T

•  int cat[10]; // array of 10 elements, each of type int

•  int bob[]; // array of an unspecified number of 'int' elements.

•  int a[10][8]; // array of 10 elements, each of type 'array of 8 int elements'

•  float f[][32]; // array of unspecified number of 'array of 32 float elements’

Pointer - char *square; long *circle;

Unions - union types are special structures which allow access to the same memory using different type descriptions