algorithms - computer sciencerobins/cs6161/slides/algorithms - slide set 1... · algorithms cs6161....

Gabriel Robins

Department of

Computer Science

University of Virginiawww.cs.virginia.edu/robins

AlgorithmsCS6161

Problem: Can 5 test tubes be spun simultaneously in a

12-hole centrifuge in a balanced way?

• What approaches fail?

• What techniques work and why?

• Lessons and generalizations

Algorithms (CS6161) Textbook

Textbook:

Introduction to Algorithms

by Cormen et al (MIT)

Third Edition, 2009

Thomas Cormen Charles Leiserson

Ronald Rivest Clifford Stein

Algorithms (CS6161) Textbook

Supplemental reading:

How to Solve It, by George Polya (MIT)

Princeton University Press, 1945

• A classic on problem solving

Good Articles / videos:www.cs.virginia.edu/robins/CS_readings.html

George Polya (1887-1985)

Algorithms Syllabus

Fundamentals:

• History of algorithms

• Problem solving

• Pigeon-hole principle

• Occam's razor

• Uncomputability

• Universality

• Asymptotic complexity

• Set theory and logic

Algorithms Syllabus

Data structures:

• Arrays

• Stacks and queues

• Linked lists

• Binary and general trees

• Height-balanced trees

• Heaps

• Hash tables

Algorithms Syllabus

Sorting and searching:

• Classical sorting methods

• Specialized sorting techniques

• Finding max & min

• Median finding and Kth selection

• Majority detection

• Meta algorithms

Algorithms Syllabus

Computational geometry: • Convex hulls

• Lower bounds

• Line segment intersection

• Planar subdivision search

• Voronoi diagrams

• Nearest neighbors

• Geometric minimum spanning trees

• Delaunay triangulations

• Distance between convex polygons

• Triangulation of polygons

• Collinear subsets

Algorithms Syllabus

Graph algorithms:

• Depth-first search

• Breadth-first search

• Minimum spanning trees

• Shortest paths trees

• Radius-cost tradeoffs

• Steiner trees

• Degree-constrained trees

Algorithms Syllabus

NP-completeness:

• Resource-constrained computation

• Complexity classes

• Intractability

• Boolean satisfiability

• Cook-Levin theorem

• Transformations

• Graph clique problem

• Independent sets

• Hamiltonian cycles

• Colorability problems

• Heuristics

PNP

NP-complete SAT

co-NP-complete TAUT

co-NP P-complete LP

Algorithms Syllabus

Other topics in algorithms:

• Linear programming

• Matrix multiplication

• String matching

• Minimum matchings

• Network flows

• Distributed algorithms

• Amortized analysis

• Zero knowledge proofs≈

• Focus on the “big picture” & “scientific method”

• Emphasis on problem solving & creativity

• Discuss applications & practice

• A primary objective: have fun!

Overarching Philosophy

Algorithms Throughout History

A brief history of computing:

• Aristotle, Euclid, Archimedes, Eratosthenes

• Abu Ali al-Hasan ibn al-Haytham

• Fibonacci, Descartes, Fermat, Pascal

• Newton, Euler, Gauss, Hamilton

• Boole, De Morgan, Babbage, Ada Agusta

• Venn, Carroll, Cantor, Hilbert, Russell

• Hardy, Ramanujan, Ramsey

• Godel, Church, Turing, von Neumann

• Shannon, Kleene, Chomsky

An Ancient Computer: The Antikythera• Oldest known mechanical computer

• Built around 150-100 BCE !

• Calculates eclipses and astronomical

positions of sun, moon, and planets

• Very sophisticated for its era

• Contains dozens of intricate gears

• Comparable to 1700’s Swiss clocks

• Has an attached “instructions manual”

• Still the subject of ongoing research

• Some discrete math & algorithms knowledge

• Ideally, should have taken CS4102

• Course will “bootstrap”

(albeit quickly) from first

principles

• Critical: Tenacity, patience

Prerequisites

• Exams: probably take home

– Decide by vote

– Flexible exam schedule

• Problem sets:

– Lots of problem solving

– Work in groups!

– Not formally graded

– Many exam questions will

come from homeworks!

• Project and demo

• Extra credit problems

– In class & take-home

– Find mistakes in slides, handouts, etc.

• Course materials posted on Web site

www.cs.virginia.edu/robins/algorithms

Course Organization

• Attendance 10%

• Readings 20%

• Midterm 25%

• Final 25%

• Project 20%

• Extra credit 10%

Total: 110% +

Best strategy:

• Solve lots of problems!

• Do lots of readings / EC!

• “Ninety percent of success is just showing up.” – Woody Allen

Grading Scheme

• Cheating / plagiarism is strictly prohibited

• Serious penalties for violators

• Please review the UVa Honor Code

• Examples of Cheating / plagiarism:

– Mass-copying of solutions from others / Web

– Mass-sharing of solutions with others / Web

– Cutting-and-pasting from other people / Web

– Copying article/book/movie reviews from people / Web

– Other people / Web solving entire problems for you

– Providing other people / Web with verbatim solutions

– This list is not exhaustive!

• We have automated cheating / plagiarism detection tools!

• We encourage collaborations / brainstorming

• Lets keep it positive (and not play “gotcha”)

Cheating Policy

Professor Gabriel Robins

Office: 406 Rice Hall

Phone: (434) 982-2207

Email: [email protected]

Web: www.cs.virginia.edu/robins

www.cs.virginia.edu/robins/theory

Office hours: right after class• Any other time

• By email (preferred)

• By appointment

• Q&A blog posted on class Web site

Contact Information

Course Readingswww.cs.virginia.edu/robins/CS_readings.html

Goal: broad exposure to lots of cool ideas & technologies!

• Required: total of at least 36 items over the semester

• Diversity: minimums in each of 3 categories:

1. Minimum of 15 videos

2. Minimum of 15 papers / Web sites

3. Minimum of 6 books

• More than 36 total is even better! (extra credit)

• Some required items in each category

o Remaining “elective” items should be a diverse mix

• Email all submissions to: [email protected]

Required Readingswww.cs.virginia.edu/robins/CS_readings.html

• Required videos:

– Last Lecture, Randy Pausch, 2007

– Time Management, Randy Pausch, 2007

– Powers of Ten, Charles and Ray Eames, 1977

http://www.cs.virginia.edu/~robins/Randy/Randy_TM_jow0247.jpg

http://www.cs.virginia.edu/~robins/Randy/Randy_TM_jow0247.jpg

Required Reading• “Scale of the Universe”, Cary and Michael Huang, 2012

• 10-24 to 1026 meters 50 orders of magnitude!

http://htwins.net/scale2/


• More required videos:

– Claude Shannon - Father of the Information Age, UCTV

– The Pattern Behind Self-Deception, Michael Shermer, 2010

Claude Shannon

(1916–2001)

Michael Shermer


• Required articles:

– Decoding an Ancient Computer, Freeth, 2009

– Alan Turing’s Forgotten Ideas, Copeland and Proudfoot, 1999

– You and Your Research, Richard Hamming, 1986

– Who Can Name the Bigger Number, Scott Aaronson, 1999

Scott AaronsonRichard HammingAlan TuringAntikythera computer, 200BC

http://www.cs.virginia.edu/robins/cs6161/basics.pdf

http://www.cs.virginia.edu/robins/cs6161/discrete_math_review_slides.pdf

Discrete Math Review Slides


• Required books:

– “How to Solve It”, Polya, 1957

– “Infinity and the Mind”, Rucker, 1995

– “Godel, Escher, Bach”, Hofstadter, 1979

– “The Demon-Haunted World”, Sagan, 2009

– “What If”, Munroe, 2014


• Remaining videos / articles / books are “electives”

• Pacing: at least 2 submissions per week (due 5pm Monday)

- Policy intended to help you avoid “cramming”

• Length: 1-2 paragraphs per article / video

1-2 pages per book

• Books are worth more credit than articles / videos

• Email all submissions to: [email protected]

• Additional readings beyond 36 are welcome! (extra credit)

Other “Elective” Readingswww.cs.virginia.edu/robins/CS_readings.html

• Theory and Algorithms:

– Who Can Name the Bigger Number, Scott Aaronson, 1999

– The Limits of Reason, Gregory Chaitin, Scientific American, March

2006, pp. 74-81.

– Breaking Intractability, Joseph Traub and Henryk Wozniakowski,

Scientific American, January 1994, pp. 102-107.

– Confronting Science's Logical Limits, John Casti, Scientific

American, October 1996, pp. 102-105.

– Go Forth and Replicate, Moshe Sipper and James Reggia, Scientific

American, August 2001, pp. 34-43.

– The Science Behind Sudoku, Jean-Paul Delahaye, Scientific

American, June 2006, pp. 80-87.

– The Traveler's Dilemma, Kaushik Basu, Scientific American, June

2007, pp. 90-95.


• Biological Computing:

– Computing with DNA, Leonard Adleman, Scientific American,

August 1998, pp. 54-61.

– Bringing DNA Computing to Life, Ehud Shapiro and Yaakov

Benenson, Scientific American, May 2006, pp. 44-51.

– Engineering Life: Building a FAB for Biology, David Baker et

al., Scientific American, June 2006, pp. 44-51.

– Big Lab on a Tiny Chip, Charles Choi, Scientific American,

October 2007, pp. 100-103.

– DNA Computers for Work and Play, Macdonald et al, Scientific

American, November 2007, pp. 84-91.

Email all submissions to: [email protected]


• Quantum Computing:

– Quantum Mechanical Computers, Seth Lloyd, Scientific

American, 1997, pp. 98-104.

– Quantum Computing with Molecules, Gershenfeld and Chuang,

Scientific American, June 1998, pp. 66-71.

– Black Hole Computers, Seth Lloyd and Jack Ng, Scientific

American, November 2004, pp. 52-61.

– Computing with Quantum Knots, Graham Collins, Scientific

American, April 2006, pp. 56-63.

– The Limits of Quantum Computers, Scott Aaronson, Scientific

American, March 2008, pp. 62-69.

– Quantum Computing with Ions, Monroe and Wineland,

Scientific American, August 2008, pp. 64-71.


• History of Computing:

– The Origins of Computing, Campbell-Kelly, Scientific

American, September 2009, pp. 62-69.

– Ada and the First Computer, Eugene Kim and Betty Toole,

Scientific American, April 1999, pp. 76-81.

• Security and Privacy:

– Malware Goes Mobile, Mikko Hypponen, Scientific American,

November 2006, pp. 70-77.

– RFID Powder, Tim Hornyak, Scientific American, February

2008, pp. 68-71.

– Can Phishing be Foiled, Lorrie Cranor, Scientific American,

December 2008, pp. 104-110.


• Future of Computing: – Microprocessors in 2020, David Patterson, Scientific American, September

1995, pp. 62-67.

– Computing Without Clocks, Ivan Sutherland and Jo Ebergen, Scientific

American, August 2002, pp. 62-69.

– Making Silicon Lase, Bahram Jalali, Scientific American, February 2007,

pp. 58-65.

– A Robot in Every Home, Bill Gates, Scientific Am, January 2007, pp. 58-65.

– Ballbots, Ralph Hollis, Scientific American, October 2006, pp. 72-77.

– Dependable Software by Design, Daniel Jackson, Scientific American, June

2006, pp. 68-75.

– Not Tonight Dear - I Have to Reboot, Charles Choi, Scientific American,

March 2008, pp. 94-97.

– Self-Powered Nanotech, Zhong Lin Wang, Scientific American, January

2008, pp. 82-87.


• The Web: – The Semantic Web in Action, Lee Feigenbaum et al., Scientific American,

December 2007, pp. 90-97.

– Web Science Emerges, Nigel Shadbolt and Tim Berners-Lee, Scientific

American, October 2008, pp. 76-81.

• The Wikipedia Computer Science Portal: – Theory of computation and Automata theory

– Formal languages and grammars

– Chomsky hierarchy and the Complexity Zoo

– Regular, context-free &Turing-decidable languages

– Finite & pushdown automata; Turing machines

– Computational complexity

– List of data structures and algorithms


http://upload.wikimedia.org/wikipedia/commons/b/b7/Wikipedia-logo.svg

http://upload.wikimedia.org/wikipedia/commons/b/b7/Wikipedia-logo.svg


• The Wikipedia Math Portal:

– Problem solving

– List of Mathematical lists

– Sets and Infinity

– Discrete mathematics

– Proof techniques and list of proofs

– Information theory & randomness

– Game theory

• Mathematica's “Math World”


• Ask questions ASAP

• Solve problems ASAP

• Work in study groups

• Do not fall behind

• “Cramming” won’t work

• Do lots of extra credit

• Attend every lecture

• Visit class Website often

• Solve lots of problems

Good Advice

Goal: Become a more effective problem solver!


Problem: Can 5 test tubes be spun simultaneously in a

12-hole centrifuge in a balanced way?

• What does “balanced” mean?

• Why are 3 test tubes balanced?

• Symmetry!

• Can you merge solutions?

• Superposition!

• Linearity! ƒ(x + y) = ƒ(x) + ƒ(y)

• Can you spin 7 test tubes?

• Complementarity!

• Empirical testing…

Problem: 1 + 2 + 3 + 4 + …+ 100 = ?

Proof: Induction…

1 + 2 + 3 + … + 99 + 100

100 + 99 + 98 + … + 2 + 1

101 + 101 + 101 + … + 101 + 101 =

2

)1(

1

nni

n

i

n+1

n

100*101

= (100*101)/2

= 5050

• You must a priori know the formula / result

• Easy to make mistakes in inductive proof

• Mostly “mechanical” – ignores intuitions

• Tedious to construct

• Difficult to check

• Hard to understand

• Not very convincing

• Generalizations not obvious

• Does not “shed light on truth”

• Obfuscates connections

Conclusion: only use induction as a last resort! (i.e., rarely)

Drawbacks of Induction

Oh oh!

Problem: (1/4) + (1/4)2 + (1/4)3 + (1/4)4 + … = ?

?4

1

1

ii

Extra Credit:

Find a short, geometric, induction-free proof.

Problem: (1/4) + (1/4)2 + (1/4)3 + (1/4)4 + … = ?


3

1

4

1

1

ii1

1

Problem: (1/8) + (1/8)2 + (1/8)3 + (1/8)4 + …= ?

?8

1

1

ii

Extra Credit:


Problem: (1/8) + (1/8)2 + (1/8)3 + (1/8)4 + …= ?


7

1

8

1

1

ii

Problem: 13 + 23 + 33 + 43 + …+ n3 = ?

?i1

3

n

i

Extra Credit:

find a short, geometric,

induction-free proof.

Problem: Can an 8x8 board with two opposite

corners missing be tiles with 31 dominoes?

= 31 x ?




Problem: Given any five points in/on the unit

square, is there always a pair with distance ≤ ?

1

1

2

1





equilateral triangle, is there always a pair with

distance ≤ ½ ?

1 1

1







Problem: Prove that there are an infinity of primes.

Extra Credit: Find a short, induction-free proof.




Problem: True or false: there are arbitrary long

blocks of consecutive composite integers

(i.e., big “prime deserts”)

Extra Credit: find a short, induction-free proof.




Problem: Prove that is irrational.


2




Problem: Does exponentiation preserve irrationality?

i.e., are there two irrational numbers x and y such

that xy is rational?


X = 2X

XX

X

Problem: Solve the following equation for X:

where the stack of exponentiated x’s extends forever.




Problem: Are the complex numbers closed under

exponentiation ? E.g., what is the value of ii?

Theorem [Turing]: not all problems are solvable by algorithms.

Theorem: not all functions are computable by algorithms.

Theorem: not all Boolean functions are computable by algorithms.

Theorem: most Boolean functions are not computable!

Q: Can we find a concrete example of an uncomputable function?

A: [Turing] Yes, for example, the Halting Problem.

Definition: The Halting problem: given a program P

and input I, will P ever halt if we ran it on I?

Define H:ℕℕ{0,1}

H(P,I)=1 if program P halts on input I

H(P,I)=0 otherwise

• Both P and I can be encoded as strings

• P and I can also be encoded as integers (in some canonical order )

• H is an everywhere-defined Boolean function on natural #’s

PI

yes

no

Does P(I)

halt?

H

Number of steps to termination

for the first 10,000 numbers

Theorem [Turing]: the halting problem (H) is not computable.

Ex: the “3X+1” problem (the Ulam conjecture):

• Start with any integer X>0

• If X is even, then replace it with X/2

• If X is odd then replace it with 3X+1

• Repeat until X=1 (i.e., short cycle 4, 2, 1, ...)

Ex: 26 terminates after 10 steps

27 terminates after 111 steps

Termination verified for X<1018

Q: Does this terminate for every X>0 ?

A: Open since 1937!

“Mathematics is not yet ready for such confusing,

troubling, and hard problems." - Paul Erdős, who

offered a $500 bounty for a solution to this problem

Observation: termination is

in general difficult to detect!


Corollary: we can not algorithmically detect all infinite loops.

Q: Why not? E.g., do the following programs halt?

main()

{ int k=3; }

main()

{ while(1) {} }

Halts! Runs forever! ?

main()

{ Find a Fermat

triple an+bn=cn

with n>2 then stop}

Runs forever!

Open from 1637-1995!

main()

{ Find a Goldbach

integer that is not a sum

of two primes & stop}

?

Still open since 1742!

Theorem: solving the halting problem is at least as hard

as solving arbitrary open mathematical problems!

Corollary: Its not about size!


Proof: Assume $ algorithm S that solves the halting problem

H, that always stops with the correct answer for any P & I.

PI

yes

noDoes

P(I) halt?

S

X

T

T(T) haltsQ ~Q Contradiction!

PI

yes

noDoes

P(I) halt?

S

PI

yes

noDoes

P(I) halt?

S

S cannot exist! (at least as an algorithm / program / TM)

Using S, construct algorithm / TM T:

T(T) halts

T(T) does not halt

T(T) does not halt

Theorem: all computable numbers are finitely describable.Proof: A computable number can be outputted by a TM.

A TM is a (unique) finite description.

What the unsolvability of the Halting Problem means:

There is no single algorithm / program / TM that correctlysolves all instances of the halting problem in finite time each.

This result does not necessarily apply if we allow:

• Incorrectness on some instances

• Infinitely large algorithm / program

• Infinite number of finite algorithms / programs

• Some instances to not be solved

• Infinite “running time” / steps

• Powerful enough oracles

Q: When do we want to feed a program to itself in practice?

A: When we build compilers.

Q: Why?

A: To make them more efficient!

To boot-strap the coding in the compiler’s own language!

Program Ccompiler

Executablecode

Theorem: virus detection

is not computable.

Theorem: Infinite loop

detection is not computable.

Self-Replication• Biology / DNA

• Nanotechnology

• Computer viruses

• Space exploration

• Memetics / memes

• “Gray goo”

Problem (extra credit): write a program that

prints out its own source code (no inputs of

any kind are allowed).

Self-replicating

cellular automata

designed by von Neumann

Non-Existence Proofs

• Must cover all possible (usually infinite) scenarios!

• Examples / counter-examples are not convincing!

• Not “symmetric” to existence proofs!

Ex: proofs that you

are a millionaire:

“Proofs” that you

are not a millionaire ?

PNP

Naturals ℕ 6Integers ℤ -4

Rationals ℚ 2/9

Reals ℝ

Quat

ernio

ns

ℍ1

+i+

j+k Complex ℂ 7+3i

Sur

re

al

{L

|R}

Su

rco

mple

x A

+B

i

Primes ℙ 5

Oct

onio

ns

1+

i+j+

k+

E+

I+J+

K

Hypernumbers

Sed

enio

nsS

1+

i+j+

k+

…+

e 15+

e 16

?

Boolean 1

Com

puta

ble

nu

mb

ers

Finitely describable numbers H

Alg

ebra

ic

2

Tra

nce

nden

talp

Irra

tional

s J

Theorem: some real numbers are not finitely describable!Theorem: some finitely describable real numbers are not computable!

Generalized Numbers

Pigeon-Hole Principle

• J. Dirichlet (1834)

• “Drawer principle”

• “Shelf Principle”

• “Box principle”

Theorem (pigeon-hole): There is no injective (1-to-1) function

from a finite set (domain) to a smaller finite set (range).

Generalization:N objects placed in M containers; then:

• at least 1 container must hold

• at least 1 container must hold

M

N

M

N


square, is there always a pair with distance ≤ ?

1

1

2

1





equilateral triangle, is there always a pair with

distance ≤ ½ ?

1 1

1




Problem: Given any ten points in/on the unit

square, what is the maximum pairwise distance?

1

1




Problem: Solve the following equation for X:

where the stack of exponentiated x’s extends forever.

= 2

X = 2X

XX

X

This “power tower” converges for:

0.065988 ≈ e−e < X < e1/e ≈ 1.444668

Generalization to

complex numbers:

X2=2 X=2

algorithms - computer sciencerobins/cs6161/slides/algorithms - slide set 1... · algorithms cs6161....

Documents