computability and complexity theory: an introductionmeena/papers/imi.pdf · 2006-07-19 ·...

Meena Mahajan

Computability and Complexity Theory: AnIntroduction

Meena Mahajan

[email protected]

http://www.imsc.res.in/˜meena

IMI-IISc, 20 July 2006 – p. 1

Meena Mahajan

Understanding Computation

Kinds of questions we seek answers to:

• Is a given graph planar?• Can a given integer matrix be made singular by

changing at most one entry?• Is a given number prime?• Does a player in a certain game have a winning

strategy?• Is a given formula a tautology?• Does a given Diophantine equation have a solution?


Meena Mahajan

Understanding Computation ...

Kinds of answers we seek:

• A list of answers to all instances?Too long a list; infinite.


Meena Mahajan




• A list of answers to interesting instances?Still too long.


Meena Mahajan




• A list of answers to interesting instances?Still too long.

• A finitely described procedure that, when applied toany instance, gives the correct answer in finite time.i.e. an effective procedure.


Meena Mahajan

Example: Testing planarity• Input: A Graph G


Meena Mahajan


• Strategy: Try out all possible combinatorialembeddings (cyclic ordering of edges incident at avertex)


Meena Mahajan



• For each combinatorial embedding, compute its genus.


Meena Mahajan



• For each combinatorial embedding, compute its genus.• If for some embedding, the genus is 0, stop and say

Yes.


Meena Mahajan



• For each combinatorial embedding, compute its genus.• If for some embedding, the genus is 0, stop and say

Yes.• If no such embedding is found, stop and say No.


Meena Mahajan

A Non-Example• Input: An n × n matrix A with integer entries.

Decide if A can be made singular by changing at mostone entry.


Meena Mahajan



• If A is already singular, stop and say Yes. Otherwise ...


Meena Mahajan



• If A is already singular, stop and say Yes. Otherwise ...• Strategy: The set Z of all integers is countably infinite.

For each x ∈ Z and each i, j ∈ [n], check if replacingA[i, j] by x gives a singular matrix. If Yes, halt andreport Yes, else try the next triple x′, i′, j′.


Meena Mahajan



• If A is already singular, stop and say Yes. Otherwise ...• Strategy: The set Z of all integers is countably infinite.

For each x ∈ Z and each i, j ∈ [n], check if replacingA[i, j] by x gives a singular matrix. If Yes, halt andreport Yes, else try the next triple x′, i′, j′.

• If A can be made singular, we will find a witness andreport Yes. Otherwise, the procedure will run forever.


Meena Mahajan

Another Non-Example• Input: A Diophantine equation E.

Decide if it has an integer solution.


Meena Mahajan


Decide if it has an integer solution.• Strategy: Let E have n variables. The set Z

n, of allpossible integer assignments to the variables, iscountably infinite. Try out each assignment.


Meena Mahajan


Decide if it has an integer solution.• Strategy: Let E have n variables. The set Z

n, of allpossible integer assignments to the variables, iscountably infinite. Try out each assignment.

• If there is a solution, we will eventually find it and stop.if there is none, the procedure will run forever.


Meena Mahajan

Decidability• Effective procedures (also known as recursive

procedures): finitely described procedures thateventually halt on every input.The standard formalism is via Turing machines, butthere are many other equivalent ones.


Meena Mahajan



• Problems that can be solved by effective proceduresare said to be decidable.


Meena Mahajan




• Undecidable problems have no effective procedures.


Meena Mahajan




• Undecidable problems have no effective procedures.• Some problems can be proved to be undecidable by a

diagonalization argument.


Meena Mahajan

A diagonalization example• Suppose there is an effective procedure P to decide

the Halting Problem:P

Q

w

Yes; Q halts on w

No; Q loops on w


Meena Mahajan


the Halting Problem:• Obtain P’ from P as follows:

Q

w

(Q halts on w)P

QP

P’

No(Q loops on w)

Yes


Meena Mahajan



Q

w

(Q halts on w)P

QP

P’

No(Q loops on w)

Yes

• P’(Q) loops iff P(Q,Q) says Yes iff Q halts on Q.

P’(Q) halts iff P(Q,Q) says No iff Q loops on Q.


Meena Mahajan



Q

w

(Q halts on w)P

QP

P’

No(Q loops on w)

Yes

• P’(Q) loops iff P(Q,Q) says Yes iff Q halts on Q.

P’(Q) halts iff P(Q,Q) says No iff Q loops on Q.• What does P’ do on input P’?


Meena Mahajan

Some decidable problems• Is a given graph planar?

We saw an effective procedure for this.• Can a given matrix over Z be made singular by

changing at most one entry?The procedure we saw was not effective, but there isanother procedure that is effective.

• Is a given number prime?• Given a finite set of rewrite rules α −→ β, strings w, x

and a natural number k, can x be obtained from wthrough at most k applications of rewrite rules?...


Meena Mahajan

Some undecidable problems• The Halting Problem: Does a given program never get

into an infinite loop?


Meena Mahajan


into an infinite loop?• Given a finite set of rewrite rules α −→ β and strings

w, x, can x be obtained from w through a sequence ofapplications of rewrite rules?


Meena Mahajan




• Does a given Diophantine equation have a solution?


Meena Mahajan




• Does a given Diophantine equation have a solution?• Given a matrix with entries from

{0, 1} ∪ {x1, . . . , xt}, is there an integer assignmentto the variables xi that makes the matrix singular?


Meena Mahajan

Use of Resources• How much space / time does a given procedure use?

— space / time complexity of the given procedure.• Is there an equivalent procedure that uses less? What

is the minimum needed?i.e. How much is necessary, how much is sufficient?— inherent space / time complexity of the problem.

• To show sufficiency, describe any procedure that usesthat much.

• To show necessity, a lower bound proof is needed.


Meena Mahajan

Planarity testing• The planarity testing procedure we saw earlier used

space proportional to |E|.It tried out all combinatorial embeddings one by one,so it needed enough space to write down theembedding currently being tested.


Meena Mahajan



• It also needed time as large as c|E|+|V |, since thereare those many embeddings.


Meena Mahajan



• It also needed time as large as c|E|+|V |, since thereare those many embeddings.

• But planarity testing is in fact possible in spaceproportional to log(|V | + |E|), and time proportionalto |V ||E|.


Meena Mahajan

Some complexity classes

Space bounds: logarithmic, polynomial, exponential ...

Time bounds: polynomial, exponential ...

Log // PSPACE // EXPSPACE

P // EXPTIME


Meena Mahajan




Log

BBB

BBBB

BPSPACE

&&NNNNNNNNNNNEXPSPACE

P

;;vvvvvvvvvvEXPTIME

77oooooooooooo


Meena Mahajan




Log // P // PSPACE // EXPTIME // EXPSPACE

Polynomial time P is considered to capture feasible ortractable computation .

Many important natural problems are in P:

• Decide whether a graph has a perfect matching.• Evaluate a circuit with AND, OR, NOT gates.• Decide whether there is a path from s to t in a given

directed graph.


Meena Mahajan

Short certificates andNP

• Many important natural classification problems are notknown to be in P: Are two given graphs isomorphic?Does a linear program have an integer solution?


Meena Mahajan



• However, some of these have a nice property: whenthe input belongs to the specified class, there is ashort, convincing proof of this.


Meena Mahajan




• e.g. To prove that G1 and G2 are isomorphic, just givethe mapping f : V (G1) −→ V (G2). Apolynomial-time verifier can check that f is indeed anisomorphism.


Meena Mahajan




• To prove that a linear program has an integer solution,just give such a solution. Verifying that it is indeed asolution is in P.


Meena Mahajan




• NP is exactly the class of such problems: properties Cfor which membership (statements of the form x ∈ C)has short, efficiently verifiable proofs.

• NP stands for Non-deterministic Polynomial time. Anon-deterministic procedure can guess the shortconvincing proof in polynomial time, and then verifythat it is indeed a proof.


Meena Mahajan

The classesNP andco-NP

• Clearly, P ⊆ NP. The million dollar question is whetherall problems in NP are in fact in P.


Meena Mahajan



• A related question: Are NP and co-NP the same? i.e. if aproperty C has short efficiently verifiable proofs, doesthe property C have such proofs too?

What would be a short verifiable proof that G1 and G2

are not isomorphic?


Meena Mahajan



• A related question: Are NP and co-NP the same? i.e. if aproperty C has short efficiently verifiable proofs, doesthe property C have such proofs too?

What would be a short verifiable proof that G1 and G2

are not isomorphic?• It is generally believed that P 6= NP ∩ co-NP 6= NP.


Meena Mahajan

Other nondeterministic classes• NLog: problems with nondeterministic logarithmic

space algorithms.


Meena Mahajan


space algorithms.• This is more restrictive then having a short proof

checkable in logspace. Even the process ofnondeterministically guessing a proof should not needmore than logspace.Can a graph isomorphism be guessed in logspace?


Meena Mahajan


space algorithms.• This is more restrictive then having a short proof

checkable in logspace. Even the process ofnondeterministically guessing a proof should not needmore than logspace.Can a graph isomorphism be guessed in logspace?

• NPSPACE: problems with nondeterministic polynomialspace algorithms. Proofs can be exponentially long,but need to be guessable and checkable in polynomialspace.


Meena Mahajan

Relating the classes

From the definitions,Log //

��

P //

��

PSPACE

��NLog NP ∩ co-NP // NP NPSPACE


Meena Mahajan

Relating the classes

From the definitions,Log //

��

P //

��

PSPACE

��NLog NP ∩ co-NP // NP NPSPACE

In fact, much more is known:Log

��

P

��

PSPACE

��NLog = co-NLog

66llllllllllllllll

/

33NP ∩ co-NP // NP

88rrrrrrrrrrr

= NPSPACE

OO


Meena Mahajan

The notion of reductions

How does one compare the computational difficulty ofproblems? We want to make statements like:

• Graph Isomorphism is no harder than IntegerProgramming, and may be easier.

• Integer Programming and Hamiltonian Cycle areequally hard.

Testing property C is no harder than testing property D ifany instance x can be efficiently (in P, or Log) transformedto an instance y such that x ∈ C ⇐⇒ y ∈ D Such atransformation is called a many-one reduction, ≤m.


Meena Mahajan

What reductions achieve• Reductions impose a partial order on the set of all

problems. Problems in an equivalent class ≡m reduceto each other.


Meena Mahajan



• Reasonable complexity classes do not slice through anequivalence class ≡m. They equal the union of somesuch classes.


Meena Mahajan




• In the partial order induced by ≤m, a complexity classC can have many upper bounds.


Meena Mahajan





• Problems in an ≡m class that is an upper bound for acomplexity class C are said to be hard for C.


Meena Mahajan





• Problems in an ≡m class that is an upper bound for acomplexity class C are said to be hard for C.

• If an upper bound is inside C, then its ≡m class is aleast upper bound. Such problems are said to becomplete for C.


Meena Mahajan

NP-completeness

A problem is NP-complete if• it is in NP, and• it is an upper bound for NP. i.e. every problem in NP

reduces to it.

The Cook-Levin theorem says that Boolean satisfiabilitySAT is NP-complete.

Subsequently, several other problems in NP have beenshown to be NP-complete by reducing SAT, or any otherNP-complete problem, to them. These problems are fromdiverse fields.


Meena Mahajan

Completeness and implications• If Π is complete for C, and an efficient algorithm for Π

is found, then composing it with the reduction gives anefficient algorithm for every problem in C.


Meena Mahajan



• Since SAT is NP-complete, SAT ∈ P ⇐⇒ P = NP.


Meena Mahajan



• Since SAT is NP-complete, SAT ∈ P ⇐⇒ P = NP.• Horn-SAT: the restriction of SAT to formulae in

conjunctive normal form (CNF) where each clause hasat most one negated literal. Horn-SAT is complete for P

under ≤logm

reductions. Thus,HornSAT ∈ NLog ⇐⇒ NLog = P.


Meena Mahajan



• Since SAT is NP-complete, SAT ∈ P ⇐⇒ P = NP.• Horn-SAT: the restriction of SAT to formulae in

conjunctive normal form (CNF) where each clause hasat most one negated literal. Horn-SAT is complete for P

under ≤logm

reductions. Thus,HornSAT ∈ NLog ⇐⇒ NLog = P.

• 2-SAT: The restriction of SAT to formulae in CNFwhere each clause has at most 2 literals. 2-SAT iscomplete for NLog under ≤log

mreductions. Thus,

2SAT ∈ Log ⇐⇒ Log = NLog.


Meena Mahajan

CombinatorialNP-complete problems• Boolean Formula Satisfiability: Is there a True/False

assignment to variables x1, . . . , xn that makes theBoolean formula F (x1, . . . , xn) true?

• Integer programming: Is there an integer solution tothe system of inequalities Ax ≤ b?

• Hamiltonian circuit problem: Does a directed graph Ghave a Hamiltonian cycle?

• Sorting by Reversals: Can a string x1, . . . , xn besorted in fewer than k moves, where a move consistsof picking a substring and reversing it?


Meena Mahajan

AlgebraicNP-complete problems• Radius of Non-Singularity

Instance: A rational square matrix A, a rational θ.Decide: Can A be made singular by changing each

entry by at most θ?• Satisfiability of Quadratic Polynomials:

Instance: t degree-2 polynomials P1, . . . , Pt on nelements from F2.

Decide: Do these polynomials have a common zero?


Meena Mahajan

NP-completeness in coding theory• Maximum Likelihood Decoding Given a binary linear

code (via its parity check matrix H), and a word y, findthe codeword c nearest to it.


Meena Mahajan



• For a codeword c, HcT = 0. So if y = c + e, thenHeT = HyT = s. For the codeword nearest y, theerror term e has smallest Hamming weight. So

Input: A binary m × n matrix H, a vector s ∈ Fm2 , and

an integer w > 0.Decide: Is there a vector x ∈ F

n2 of weight ≤ w, such

that HxT = s?


Meena Mahajan



• For a codeword c, HcT = 0. So if y = c + e, thenHeT = HyT = s. For the codeword nearest y, theerror term e has smallest Hamming weight. So

Input: A binary m × n matrix H, a vector s ∈ Fm2 , and

an integer w > 0.Decide: Is there a vector x ∈ F

n2 of weight ≤ w, such

that HxT = s?• The Minimum Distance problem: s = 0 and we require

x 6= 0. Also NP-hard.


Meena Mahajan

More NP-completeness in coding• Maximum Distance Separable Code Any linear code

encoding k bits into n bits can have distanced ≤ n − (k − 1). Does a given linear code achievethis maximum separation?


Meena Mahajan

More NP-completeness in coding•• Maximum Distance Separable Code Any linear code


• The following related decision version is NP-hard:


Meena Mahajan



• The following related decision version is NP-hard:Input: A prime p, positive integers m and w, and an

r × pm matrix H over Fpm .


Meena Mahajan



• The following related decision version is NP-hard:Input: A prime p, positive integers m and w, and an

r × pm matrix H over Fpm .Decide: Is there a non-zero vector x of weight ≤ w and

length pm over Fpm , such that HxT = 0?


Meena Mahajan

NLog-hardness

Some NLog-complete problems:•• Given a directed layered acyclic graph G and vertices

s, t, does G have a path from s to t?• 2SAT: Is a given 2CNF formula satisfiable?• Given a perfect matching M in a bipartite graph G,

does G have any other perfect matching?

Some NLog-hard problems in P:• Does a given graph have a perfect matching?• Is a given integer matrix singular?


Meena Mahajan

NLog-hardness of Singularity

We reduce Reachability to Non-Singularity.Input: a directed layered acyclic graph G, vertices s, t atfirst and last layer respectively.Output: a matrix A satisfying

DetA = 0 ⇐⇒ s 6;G t

Strategy:• Subdivide each edge of G.• Add self-loops at all vertices except s.• Add edge t → s.• Output adjacency matrix A of this graph H.


Meena Mahajan

The construction


Meena Mahajan

Why this works• s ; t paths in G ⇐⇒ Cycle covers in H


Meena Mahajan


• cycle covers in H ⇐⇒ Terms in Det(A)


Meena Mahajan



• Value of term = Weight of cycle cover = 1


Meena Mahajan




• Sign of term = (−1)#even cycles


Meena Mahajan




• Sign of term = (−1)#even cycles

• But in H, there are no even cycles.Hence Det(A) = #s ;G t


Meena Mahajan

Radius of Singularity

Instance: A rational square matrix A, a rational θ.

Decide: Can A be made singular by changing each entryby at most θ?


Meena Mahajan

Radius of Singularity inNP

Suppose there is a singular B with |Bij − Aij| ≤ θ foreach i, j. Can B be guessed in polynomial time?Depends on what precision B needs.


Meena Mahajan



Theorem: If there is such a singular matrix, then in factthere is a position k, l and singular B such that

• Akl − θ ≤ Bkl ≤ Akl + θ

• For ij 6= kl, Bij ∈ {Aij − θ, Aij + θ}


Meena Mahajan



Theorem: If there is such a singular matrix, then in factthere is a position k, l and singular B such that

• Akl − θ ≤ Bkl ≤ Akl + θ

• For ij 6= kl, Bij ∈ {Aij − θ, Aij + θ}

This implies that Bkl, and hence all of B, requiresprecision polynomial in A, θ.


Meena Mahajan

NP-hardness of Radius of Singularity

Reduction from MaxCUT

Instance: An undirected simple graph G = (V, E), aninteger k.

Decide: Does G have a cut of size at least k?i.e. Can V be partitioned into S, S so that|E ∩ (S × S)| ≥ k?


Meena Mahajan

The Construction

Define the matrix N = (2m + 1)I − A, where A is theadjacency matrix of G; i.e.

Nij =

2m + 1 if i = j

−1 if i 6= j and (i, j) ∈ E

0 otherwise


Meena Mahajan

The Construction


Nij =

2m + 1 if i = j

−1 if i 6= j and (i, j) ∈ E

0 otherwise

Claim: N is not singular.(Any strictly diagonally dominant matrix is non-singular.)


Meena Mahajan

The Construction


Nij =

2m + 1 if i = j

−1 if i 6= j and (i, j) ∈ E

0 otherwise

Claim: N is not singular.(Any strictly diagonally dominant matrix is non-singular.)

Theorem: G has a cut of size k iff M = N−1 can be madesingular by changing each entry by at most

1(2m+1)n+4k−2m

.


Meena Mahajan

Why it works

Fix any y, z ∈ {−1, +1}n.λ: a non-zero eigen-value of NyzT ; NyzTx = λx.

• Claim 1: λ = zTNy.


Meena Mahajan

Why it works



• Claim 2: Changes of 1/λ suffice to make N−1 singular.


Meena Mahajan

Why it works




• Claim 3: For S = {i | yi = +1}, let cut size be δ(S).Then yTNy = 4δ(S) + (2m + 1)n − 2m.


Meena Mahajan

Why it works




• Claim 3: For S = {i | yi = +1}, let cut size be δ(S).Then yTNy = 4δ(S) + (2m + 1)n − 2m.

• G has a cut (S, S) of size k =⇒ for the corresponding−1, +1 vector y: yTNy ≥ 4k + (2m + 1)n − 2m.Changes of yTNy suffice to singularize N−1.


Meena Mahajan

Why it works ... (cont’d)

Suppose a singular A satisfies|Aij − Mij| ≤ α = 1

4k+(2m+1)n−2m

• There is a t ∈ [−1, +1]n, z ∈ {−1, +1}n:M − αtzT is singular.


Meena Mahajan



4k+(2m+1)n−2m


• There is a y ∈ {−1, +1}n, 0 < β ≤ 1 such thatM − αβyzT is singular.


Meena Mahajan



4k+(2m+1)n−2m



• maxy,z∈{−1,+1}n zTNy is achieved at z = y.


Meena Mahajan



4k+(2m+1)n−2m



• maxy,z∈{−1,+1}n zTNy is achieved at z = y.

• 1αβ

NM − NyzT is singular; so zTNy = 1αβ

≥ 1α

, so

∃u ∈ {−1, +1}n : uTNu ≥ 1α

, and thecorresponding cut has size at least k.


Meena Mahajan

Radius for smaller rankInstance: A rational square matrix A, a rational θ, an

integer r.

Decide: Can the rank of A be brought down to below r bychanging each entry by at most θ?

Complexity is still unknown. In fact, it is not even known tobe decidable!


Meena Mahajan

Thank You


computability and complexity theory: an introductionmeena/papers/imi.pdf · 2006-07-19 ·...

Documents