linear programming and its usage in approximation algorithms for np hard optimization problems
DESCRIPTION
Presented at Computer Science Department, Sharif University of Technology (Optimization Seminar ).TRANSCRIPT
1
Linear Programming and its Usage in Linear Programming and its Usage in Approximation Algorithms for NP Hard Approximation Algorithms for NP Hard
Optimization ProblemsOptimization Problems
M. Reza Rahimi, M. Reza Rahimi, November 2005.November 2005.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
2
Outline
• Introduction
• Linear Programming Overview
• NP Complexity Class
• Approximation Algorithms
• Case Study1: Minimum Weight Vertex Cover
• Case Study2: MAXSAT Problem
• Conclusion
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
3
Introduction
• One of the most challenging problem in complexity theory is problem P vs. NPP vs. NP.
• Research on this open problem leads researchers to think of new methods and different approaches.
• For example PCP, IP, approximation algorithms,...
• For some problems it is proved that there does not exist any suitable approximation unless P=NP.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
4
• So it seems that research on approximation algorithms may lead us to good results about P vs. NP.
• General Method for Approximation Algorithms of NP Hard Optimization is Greedy Method.Greedy Method.
• But we must search for Suitable FrameworkSuitable Framework for studying NP problems.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
5
• Linear ProgrammingLinear Programming was this brake through method.
• In this talk I focus on LP and its usage in NP Hard Optimization Problems.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
6
Linear Programming Overview
• General formulation of linear programming is presented as follow:
Maximize C1X1 + C2X2 + … + CdXd
Subject to A1,1X1 + … + A1,dXd ≤ b1
A2,1X1 + … + A2,dXd ≤ b2
…
An,1X1 + … + An,dXd ≤ bn
All calculation and Numbers are on Real Numbers.All calculation and Numbers are on Real Numbers.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
7
• There are several Polynomial Algorithms for this problem such as EllipsoidEllipsoid and InteriorInterior pointpoint methodmethod.
• I neglect talking about them.
• For these algorithms please refer to optimization references.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
8
NP Complexity Class
• NP Complexity Class is only related to Decision Problem.
• For example:
• So for any optimization problem we consider its related decision problem.
• This will give us intuition about its hardness.
}.assignment satisfying a is :{ SAT
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
9
• We know the following definition about NP:
• We can look at this process like this:
rejects. y)V(x, and )xP(y y, L x2.
accepts. y)V(x, and )xP(y y, L x1.
, x P(.), P, V(.,.) NPL
x x
yFind Certificate Y V(X,Y)Y
X X
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
10
• Example:
• There is one another important concept in complexity which is Reduction.Reduction.
.assignment trueno is Then there 2)if
time.polynomialin it check could and ,
assignment Truegiven are We if 1)
:Because NP
}.assignment satisfying a is :{
0
0
SAT
SAT
SAT
SAT
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
11
.)( Function) Time l(Polynomia *
:
*LxfLxfLL
Definition
P
*L
L
L
L *L
f
f
Fig1: Graphical Diagram of Reduction Concept
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
12
)xx(x)xxx()xx(x
:examplefor form in this is 3(
.3SAT
: that show we
e}.satisfiabl and cnf-3 is |{-3
}.assignment esatisfiabl is |{
:
321321321
P
CNF
CNF
CNF
SAT
Example
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
13
)()( 2321 xxxx
-x1
->
x2 x3 x4
^
<->y1 y2
le. truth tabusing
form OR-3 into converted becan sparanthesieach Now
)())(())(( 21243211 yyyxxxxy
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
14
10 ,
1)1(1
)1(
:Proof
. 3
: thatshow willWe
. Obviously
}. ,
matrices,integer are and |,{
:
321321
321321
111
11
ii
P
mnnmn
n
mnmmnm
XX
XXXXXX
XXXXXX
GPROGRAMMININTEGERCNF
NPGPROGRAMMININTEGER
bXAX
bAbAGPROGRAMMININTEGER
Example
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
15
• We conclude this section by the following theorem:
Every language in NP is polynomial time reducible to SAT language.
• Language like SAT, INTEGER-PROGRAMMING, Hamiltonian Cycle, and 3-CNF are also the same and they belong to NP-Complete set.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
16
Approximation Algorithms
• The following describes the connection of NP optimization Complete problem and its related optimization problem.
minimun. iscost itswhich
Cyclen Hamiltonia thefind ,Ggraph weightednegativenon complete have we
:
:
b}. equal
or than less iscost itssuch that Cyclen Hamiltonia exists
thereandgraph weightednegativenon complete is G |,{
:
nn
nn
TSPOP
VersiononOptimizati
bGTSP
VersionDecision
nn
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
17
• Bound of Approximation:Bound of Approximation:• Suppose that problem P has an optimum solution with cost C-
op. Algorithm X solves problem P
with bound ρ(n) if it finds feasible solution with cost C such as:
max {c/c-op, c-op/c} ≤ ρ(n).
• Note that with this definition we always have ρ(n) ≥1.
• In the following I try to show that sometimes, finding good approximation for optimization problems is very hard.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
18
• Theorem: Theorem:
It does not exist any polynomial timepolynomial time approximation algorithm with Polynomial BoundPolynomial Bound for OPT-TSPOPT-TSP unless P=NP.P=NP.
• Proof:
we prove that if such an algorithm exists then we can Solve Hamiltonian Cycle in Polynomial Time.
For each graph G with n vertex convertIt into weighted complete graph G1 as the following procedure: 1) Assign 1 to each of its edge.2) Assign nρ(n) to the other edges.
G has Hamiltonian Cycle Iff
OPT-TSP(G1)=n
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
19
• I think that it’s now time to study Randomized Rounding technique for solving optimization problems.
• I explore it with two examples.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
20
Case Study1: Minimum Weight Vertex Cover
• Now we study randomized roundingrandomized rounding method in approximation algorithms with an example.
• Definition:• Vertex Cover:Vertex Cover: In undirected graph G=(V,E),G=(V,E), Set AA of
vertices is said vertex cover if
for every (u,v) (u,v) εε EE then u u εε A or v A or v εε AA . .
• Minimum Weight Vertex Cover:Minimum Weight Vertex Cover: In undirected graph G=(V,E)G=(V,E) which each vertex has w(v)w(v) as positive weight Set AA of vertices is said minimum weight vertex cover if for every (u,v) (u,v) εε EE then u u εε A or v A or v εε A and w(A) is the least.A and w(A) is the least.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
21
Fig2: Vertex Cover
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
22
• It is proved that decision version of this optimization problem is NP Hard.
• At the first step we model it as integer programming.
1)(0
1)()( Vv,Vvw(v)x(v)min
:Method gProgramminInteger
0. x(v)else
1.x(v)SetCover Vertex Weight Minimum vif
{0,1}V:x
vx
uxvxu
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
23
• As it was known it is believed that there is no polynomial time algorithm for integer programming.
• We must think about another method.
• We investigate its related linear programming.
1)(0
1)()( Vv,Vvw(v)x(v)min
:Method gProgramminLinear
[0,1]V:x
vx
uxvxu
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
24
• The Last method is called RelaxationRelaxation to linearlinear programmingprogramming.
• It’s now time to state the approximation
algorithm.
C.Return 6)
{v}CC then 5)
/*Method Rounding * /0.5(v)x if do 4)
Veach vfor 3)
g.programminlinear osolution t optimalan x Compute 2)
C 1)
w)Cover(G,Vertex Min Weightion Approximat
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
25
2)C(Z)C(5.0
).(5.05.0)(5.0)(
)()()()(Z
.cover vertex is C Obviously
).C(Z have weObviously
g.programminlinear theof valueoptimal The ::Z
g.programmininteger ofset optimal real The::C
algorithm. ofsoulution The ::C
:Proof
**
5.0)(,
5.0)(,
*
**
*
*
ww
Cwvwvw
vxvwvxvw
w
CvvxVv
Vv vxVv
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
26
• HastadHastad proved that there is no Polynomial time algorithm for vertex cover that achieves an approximation better that 1.161.16 unless P=NP (1997).
• Now we consider another example.
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
27
Case Study2: MAXSAT ProblemCase Study2: MAXSAT Problem
trueis Cj
343213
243212
143211
432143214321
trueis Cj
jj
j
j
wMax
w)xxx(xC
w)xxxx(C
w)xxx(xC
).xxx)(xxxxx)(xxx(x
:Example
w
:maximizes that variables toassignment Find
.0 weight Whas C Clauseeach with Formula CNF-KGiven
:MAXSAT
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
28
• We state the following randomized algorithm for this problem. (Johnson 1974).
bound.ion approximat )]2
1-[(1 has algorithm preceding The
:Claim
return 2)
0.5.y probabilit with 1or 0 to
assign randomly ,...x xlesfor variab 1)
1k
C
n1
J
istrue
jw
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
29
1k
kC
jjC
jj trueis C
j
kj
kj
Cjj
trueis Cj
jj
)]2
1-1[(
.)2
1-1(}E{Iw}Iw{}w{
).2
1() 0p(I
).2
1-1() 1p(I
Iww
.0not if esatisfiablC clause if 1I
: variblesrandom following theDefine
:Proof
jjj
jj
jCjwEE
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
30
• We can extend Johnson algorithm as follow:
same. just the is proof The
))1(1(}w{
:Claim
return 2)
p-1 and py probabilit with 0or 1 to
assign tly independen andrandomly ,...x xlesfor variab 1)
trueis Cj
C
ii
n1
j
J
iij xif
ixif
iC
j
istruej
ppwE
w
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
31
• Now we model it with IP and relax it.
z,y 1z0
1y0
z)y-(1y
zwMax
:MAXSATfor Model gProgramminInteger
jij
i
j x if
i xif
i
Cjjj
ii
jC
z,y 1z0
1y0
z)y-(1y
zwMax
:MAXSATfor Relaxation
jij
i
j x if
i xif
i
Cjjj
ii
jC
RelaxationRelaxation
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
32
ratio.ion approximat 1/e)-(1 has algorithm above The
:Claim
))1
1(1()(y)y-(1-1
have we
Literals,k with C clauseany for and LP to)z,(ysolution feasibleany for
:Lemma
return )3
.yp with algorithmJohnson extended use 2)
).z,(y vector find and LP theSolve 1)
Algorithm
1-
x ifi
xifi
j**
i*
i
**
ii
jk z
k
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
33
1**
***
)/11(
))1
1(1())1
1(1(
))1
1(1(})()1(1{
})()1(1{
:Proof
e
Zk
Zk
zwk
yyw
ppw
IPk
LPk
Cjj
k
x xii
Cj
x xii
Cj
ji ij
i ij
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
34
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
35
• Algorithm
1. Compute the value from Johnson algorithm (w1).
2. Compute the value from LP Based algorithm (w2).
3. Return MAX(w1,w2).
Linear Programming and its Usage in Approximation Algorithms for NP Hard Optimization Problems
36
Conclusion
• In this talk the randomized rounding method is explored with two example.
• This new method opens new insight into approximation algorithms and complexity theory.
The EndThe End