1 主讲教师:张德富 tel: 18959217108 email: [email protected]@xmu.edu.cn...
TRANSCRIPT
1
主讲教师:张德富 Tel: 18959217108 Email: [email protected]
课件下载地址: http://algorithm.xmu.edu.cn:10000/Download.aspx#p3
2
教材及参考书 教材 张德富 . 算法设计与分析(高级教程) , 国防工业出版社 ,
2007 参考书
T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein, Introduction to Algorithms (the second edition) , The MIT Press , 2001 ,中文名《算法导论(第二版)》(影印版),高等教育出版社, 2003
Brassard, G., Bratley, P., Fundamentals of algorithmics, Prentice hall, 2003
D. S. Hochbaum, Approximation Algorithms for NP-hard Problems, PWS Pulishing Company, 1995
Rajeev Motwani and Prabhakar Raghavan, Randomized Algorithms, Cambridge University Press, 1995
5
Summary
Introduction Mathematics Fundaments Probability Fundaments Optimization problem Algorithm Linear Programming vs Integer programming Computational complexity Dealing with NP problems Performance Measurements
6
1.1 Mathematics Fundaments
Floors and ceiling
Modular arithmetic
Exponential and Logarithm functions
Harmonic number
.11 xxxxx
./ mod nnaana
.1||for 11 2 xxxex x
. allfor )1(lim xen
x xn
n
.1for )1ln(1
xxxx
x
.1ln1
)()1ln(1
nk
nHnn
k
7
Probability fundaments
Discrete random variable
A (discrete) random variable X is a function from a finite or countably infinite sample space S to the real numbers.
For a random variable X and a real number x, we define the event X = x to be {s∈S : X(s) =x}; thus,
Especially, suppose we are given a sample space S and an event s. Then the random variable I{s} associated with event s is defined as
})(:{
}Pr{}Pr{xsXSs
sxX
otherwise0
occurs if1}{
ssI
8
Conditional probability formalizes the notion of having prior partial knowledge of the outcome of an experiment. The conditional probability of an event s1 given that another event s2 occurs is defined to be
Pr(s1|s2)=Pr(s1∩s2)/Pr(s2) Expected value of a random variable
The simplest and most useful summary of the distribution of a random variable is the "average" of the values it takes on. The expected value of a discrete random variable X is
x
xXxXE }Pr{][
9
Variance and standard deviation The notion of variance mathematically expresses how
far from the mean a random variable‘s values are likely to be. The variance of a random variable X with mean E[X] is
Var[X]=E[(X-E[X])2] The standard deviation δ of a random variable X is
the positive square root of the variance of X. Namely,δ2=Var[X].
10
Normal distribution Binomial distribution Central Limit Theorem
It turns out that for large samples the sampling distribution of the mean is approximately Normal whatever the distribution of the observations. This result is called the Central Limit Theorem. It allows us to use the Normal distribution as a good approximation for large sample size.
Confidence Intervals
11
For example
Flipping a coin.
. if0
, if1{}{
TY
HYHYIX H
The expected number of heads obtained in one flip of the coin is simply the expected value of our random variable XH:
E[XH] = E[I{Y = H}]
= 1 · Pr{Y = H} + 0 · Pr{Y = T}
= 1 · (1/2) + 0 · (1/2)= 1/2.
12
Lemma
Given a sample space S and an event A in the sample space S, let XA = I{A}. Then E[XA] =Pr{A}.
Proof :By the definition of a random variable and the definition of expected value, we have
E[XA] = E[I{A}]
= 1 · Pr{A} + 0 · Pr{Ā}
= Pr{A},
13
Let Xi be the indicator random variable associated with the event in which the ith flip comes up heads. Letting Yi be the random variable denoting the outcome of the ith flip,we have that Xi = I{Yi = H}. Let X be the random variable denoting the total number of heads in the n coin flips, so that
n
iiXX
1
14
We wish to compute the expected number of heads, so we take the expectation of both sides of the above equation to obtain
n
i
n
ii
n
ii
n
XE
XEXE
1
1
1
.2/2/1
][
][][
15
1.2 Computational complexity
Optimization problem
Definition 1 An optimization problem Q is characterized by three components
Instances D: a set of input instances
Solutions S(I): the set of all feasible solutions for an instance I∈D
Value f: a function which assigns a value to each solution ie f: S(I) →R
16
For example
The maximization problem Q:
namely, any given I∈D, find a solution s0∈S(I) such that
Similarly, one can define the minimization problem. For convenience, we will also refer to the value f(s0)
(maximization or minimum value) of the optimal solution as OPT(I), namely OPT(I)= f(s0).
)()( ),( 0 sfsfISs
)(s )(max ISsf
17
Algorithm
An algorithm is a sequence of computational steps that transform the input into the output to solve a given problem.
Algorithm—problem (optimization problem).
A mapping/relation between a set of input instances (domain) and an output set (range)
18
Uncomputable problems vs Intractable problems Genuinely uncomputable problems that just can’t be
solved Intractable or impractical problems that we don’t
have enough computational power to solve in general.
Intractable problems: Easy and hard problems Reductions Decision problem vs. optimization problem
19
P,NP NP-hard, NP-complete Some NP-complete problems
SAT, TSP, 0/1 knapsack problem, n queens problem, packing and covering problem, scheduling problem, timetabling problem, network routing problem et al.
20
Dealing with NP problems
In practice, we quite often encounter instances of intractable problems that we actually need solutions for: The problem might be generally intractable, but we
have a specific instance. For example, scheduling and networking problems
often fit into this class. Even when finding optimal solutions is intractable,
finding near-optimal solutions might be possible.
21
Dealing with NP problems
Your boss asks you to implement an algorithm for an NP-hard optimization problem. What do you do?
I couldn’t find a polynomial-time algorithm; I guess I’m too dumb.
22
Dealing with NP problems
I couldn’t find a polynomial-time algorithm, but neither could all these other smart people.
23
Dealing with NP problems
I couldn’t find a polynomial-time algorithm, because no such algorithm exists!
24
Coping with NP problems
Your boss asks you to implement an algorithm for an NP-hard optimization problem. What do you do?
-Exact algorithms / Global optimization Dynamic Programming, Backtracking, Branch-and-
bound and variants (either hope that the input is very small or that worst
case manifests itself very rarely) (Special cases maybe solvable in polynomial time) -Linear programming algorithms
25
-Heuristic algorithms / Local optimization
Greedy, Local Search, Meta-heuristics (Tabu search, GA, SA, Ant Algorithms, NN) etc
Approximation algorithms
Polynomial-time algorithms with provable performance bounds
Randomized algorithms
DNA and quantum computation
26
Other problems except NP problems
The world behaves randomly and fuzzily, so randomization and approximation are themes that cut across many areas of science and engineering.
Other problems except NP problems can be solved by making use of randomized and approximation algorithms, for example
Deadlock and symmetry problems Computational number theory( cryptography) Random sampling pattern matching ……
27
Performance Measurements
Heuristic algorithm
What do we expect from a “good” heuristic:
1. It should do so quickly
2. It should find a solution of good quality Tools to measure heuristic performances:
1. Run time analysis
2. Analysis on solution quality (experiment)
28
Performance Measurements
Approximation algorithms
What do we expect from a “good” approximation:
1. It runs quickly
2. It should find a solution of good quality Tools to measure approximation performances:
1. Run time analysis
2. Solution quality with performance guarantee( approximation ratio)
29
Performance Measurements
Randomized algorithms
What do we expect from a “good” randomized algorithm:
1. It runs quickly
2. It should make a mistake as small as possible Tools to measure randomized performances:
1. Run time analysis
2. Analysis on solution quality ( The error probability)
30
1.3 Linear programming problem(LP)
Specify the objective of the optimization problem as a linear function of
certain variables, and specify the constraints on resources as equalities or inequalities on those variables.
31
General linear programs
Given a set of real numbers a1, a2, ..., an and a set of variables x1, x2,..., xn, a linear function f on those variables is defined by
linear equality: linear inequalities: and
bxxf n ),,( 1
bxxf n ),,( 1 bxxf n ),,( 1
n
jjjnnn xaxaxaxxf
1111 ),,(
32
Linear constraints equalities or linear equalities. Formally a linear-programming problem is the
problem of either minimizing or maximizing a linear function subject to a finite set of linear constraints.
Minimization linear program Maximization linear program
34
Standard forms
Given n real numbers c1, c2, ..., cn; m real numbers b1, b2, ..., bm; and mn real numbers aij for i = 1, 2, ..., m and j = 1, 2, ..., n. We wish to find n real numbers x1, x2,..., xn that
Subject to
j
n
jj xc
1
max
.,,1for 0
,,1for1
njx
mibxa
j
jj
n
jij
35
Feasible solution Infeasible solution Objective value Optimal solution Optimal objective value Unbounded
38
Example
0,,,,,
224
7
7
332
654321
3216
3215
3214
321
xxxxxx
xxxx
xxxx
xxxx
xxxz
0,,
422
7
7
s.t.
332max
321
321
321
321
321
xxx
xxx
xxx
xxx
xxx
Standard forms Slack forms
39
The simplex algorithm
The simplex algorithm takes as input a linear program and returns an optimal solution. It starts at some vertex of the simplex and performs a sequence of iterations. In each iteration, it moves along an edge of the simplex from a current vertex to a neighboring vertex whose objective value is no smaller than that of the current vertex (and usually is larger.)
40
The simplex algorithm terminates when it reaches a local maximum, which is a vertex from which all neighboring vertices have a smaller objective value. Because the feasible region is convex and the objective function is linear, this local optimum is actually a global optimum.
41
Integer Programming (IP)
Integer Programming is simply Linear Programming with an added condition:
All variables must be integers Many problems can be stated as Integer Programs. For example, the Vertex Cover problem can be stated
as an integer programming problem.
Given an undirected graph G = (V, E), find a minimum subset V' of V such that if (u, v)∈E,
then u∈V' or v ∈V' (or both).
42
Suppose that we associate a variable x(v) with each vertex v∈V , and let us require that x(v) {0, 1} for ∈each v∈V. 0-1 integer program for finding a minimum vertex cover(VC):
s.t.
for any edge (u, v) E∈ for each v V∈
1)()( vxux
'
)( minVv
vx
}1,0{)( vx
43
0-1 linear programming relaxation :
s.t.
for any edge (u, v) E∈ for each v V∈ for each v V∈
1)()( vxux1)( vx0)( vx
'
)( minVv
vx