劣モジュラ最適化submodular function minimization γ+ o n n 7 8 ( ) 5 γ o n m ( log ) 7 γ o...
Post on 15-Jul-2020
2 Views
Preview:
TRANSCRIPT
岩田 覚
(京都大学数理解析研究所)
劣モジュラ最適化
Outline• Submodular Functions
ExamplesDiscrete Convexity
• Minimizing Submodular Functions • Symmetric Submodular Functions• Maximizing Submodular Functions• Approximating Submodular Functions
Submodular Functions
• Cut Capacity Functions • Matroid Rank Functions • Entropy Functions
)()()()( YXfYXfYfXf ∪+∩≥+
VYX ⊆∀ ,
VX Y
R2: →Vf:V Finite Set
Cut Capacity Function
Cut Capacity ∑= } leaving :|)({)( XaacXκ
X
s t
Max Flow Value＝Min Cut Capacity
0)( ≥ac
Matroid Rank Functions
:)()(
||)(,
ρρρ
ρYXYX
XXVX≤⇒⊆≤⊆∀
],[rank)( XUAX =ρMatrix Rank Function
Submodular
=A
X
V
U
Whitney (1935)
Entropy Functions
:)(Xh
0)( =φh
X
Information Sources
0≥
)()()()( YXhYXhYhXh ∪+∩≥+
Entropy of the Joint Distribution
Conditional Mutual Information
Positive Definite Symmetric Matrices
0)( =φf
=A ][XA][detlog)( XAXf =
X
)()()()( YXfYXfYfXf ∪+∩≥+
Ky Fan’s Inequality
Extension of the Hadamard Inequality∏∈
≤Vi
iiAAdet
Discrete Concavity
})({}){()(}){( ufuTfSfuSfTS
−∪≥−∪⇒⊆
V
T
Diminishing Returns
S u
Discrete Convexity
)()()()( YXfYXfYfXf ∪+∩≥+
V
X Y
Convex Function
x
y
Discrete Convexity
},{ ba
},,{ cbaV =
}{b
}{a}{c
},{ cb
φ
Lovász (1983)
f̂
f
f̂
： Linear Interpolation
： Convex
： Submodular
Murota (2003)
Discrete Convex Analysis
Submodular Function Minimization
X
)(Xf
?}|)(min{ VYYf ⊆
MinimizationAlgorithm
Evaluation Oracle
Minimizer
0)( =φfAssumption:
Ellipsoid MethodGrötschel, Lovász, Schrijver (1981)
Base Polyhedra
Submodular Polyhedron
)}()(,,|{)( YfYxVYxxfP V ≤⊆∀∈= R
∑∈
=Yv
vxYx )()(
}|{ RR →= VxV
)}()(),(|{)( VfVxfPxxfB =∈=Base Polyhedron
Greedy Algorithm
v
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
))((
))(())((
)(
)()(
1110
11001
2
1
2
1
nn vLf
vLfvLf
vy
vyvy
MM
L
OMM
MO
L
}){)(())(()( vvLfvLfvy −−=
Edmonds (1970)Shapley (1971)
Extreme Base
)(vL
)( Vv∈
:y
2v1v nv
Theorem
Min-Max Theorem
)}(|)(max{ )(min fBxVxYfVY
∈= −
⊆
)}(,0min{:)( vxvx =−
)()()( YfYxVx ≤≤−
Edmonds (1970)
Combinatorial Approach
Extreme Base
Convex Combination
Cunningham (1985)
LL
L yx ∑Λ∈
= λ
)( fByL ∈
x
|)(|max XfMVX⊆
=
)log( 6 nMMnO γ
Submodular Function Minimization
)( 87 nnO +γ)log( 5 MnO γ
)log( 7 nnO γ
Grötschel, Lovász, Schrijver (1981, 1988)
Iwata, Fleischer, Fujishige (2000) Schrijver (2000)
Iwata (2003)
Fleischer, Iwata (2000)
)log)(( 54 MnnO +γOrlin (2007)
)( 65 nnO +γ
Iwata (2002)
Fully Combinatorial
Ellipsoid Method
Cunningham (1985)
Iwata, Orlin (2009)
)( subject to Minimize 2
fBxx∈
The Minimum-Norm Base
sol. opt. :∗x}0)(|{: <= ∗
− vxvX}0)(|{:0 ≤= ∗ vxvX
Minimal MinimizerMaximal Minimizer
∗x
Theorem Fujishige (1984)
Wolfe’s Algorithm Practically Efficient
Evacuation Problem （Dynamic Flow）
:)(Xo
:)(aτ
Hoppe, Tardos (2000)
:)(vb
:)(ac
SX ∩ XT \
TSXXoXb ∪⊆∀≤ ),()(
TSCapacity
Transit Time
Supply/Demand
Maximum Amount of Flow from to .
Feasible
Multiterminal Source Coding
X XR
Y
Encoder
Encoder YRDecoder
)(YH
)(XH
)|( XYH
)|( YXH
),( YXH
),( YXHXR
YR
Slepian, Wolf (1973)
Multiclass Queueing Systems
iμiλ
Server
Control Policy
Arrival Rate Service Rate
Multiclass M/M/1Preemptive
Arrival Interval Service Time
Performance Region
VXS
Xii
Xiii
Xiii ⊆∀
−≥
∑∑
∑∈
∈
∈
,1 ρ
μρρ
:s
,: iii μλρ =
YRjs
is
:js Expected Staying Time of a Job in j
Achievable
Coffman, Mitrani (1980)
:s1<∑
∈Viiρ
A Class of Submodular FunctionsVzyx +∈R,,
))(()()()( XxhXyXzXf −=
:h
VXS
Xii
Xiii
Xiii ⊆∀
−≥
∑∑
∑∈
∈
∈
,1 ρ
μρρ
Nonnegative, Nondecreasing, Convex
)( VX ⊆
i
iiy
μρ
=:
xxh
−=
11:)(
iii Sz ρ=:
iix ρ=:
Submodular
Itoko & Iwata (2005)
Zonotope in 3D))(),(),(()( XzXyXxXw =
}|)({conv VXXwZ ⊆=
)(),,(~ xyhzzyxf −=
Zonotope
} ofPoint ExtremeLower :),,(|),,(~min{
}|)(min{
Zzyxzyxf
VXXf
=
⊆
Remark: ),,(~ zyxf is NOT concave!
z
Line Arrangementβ
iii zyx =+ βα
α
Topological Sweeping MethodEdelsbrunner, Guibas (1989) )( 2nO
Enumerating All the Cells
Symmetric Submodular Functions
. ),\()( VXXVfXf ⊆∀=
?},|)(min{ VXVXXf ≠⊂≠φ
)()()()( ,
YXfYXfYfXfVYXYX
∩+∪≥+⇒≠∪≠∩ φ
R→Vf 2:
Symmetric
Crossing Submodular
Symmetric Submodular Function Minimization
Maximum Adjacency Ordering
• Minimum Cut Algorithm by MA-orderingNagamochi & Ibaraki (1992)
• Simpler ProofsFrank (1994), Stoer & Wagner (1997)
• Symmetric Submodular Functions Queyranne (1998)
• Alternative ProofsFujishige (1998), Rizzi (2000)
Minimum Degree OrderingNagamochi (2007)
ISAAC’07, Sendai, Japan
Finding the family of all extreme sets for symmetric crossing submodular functionsin time. )( 3γnO
Symmetric Submodular Function Minimization
Extreme Sets
.: ),()( XZXZXfZf ≠≠⊂∀> φ
)\()\()()( XYfYXfYfXf +≥+
:f Symmetric Crossing Submodular Function
Extreme Set:X
The family of all extreme sets forms a laminar.
X YX Y
Flat Pair for Symmetric Submodular Functions
,}|)(min{)( XxxfXf ∈≥
.1|},{| s.t. =∩⊆∀ vuXVX
)( },{ vuVvu ≠⊆Flat Pair
uv
X
},{ vu
No Extreme SetsSeparate andu .v
Shrink into a single vertex.
},{ vu
MD-Ordering for Symmetric Submodular Functions
X
)\( )()(:)( iii VVXXVfXfXf ⊆∪+=
iV
Symmetric, Crossing Submodular
MD-orderingVvvvv nn ∈− ,,...,, 121
jvEach has minimum value of among )(1 vf j− .\ 1−∈ jVVv
},...,{: 1 ii vvV =
MD-Ordering for Symmetric Submodular Functions
nn vv ,1−
1−nv2−= ni
The last two vertices of an MD-ordering form a flat pair.
Proof by Induction:
:},{ 1 nn vv − if
.0,1,...,2−= ni
Flat Pair for on
nv
iVV \
Time Complexity
• Finding an MD-ordering in time.
• Finding all the extreme sets in time.
• Minimizing symmetric submodular functions in time.
)( 3γnO
)( 2γnO
)( 3γnO
Application to Clustering
A
V
AV \nXX ,...,1
),()()( )|()();(
BABA
ABBBA
XXHXHXHXXHXHXXI−+=
−=
Random variables
VPartition into andas independent as possible
);( \ AVA XXIMinimize subject to VA ≠≠φ
AV \A
Application to Clustering
)( jiVV ji ≠=∩ φ
∑=
k
iiVf
1)(Minimize
subject tokVVV ∪∪= L1
Greedy Split
)12( k− -Approximation
Submodurlar Function Maximization
Approximation AlgorithmsNemhauser, Wolsey, Fisher (1978)
Monotone SF / Cardinality Constraint(1-1/e)-Approximation
Feige, Mirrokni, Vondrák (FOCS 2007)Nonnegative SF2/5-Approximation
Vondrák (STOC 2008) Monotone SF / Matroid Constraint(1-1/e)-Approximation
Approximate Maximization
)()(:
}{:
}){(maxarg::
1
1
1
0
−
−
−
−=
∪=
∪==
jjj
jjj
jj
TfTf
vTT
vTfvT
ρ
φ
1−jv1vGreedy Algorithm
)(SfMaximize subject to kS ≤||
kj ,...,1=
1−jT
( ) kSSSfTf k ≤∀−≥ |:| ),(e11)(
Approximate Maximization
),...,1( kj =
)(: *Sf=η:*S
∑−
=
+≤1
1
j
iijk ρρη
[ ]
jj
TSujjj
j
kTf
TfuTfTf
TSfSf
j
ρ+≤
−∪+≤
∪≤
−
∈−−−
−
∑−
)(
)(}){()(
)()( )
1
\111
1**
1*
Q
Optimal Solution
∑−
=
1
1
j
iiρ
∑=
=k
iikTf
1)( ρ
Approximate Maximization
),...,1( 0 kjj =≥ρ
111:ˆ −
⎟⎠⎞
⎜⎝⎛ −=
j
j kkηρ
),...,1( kj =
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
≥
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
η
ηη
ρ
ρρ
MM
L
OOM
MO
L
kk
kk
2
1
110
100
Minimize ∑=
k
ii
1ρ
subject to
⎟⎠⎞
⎜⎝⎛ −≥
⎥⎥⎦
⎤
⎢⎢⎣
⎡⎟⎠⎞
⎜⎝⎛ −−=∑
= e11111ˆ
1ηηρ
kk
ii k
)(: *Sf=η
∑=
=k
iki Tf
1)(ρ
Feature Selection
nXXY ,...,, 1
AX
)|()(:);( AA XYHYHYXI −=
Random Variables
1XYPredict from subset
2X 3X
Y“Sick”
“Fever” “Rash” “Cough”);( YXI AMaximize
subject to kA ≤|| :X Conditionally Independent
)|()( YXHXH AA −= Submodular
Krause & Guestrin (2005)
Submodular Welfare Problem
(Monotone, Submodular)kff ,...,1Utility Functions
)( jiVV ji ≠=∩ φ
∑=
k
iiVf
1)(Maximize
subject tokVVV ∪∪= L1
e)11( − -ApproximationVondrák (2008)
Approximating Submodular Functions
X
)(XfAlgorithm Evaluation
Oracle
Function f̂ Y)(ˆ Yf
Construct a set function such that
For what function is this possible?
Approximating Submodular Functions
. ),(ˆ)()()(ˆ VXXfnXfXf ⊆∀≤≤ α
α
,0)( =φfAssumption . 0,)( VXXf ⊆∀≥Problem
f̂
1)( =nαnn =)(α
Remarksfor cut capacity functions
for general monotonesubmodular functions
Approximating Submodular Functions
• Algorithm with for matroid rank functions.
• Algorithm with for monotone submodular functions.
• No polynomial algorithm can achieve a factor better than even for matroid rank functions.
)log()( nnOn =α
)log()( nnn Ω=α
1)( += nnα
Goemans, Harvey, Iwata, Mirrokni (2009)
Submodular Load Balancing
?)(maxmin},...,{ 1
jjjVVVf
m
∑∈
=Xv
jvj pXf :)(
:,...,1 mffSvitkina & Fleischer (2008)
Monotone Submodular Functions
Scheduling
2-Approximation AlgorithmLenstra, Shmoys, Tardos (1990)
)log( nnO -Approximation Algorithm
Pointers
• S. Fujishige: Submodular Functions and Optimization, Elsevier, 2005.
• ICML 2008 Tutorial Session http://www.submodularity.org
• 永野清仁，河原吉伸，岡本吉央：離散凸最適化手法による機械学習の諸問題へのアプローチ，回路とシステム軽井沢ワークショップ，2009.
top related