5 maximizing submodular functions minimizing convex functions: polynomial time solvable! minimizing...
TRANSCRIPT
5
Maximizing submodular functions
Minimizing convex functions:
Polynomial time solvable!Minimizing submodular functions:
Polynomial time solvable!
Maximizing convex functions:
NP hard!Maximizing submodular functions:
NP hard!
But can get approximation guarantees
6
Example: Set cover
Node predictsvalues of positionswith some radius
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE
For A µ V: z(A) = “area covered by sensors placed at A”
Formally: W finite set, collection of n subsets Si
µ WFor A µ V={1,…,n} define z(A) = |i2 A Si|
Want to cover floorplan with discsPlace sensorsin building
Possiblelocations
V
7
Set cover is submodular
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE
SERVER
LAB
KITCHEN
COPYELEC
PHONEQUIET
STORAGE
CONFERENCE
OFFICEOFFICE
S1 S2
S1 S2
S3
S4 S’
S’
A={S1,S2}
B = {S1,S2,S3,S4}
z(A[{S’})-z(A)
z(B[{S’})-z(B)
¸
8
Example: Feature selection• Given random variables Y, X1, … Xn
• Want to predict Y from subset XA = (Xi1,…,Xik
)
Want k most informative features:
A* = argmax IG(XA; Y) s.t. |A| · k
where IG(XA; Y) = H(Y) - H(Y | XA)
Y“Sick”
X1
“Fever”X2
“Rash”X3
“Male”
Naïve BayesModel
Uncertaintybefore knowing XA
Uncertaintyafter knowing XA
9
Example: Submodularity of info-gain
Y1,…,Ym, X1, …, Xn discrete RVs
z(A) = IG(Y; XA) = H(Y)-H(Y | XA)
• z(A) is always monotonic• However, NOT always submodular
Theorem [Krause & Guestrin UAI’ 05]If Xi are all conditionally independent given Y,then z(A) is submodular!
Y1
X1
Y2
X2
Y3
X4X3
Hence, greedy algorithm works!
In fact, NO algorithm can do better than (1-1/e) approximation!
10
• People sit a lot• Activity recognition in
assistive technologies• Seating pressure as
user interface
Equipped with 1 sensor per cm2!
Costs $16,000!
Can we get similar accuracy with fewer,
cheaper sensors?
Leanforward
SlouchLeanleft
82% accuracy on 10 postures! [Tan et al]
Building a Sensing Chair [Mutlu, Krause, Forlizzi, Guestrin, Hodgins UIST ‘07]
11
How to place sensors on a chair?• Sensor readings at locations V as random variables• Predict posture Y using probabilistic model P(Y,V)• Pick sensor locations A* µ V to minimize entropy:
Possible locations V
Accuracy CostBefore 82% $16,000 After 79% $100
Placed sensors, did a user study:
Similar accuracy at <1% of the cost!
12
Bounds on optimal solution[Krause et al., J Wat Res Mgt ’08]
Submodularity gives data-dependent bounds on the performance of any algorithm
Sensi
ng q
ualit
y z
(A)
Hig
her
is b
ett
er
Water networks
data
0 5 10 15 200
0.2
0.4
0.6
0.8
1
1.2
1.4Offline
(Nemhauser)bound Data-dependent
bound
Greedysolution
Number of sensors placed
Summary (1)
• Minimization of submodular functions– Submodularity and convexity– Submodular Polyhedron– Symmetric submodular functions
Summary (2)
• Pseudo-boolean functions– Representation (polynomial, posiform, tableau, graph
cut)– Reduction to quadratic polynomial– Necessary and sufficient conditions for submodularity– Minimization of quadratic and cubic submodular
functions via graph cuts– Lower bound via roof duality
• LP via posiform representation• LP via linear relaxation• Max flow via symmetric graph construction
Further reading
• Combinatorial algorithms for submodular (and bisubmodular) function minimization
• More algorithms/bounds for maximizing submodular functions
• Linear and semidefinite relaxations
• Matroids, greedoids, intersection of matroids, polymatroids and more
• Generalized roof duality