revising the handling of nonlinear constraints in scip ...stefan/2018_ismp.pdf · revising the...
Post on 21-Mar-2020
5 Views
Preview:
TRANSCRIPT
Revising the handling of nonlinear constraints in SCIP
Work in Progress Report
Benjamin Muller, Felipe Serrano, Stefan Vigerske, and Fabian Wegscheider
ISMP · July 2, 2018
SCIP: Solving Constraint Integer Programs
• modular branch-cut-and-price
framework for constraint integer
programming
• includes full-fledged
MIP/MINLP solver
• part of SCIP Optimization Suite
(GCG, SCIP, SoPlex, UG, ZIMPL)
• new version 6.0.0 out TODAY
Release Report: The SCIP
Optimization Suite 6.0 by Gleixner,
Bastubbe, Eifler, Gally, Gamrath,
Gottwald, Hendel, Hojny, Koch,
Lubbecke, Maher, Miltenberger,
Muller, Pfetsch, Puchert, Rehfeldt,
Schlosser, Schubert, Serrano,
Shinano, Viernickel, Wegscheider,
Witt, Witzig
• free for academic use
Download at scip.zib.de:
2/23
Mixed-Integer Nonlinear Programming
min cTx
s.t. gk(x) ≤ 0 ∀k ∈ [m]
xi ∈ Z ∀i ∈ I ⊆ [n]
xi ∈ [`i , ui ] ∀i ∈ [n]
The functions gk : [`, u]→ R can be
−1
1−1
1
5
10
convex
or
0100
200300
0
200
−200
0
200
nonconvex
and are given in algebraic form.
3/23
SCIP solves MINLPs by spatial Branch & Bound
Ingredients:
• constructing an LP relaxation by
• relaxing integrality
• convexifying non-convexities
• branching on
• fractional integer variables
• variables in violated nonconvex constraints
• tightening of variable bounds (domain propagation)
• primal heuristics
• presolving / reformulation
0.5 1.0 1.5 2.0 2.5 3.0
-2.0
-1.5
-1.0
-0.5
0.5
1.0
-1.0 -0.5 0.5 1.0
-1.0
-0.5
0.5
1.0
4/23
SCIP solves MINLPs by spatial Branch & Bound
Ingredients:
• constructing an LP relaxation by
• relaxing integrality
• convexifying non-convexities
• branching on
• fractional integer variables
• variables in violated nonconvex constraints
• tightening of variable bounds (domain propagation)
• primal heuristics
• presolving / reformulation
-1.0 -0.5 0.5 1.0
0.2
0.4
0.6
0.8
1.0
-1.0 -0.5 0.5 1.0
0.2
0.4
0.6
0.8
1.0
4/23
SCIP solves MINLPs by spatial Branch & Bound
Ingredients:
• constructing an LP relaxation by
• relaxing integrality
• convexifying non-convexities
• branching on
• fractional integer variables
• variables in violated nonconvex constraints
• tightening of variable bounds (domain propagation)
• primal heuristics
• presolving / reformulation
-1.0 -0.5 0.0 0.5 1.0
-1.0
-0.5
0.0
0.5
1.0
4/23
SCIP source files that are (primarily) for MINLP (SCIP 5.0.1)
wc -l <file>
17082 nlpi/expr.c
16718 scip/cons_quadratic.c
10028 scip/cons_nonlinear.c
8085 scip/cons_bivariate.c
7429 scip/cons_abspower.c
6368 scip/nlp.c
5540 scip/cons_soc.c
4174 scip/intervalarith.c
3817 scip/reader_pip.c
3680 scip/heur_undercover.c
3315 scip/prop_obbt.c
3158 nlpi/nlpioracle.c
2989 nlpi/nlpi_ipopt.cpp
2906 nlpi/nlpi_filtersqp.c
2892 scip/sepa_eccuts.c
2837 scip/reader_gms.c
2808 scip/heur_nlpdiving.c
2670 nlpi/exprinterpret_cppad.cpp
2588 scip/reader_osil.c
2545 scip/heur_subnlp.c
2459 nlpi/nlpi_worhp.c
2009 scip/presol_qpkktref.c
1757 nlpi/pub_expr.h
1225 nlpi/nlpi_all.c
1101 scip/heur_multistart.c
1085 scip/sepa_gauge.c
wc -l <file>
926 scip/sepa_convexproj.c
853 nlpi/nlpi.c
841 scip/cons_quadratic.h
834 scip/nlp.h
769 scip/heur_mpec.c
758 scip/prop_nlobbt.c
697 scip/intervalarith.h
516 nlpi/type_nlpi.h
491 nlpi/nlpi.h
473 scip/cons_nonlinear.h
442 nlpi/nlpioracle.h
424 nlpi/intervalarithext.h
339 nlpi/nlpi_ipopt_dummy.c
314 nlpi/type_expr.h
237 scip/cons_soc.h
230 scip/pub_nlp.h
211 scip/cons_abspower.h
205 nlpi/exprinterpret_none.c
188 nlpi/struct_expr.h
184 nlpi/exprinterpret.h
181 scip/cons_bivariate.h
179 scip/struct_nlp.h
141 scip/heur_subnlp.h
112 nlpi/nlpi_ipopt.h
111 scip/prop_nlobbt.h
103 scip/presol_qpkktref.h
wc -l <file>
100 scip/heur_mpec.h
96 scip/heur_multistart.h
92 scip/sepa_eccuts.h
91 scip/reader_pip.h
90 scip/sepa_gauge.h
88 nlpi/struct_nlpi.h
85 scip/sepa_convexproj.h
82 scip/reader_gms.h
78 scip/heur_undercover.h
71 nlpi/nlpi_worhp.h
69 scip/prop_obbt.h
56 nlpi/nlpi_filtersqp_dummy.c
53 scip/heur_nlpdiving.h
53 nlpi/nlpi_all.h
52 scip/reader_osil.h
52 nlpi/nlpi_worhp_dummy.c
52 nlpi/nlpi_filtersqp.h
48 nlpi/type_exprinterpret.h
38 scip/type_nlp.h
26 nlpi/intervalarithext.cpp
133396 total
[src]$ wc -l blockmemshell/* dijkstra/* lpi/* nlpi/* objscip/* scip/* symmetry/* tclique/* tpi/* main.c | tail -1
773866 total5/23
SCIP source files that are (primarily) for MINLP (SCIP 5.0.1)
wc -l <file>
17082 nlpi/expr.c
16718 scip/cons_quadratic.c
10028 scip/cons_nonlinear.c
8085 scip/cons_bivariate.c
7429 scip/cons_abspower.c
6368 scip/nlp.c
5540 scip/cons_soc.c
4174 scip/intervalarith.c
3817 scip/reader_pip.c
3680 scip/heur_undercover.c
3315 scip/prop_obbt.c
3158 nlpi/nlpioracle.c
2989 nlpi/nlpi_ipopt.cpp
2906 nlpi/nlpi_filtersqp.c
2892 scip/sepa_eccuts.c
2837 scip/reader_gms.c
2808 scip/heur_nlpdiving.c
2670 nlpi/exprinterpret_cppad.cpp
2588 scip/reader_osil.c
2545 scip/heur_subnlp.c
2459 nlpi/nlpi_worhp.c
2009 scip/presol_qpkktref.c
1757 nlpi/pub_expr.h
1225 nlpi/nlpi_all.c
1101 scip/heur_multistart.c
1085 scip/sepa_gauge.c
wc -l <file>
926 scip/sepa_convexproj.c
853 nlpi/nlpi.c
841 scip/cons_quadratic.h
834 scip/nlp.h
769 scip/heur_mpec.c
758 scip/prop_nlobbt.c
697 scip/intervalarith.h
516 nlpi/type_nlpi.h
491 nlpi/nlpi.h
473 scip/cons_nonlinear.h
442 nlpi/nlpioracle.h
424 nlpi/intervalarithext.h
339 nlpi/nlpi_ipopt_dummy.c
314 nlpi/type_expr.h
237 scip/cons_soc.h
230 scip/pub_nlp.h
211 scip/cons_abspower.h
205 nlpi/exprinterpret_none.c
188 nlpi/struct_expr.h
184 nlpi/exprinterpret.h
181 scip/cons_bivariate.h
179 scip/struct_nlp.h
141 scip/heur_subnlp.h
112 nlpi/nlpi_ipopt.h
111 scip/prop_nlobbt.h
103 scip/presol_qpkktref.h
wc -l <file>
100 scip/heur_mpec.h
96 scip/heur_multistart.h
92 scip/sepa_eccuts.h
91 scip/reader_pip.h
90 scip/sepa_gauge.h
88 nlpi/struct_nlpi.h
85 scip/sepa_convexproj.h
82 scip/reader_gms.h
78 scip/heur_undercover.h
71 nlpi/nlpi_worhp.h
69 scip/prop_obbt.h
56 nlpi/nlpi_filtersqp_dummy.c
53 scip/heur_nlpdiving.h
53 nlpi/nlpi_all.h
52 scip/reader_osil.h
52 nlpi/nlpi_worhp_dummy.c
52 nlpi/nlpi_filtersqp.h
48 nlpi/type_exprinterpret.h
38 scip/type_nlp.h
26 nlpi/intervalarithext.cpp
133396 total
[src]$ wc -l blockmemshell/* dijkstra/* lpi/* nlpi/* objscip/* scip/* symmetry/* tclique/* tpi/* main.c | tail -1
773866 total5/23
The “classical” framework for (MI)NLP
in SCIP
Main SCIP Constraint Handlers for NLP
Quadratic Constraints lhs ≤n∑
i,j=1
ai,jxixj +n∑
i=1
bixi ≤ rhs
Absolute Power Constraints lhs ≤ sign(x + a)|x + a|n + c z ≤ rhs, n ∈ R≥1
Nonlinear Constraints lhs ≤n∑
i=1
aixi +m∑j=1
cj fj(x) ≤ rhs
fj(·) is given as expression tree
Thus, nonlinearities are distinguished into
• convex and concave (cons nonlinear)
• quadratic, esp. x · y (cons quadratic)
• odd power (x3, x5, sign(x)|x |n) (cons abspower)
Separation (LP relaxation) is implemented for these 4 types.
Additional specialized handlers: SOC and bivariate 1-convex. These are optional.
6/23
Main SCIP Constraint Handlers for NLP
Quadratic Constraints lhs ≤n∑
i,j=1
ai,jxixj +n∑
i=1
bixi ≤ rhs
Absolute Power Constraints lhs ≤ sign(x + a)|x + a|n + c z ≤ rhs, n ∈ R≥1
Nonlinear Constraints lhs ≤n∑
i=1
aixi +m∑j=1
cj fj(x) ≤ rhs
fj(·) is given as expression tree
Thus, nonlinearities are distinguished into
• convex and concave (cons nonlinear)
• quadratic, esp. x · y (cons quadratic)
• odd power (x3, x5, sign(x)|x |n) (cons abspower)
Separation (LP relaxation) is implemented for these 4 types.
Additional specialized handlers: SOC and bivariate 1-convex. These are optional.
6/23
Main SCIP Constraint Handlers for NLP
Quadratic Constraints lhs ≤n∑
i,j=1
ai,jxixj +n∑
i=1
bixi ≤ rhs
Absolute Power Constraints lhs ≤ sign(x + a)|x + a|n + c z ≤ rhs, n ∈ R≥1
Nonlinear Constraints lhs ≤n∑
i=1
aixi +m∑j=1
cj fj(x) ≤ rhs
fj(·) is given as expression tree
Thus, nonlinearities are distinguished into
• convex and concave (cons nonlinear)
• quadratic, esp. x · y (cons quadratic)
• odd power (x3, x5, sign(x)|x |n) (cons abspower)
Separation (LP relaxation) is implemented for these 4 types.
Additional specialized handlers: SOC and bivariate 1-convex. These are optional.
6/23
Expression trees and graphs
cons nonlinear (lhs ≤n∑
i=1
aixi +m∑j=1
cj fj(x) ≤ rhs) stores the nonlinear functions fj of
all constraints in one expression graph (DAG).
For example (instance nvs01):
420.169√
900 + x21 − x3x1x2 = 0
2960.88 + 296088 · 0.0625x22
7200 + x21
− x3 ≥ 0
xobj − 0.047x2
√900 + x2
1 ≥ 0
y 1
y 2
y2
y1y
3
y4
y 1
y1
y1
y2
y1
x2x3 x1
7200 + y 21900 + y 2
1
−y1y2y4 + 420.169y 0.53
420.169√
900 + x21 − x3x1x2
÷
2960.88 + 18505.5x22
7200 + x21
−0.047y1y0.52
−0.047x2
√900 + x2
1
2960.88 + 18505.5y 21
• some use of common subexpression7/23
Expression operators
Operators:
• variable index, constant
• +, −, ∗, ÷
• ·2,√·, ·p (p ∈ R), ·n (n ∈ Z), x 7→ x |x |p−1 (p > 1)
• exp, log
• min, max, abs
•∑
,∏
, affine-linear, quadratic, signomial
• (user)
8/23
Reformulation in cons nonlinear (during presolve)
Goal: Reformulate constraints such that only elementary cases (convex, concave, odd
power, quadratic) remain.
900 + x21 = z1 7200 + x2
1 = z4 420.169√z1 − x3z5 = 0
−z3 + z2z4 = 0 x1x2 = z5 z2 − x3 ≥ 0
2960.88 + 18505.5x22 = z3
√z1 = z6 0.047x2z6 ≤ xobj
y2
y 1y 1
y 1
y1
y2
y1
y2
y1
y1y 3y 1
y1
y2
y2
y1
y 1
y 1z5 z4z3z2z1x2 x3x1
420.169y 0.51
z6
7200 + y 21
7200 + x21
900 + y 21
900 + x21
0.047y1
0.047x2z6
−y1 + y2y3
−z3 + z2z4
y1 + y2
420.169√z1 − x3z5
−y1
2960.88 + 18505.5y 21
2960.88 + 18505.5x22
√y1
√z1
y1y2
y1y2
x1x2
y1y2
• reformulates constraints by introducing new variables and new constraints
• other cons. handler can participate (SCIP DECL EXPRGRAPHNODEREFORM callback) 9/23
So why rewriting a seemingly perfect piece of code?
Considermin z
s.t. exp(ln(1000) + 1 + x y) ≤ z
x2 + y 2 ≤ 2
An optimal solution:
x = −1
y = 1
z = 1000
SCIP reports
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.08
Solving Nodes : 5
Primal Bound : +9.99999656552062e+02 (3 solutions)
Dual Bound : +9.99999656552062e+02
Gap : 0.00 %
[nonlinear] <e1>: exp((7.9077552789821368151 +1 (<x> * <y>)))-1<z>[C] <= 0;
violation: right hand side is violated by 0.000673453314561812
best solution is not feasible in original problem
x -1.00057454873626 (obj:0)
y 0.999425451364613 (obj:0)
z 999.999656552061 (obj:1)
10/23
So why rewriting a seemingly perfect piece of code?
Considermin z
s.t. exp(ln(1000) + 1 + x y) ≤ z
x2 + y 2 ≤ 2
An optimal solution:
x = −1
y = 1
z = 1000
SCIP reports
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.08
Solving Nodes : 5
Primal Bound : +9.99999656552062e+02 (3 solutions)
Dual Bound : +9.99999656552062e+02
Gap : 0.00 %
[nonlinear] <e1>: exp((7.9077552789821368151 +1 (<x> * <y>)))-1<z>[C] <= 0;
violation: right hand side is violated by 0.000673453314561812
best solution is not feasible in original problem
x -1.00057454873626 (obj:0)
y 0.999425451364613 (obj:0)
z 999.999656552061 (obj:1)
10/23
So why rewriting a seemingly perfect piece of code?
Considermin z
s.t. exp(ln(1000) + 1 + x y) ≤ z
x2 + y 2 ≤ 2
An optimal solution:
x = −1
y = 1
z = 1000
SCIP reports
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.08
Solving Nodes : 5
Primal Bound : +9.99999656552062e+02 (3 solutions)
Dual Bound : +9.99999656552062e+02
Gap : 0.00 %
[nonlinear] <e1>: exp((7.9077552789821368151 +1 (<x> * <y>)))-1<z>[C] <= 0;
violation: right hand side is violated by 0.000673453314561812
best solution is not feasible in original problem
x -1.00057454873626 (obj:0)
y 0.999425451364613 (obj:0)
z 999.999656552061 (obj:1)
10/23
SCIP output
min z s.t. exp(ln(1000) + 1 + x y) ≤ z , x2 + y 2 ≤ 2presolving (5 rounds: 5 fast, 1 medium, 1 exhaustive):
0 deleted vars, 0 deleted constraints, 1 added constraints, 6 tightened bounds, 0 added holes, 0 changed sides, 0 changed coefficients
0 implications, 0 cliques
presolved problem has 4 variables (0 bin, 0 int, 0 impl, 4 cont) and 3 constraints
2 constraints of type <quadratic>
1 constraints of type <nonlinear>
Presolving Time: 0.00
time | node | left |LP iter|LP it/n| mem |mdpt |extbr|vars |cons |cols |rows |cuts |confs|strbr| dualbound | primalbound | gap
0.0s| 1 | 0 | 1 | - | 565k| 0 | 0 | 4 | 3 | 4 | 8 | 0 | 0 | 0 | 3.678794e+02 | 1.000000e+05 | Large
******************************************************************************
This program contains Ipopt, a library for large-scale nonlinear optimization.
Ipopt is released as open source code under the Eclipse Public License (EPL).
For more information visit http://projects.coin-or.org/Ipopt
******************************************************************************
q 0.0s| 1 | 0 | 1 | - | 569k| 0 | 0 | 4 | 3 | 4 | 8 | 0 | 0 | 0 | 3.678794e+02 | 9.999999e+02 | 171.83%
...
0.1s| 1 | 2 | 39 | - | 629k| 0 | 2 | 4 | 3 | 4 | 21 | 14 | 0 | 0 | 4.367357e+02 | 9.999999e+02 | 128.97%
* 0.1s| 2 | 1 | 56 | 42.0 | 651k| 1 | 0 | 4 | 3 | 4 | 18 | 28 | 0 | 0 | 4.367357e+02 | 9.999997e+02 | 128.97%
0.1s| 3 | 2 | 66 | 26.0 | 653k| 1 | 2 | 4 | 3 | 4 | 12 | 35 | 0 | 0 | 8.291193e+02 | 9.999997e+02 | 20.61%
0.1s| 4 | 1 | 66 | 17.3 | 653k| 2 | 0 | 4 | 3 | 0 | 0 | 35 | 0 | 0 | 8.291193e+02 | 9.999997e+02 | 20.61%
0.1s| 5 | 0 | 72 | 14.5 | 653k| 2 | 0 | 4 | 3 | 4 | 11 | 38 | 0 | 0 | 9.999997e+02 | 9.999997e+02 | 0.00%
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.08
Solving Nodes : 5
Primal Bound : +9.99999656552062e+02 (3 solutions)
Dual Bound : +9.99999656552062e+02
Gap : 0.00 %
[nonlinear] <e1>: exp((7.9077552789821368151 +1 (<x> * <y>)))-1<z>[C] <= 0;
violation: right hand side is violated by 0.000673453314561812 (scaled: 0.000673453314561812)
best solution is not feasible in original problem
11/23
Reformulated problem
Reformulation takes apart exp(ln(1000) + 1 + x y), thus SCIP actually solves
min z
Violation
s.t. exp(w) ≤ z
0.4659 · 10−6 ≤ numerics/feastol X
ln(1000) + 1 + x y = w
0.6731 · 10−6 ≤ numerics/feastol X
x2 + y 2 ≤ 2
0.6602 · 10−6 ≤ numerics/feastol X
Solution (found by <relaxation>):
x = -1.000574549
y = 0.999425451
z = 999.999656552
w= 6.907754936
⇒ Explicit reformulation of constraints ...
• ... looses the connection to the original problem.
• ... looses distinction between original and auxiliary variables. Thus, we may
branch on auxiliary variables.
• ... prevents simultaneous exploitation of overlapping structures.
12/23
Reformulated problem
Reformulation takes apart exp(ln(1000) + 1 + x y), thus SCIP actually solves
min z Violation
s.t. exp(w) ≤ z 0.4659 · 10−6 ≤ numerics/feastol X
ln(1000) + 1 + x y = w 0.6731 · 10−6 ≤ numerics/feastol X
x2 + y 2 ≤ 2 0.6602 · 10−6 ≤ numerics/feastol X
Solution (found by <relaxation>):
x = -1.000574549
y = 0.999425451
z = 999.999656552
w= 6.907754936
⇒ Explicit reformulation of constraints ...
• ... looses the connection to the original problem.
• ... looses distinction between original and auxiliary variables. Thus, we may
branch on auxiliary variables.
• ... prevents simultaneous exploitation of overlapping structures.
12/23
Reformulated problem
Reformulation takes apart exp(ln(1000) + 1 + x y), thus SCIP actually solves
min z Violation
s.t. exp(w) ≤ z 0.4659 · 10−6 ≤ numerics/feastol X
ln(1000) + 1 + x y = w 0.6731 · 10−6 ≤ numerics/feastol X
x2 + y 2 ≤ 2 0.6602 · 10−6 ≤ numerics/feastol X
Solution (found by <relaxation>):
x = -1.000574549
y = 0.999425451
z = 999.999656552
w= 6.907754936
⇒ Explicit reformulation of constraints ...
• ... looses the connection to the original problem.
• ... looses distinction between original and auxiliary variables. Thus, we may
branch on auxiliary variables.
• ... prevents simultaneous exploitation of overlapping structures.
12/23
Reformulated problem
Reformulation takes apart exp(ln(1000) + 1 + x y), thus SCIP actually solves
min z Violation
s.t. exp(w) ≤ z 0.4659 · 10−6 ≤ numerics/feastol X
ln(1000) + 1 + x y = w 0.6731 · 10−6 ≤ numerics/feastol X
x2 + y 2 ≤ 2 0.6602 · 10−6 ≤ numerics/feastol X
Solution (found by <relaxation>):
x = -1.000574549
y = 0.999425451
z = 999.999656552
w= 6.907754936
⇒ Explicit reformulation of constraints ...
• ... looses the connection to the original problem.
• ... looses distinction between original and auxiliary variables. Thus, we may
branch on auxiliary variables.
• ... prevents simultaneous exploitation of overlapping structures.12/23
A new framework for NLP in SCIP
Main Ideas
Everything is an expression.
• ONE constraint handler: cons expr
• represent all nonlinear constraints in one expression graph (DAG)
lhs ≤ expression-node ≤ rhs
• all algorithms (check, separation, propagation, etc.) work on the expression graph
(no upgrades to specialized nonlinear constraints)
• separate expression operators (+, ×) and high-level structures (quadratic, etc.)
⇒ avoid redundancy / ambiguity of expression types (classic: +,∑
, linear, quad., . . . )
• stronger identification of common subexpressions
Do not reformulate constraints.
• introduce auxiliary variables for the relaxation only
13/23
Main Ideas
Everything is an expression.
• ONE constraint handler: cons expr
• represent all nonlinear constraints in one expression graph (DAG)
lhs ≤ expression-node ≤ rhs
• all algorithms (check, separation, propagation, etc.) work on the expression graph
(no upgrades to specialized nonlinear constraints)
• separate expression operators (+, ×) and high-level structures (quadratic, etc.)
⇒ avoid redundancy / ambiguity of expression types (classic: +,∑
, linear, quad., . . . )
• stronger identification of common subexpressions
Do not reformulate constraints.
• introduce auxiliary variables for the relaxation only
13/23
Main Ideas
Everything is an expression.
• ONE constraint handler: cons expr
• represent all nonlinear constraints in one expression graph (DAG)
lhs ≤ expression-node ≤ rhs
• all algorithms (check, separation, propagation, etc.) work on the expression graph
(no upgrades to specialized nonlinear constraints)
• separate expression operators (+, ×) and high-level structures (quadratic, etc.)
⇒ avoid redundancy / ambiguity of expression types (classic: +,∑
, linear, quad., . . . )
• stronger identification of common subexpressions
Do not reformulate constraints.
• introduce auxiliary variables for the relaxation only
13/23
Enforcement
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
This formulation is used to
• check feasibility,
• presolve,
• propagate domains, ...
(Implicit) Reformulation: Used to construct
LP relaxation.
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
14/23
Enforcement
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
This formulation is used to
• check feasibility,
• presolve,
• propagate domains, ...
(Implicit) Reformulation:
w1 ≤ 4
log(x)2 + 2 log(x)y + y 2 = w1
Used to construct LP relaxation.
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
14/23
Enforcement
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
This formulation is used to
• check feasibility,
• presolve,
• propagate domains, ...
(Implicit) Reformulation:
w1 ≤ 4
w2 + 2w3 + w4 = w1
log(x)2 = w2
log(x)y = w3
y 2 = w4
Used to construct LP relaxation.
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
14/23
Enforcement
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
This formulation is used to
• check feasibility,
• presolve,
• propagate domains, ...
(Implicit) Reformulation:
w1 ≤ 4
w2 + 2w3 + w4 = w1
w 25 = w2
w5y = w3
y 2 = w4
log(x) = w5
Used to construct LP relaxation.
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
14/23
Enforcement
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
This formulation is used to
• check feasibility,
• presolve,
• propagate domains, ...
(Implicit) Reformulation:
w1 ≤ 4
w2 + 2w3 + w4 = w1
w 25 = w2
w5y = w3
y 2 = w4
log(x) = w5
Used to construct LP relaxation.
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
14/23
“Semi-auxiliaries” (see Couenne)
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
(Implicit) Reformulation:
taking monotonicity
into account:
w1 ≤ 4
w2 + 2w3 + w4 = w1
w 25 = w2
w5y = w3
y 2 = w4
log(x) = w5
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
If x ≥ 1 and y ≥ 0, then log(x) = w5 can be relaxed to log(x) ≤ w5.
If x ≤ 1 and y ≤ 0, then log(x) = w5 can be relaxed to log(x) ≥ w5.
15/23
“Semi-auxiliaries” (see Couenne)
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
(Implicit) Reformulation taking monotonicity
into account:
w1 ≤ 4
w2 + 2w3 + w4 ≤ w1
w 25 ≤ w2
w5y ≤ w3
y 2 ≤ w4
log(x) = w5
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
If x ≥ 1 and y ≥ 0, then log(x) = w5 can be relaxed to log(x) ≤ w5.
If x ≤ 1 and y ≤ 0, then log(x) = w5 can be relaxed to log(x) ≥ w5.
15/23
“Semi-auxiliaries” (see Couenne)
Constraint:
log(x)2 + 2 log(x)y + y 2 ≤ 4
(Implicit) Reformulation taking monotonicity
into account:
w1 ≤ 4
w2 + 2w3 + w4 ≤ w1
w 25 ≤ w2
w5y ≤ w3
y 2 ≤ w4
log(x) = w5
+
·2
log
x
×
2
·2
y
w1
w2 w3 w4
w5
If x ≥ 1 and y ≥ 0, then log(x) = w5 can be relaxed to log(x) ≤ w5.
If x ≤ 1 and y ≤ 0, then log(x) = w5 can be relaxed to log(x) ≥ w5.
15/23
Expression handler
Each operator type (+, ×, pow, etc.) is implemented by an expression handler, which
can provide a number of callbacks:
• evaluate and differentiate expression w.r.t. operands
• interval evaluation and tighten bounds on operands
• provide linear under- and over-estimators
• distribute branching scores to operands
• inform about curvature, monotonicity, integrality
• simplify, compare, print, parse, hash, copy, etc.
Expression handler are like other SCIP plugins, thus new ones can be added by users.
16/23
Exploiting structure
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
Smarter reformulation:
• Recognize that log(x)2 + 2 log(x)y + y 2 is convex in (log(x), y).
⇒ Introduce auxiliary variable for log(x) only.
w 2 + 2wy + y 2 ≤ 4
log(x) = w
Separate w 2 + 2wy + y 2 ≤ 4 by linearization of w 2 + 2wy + y 2.
Nonlinearity Handler (any suggestion for a better name?):
• Adds additional separation and propagation algorithms for structures that can be
identified in the expression graph.
• Attached to nodes in expression graph, but does not define expressions or
constraints.
• Examples: quadratics, convex subexpressions, vertex-polyhedral
17/23
Exploiting structure
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
Smarter reformulation:
• Recognize that log(x)2 + 2 log(x)y + y 2 is convex in (log(x), y).
⇒ Introduce auxiliary variable for log(x) only.
w 2 + 2wy + y 2 ≤ 4
log(x) = w
Separate w 2 + 2wy + y 2 ≤ 4 by linearization of w 2 + 2wy + y 2.
Nonlinearity Handler (any suggestion for a better name?):
• Adds additional separation and propagation algorithms for structures that can be
identified in the expression graph.
• Attached to nodes in expression graph, but does not define expressions or
constraints.
• Examples: quadratics, convex subexpressions, vertex-polyhedral
17/23
Exploiting structure
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
Smarter reformulation:
• Recognize that log(x)2 + 2 log(x)y + y 2 is convex in (log(x), y).
⇒ Introduce auxiliary variable for log(x) only.
w 2 + 2wy + y 2 ≤ 4
log(x) = w
Separate w 2 + 2wy + y 2 ≤ 4 by linearization of w 2 + 2wy + y 2.
Nonlinearity Handler (any suggestion for a better name?):
• Adds additional separation and propagation algorithms for structures that can be
identified in the expression graph.
• Attached to nodes in expression graph, but does not define expressions or
constraints.
• Examples: quadratics, convex subexpressions, vertex-polyhedral
17/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
Nonlinearity Handler in Expression Graph
• Nodes in the expression graph can have one or several nlhdlrs attached.
• At beginning of solve, detection callbacks are run only for nodes that have
auxiliary variable. Detection callback may add auxiliary variables.
Constraint: log(x)2 + 2 log(x)y + y 2 ≤ 4
w1 ≤ 4
w 22 + 2w2y + y 2 ≤ w1 [nlhdlr quadratic]
log(x) = w2 [expr log]
1. Add auxiliary variable w1 for root.
2. Run detect of all nlhdlrs on + node.
• nlhdlr quadratic detects a convex quadratic
structure and signals success.
• nlhdlr quadratic adds an auxiliary variable w2
for log node.
3. Run detect of all nlhdlrs on log node.
• No specialized nlhdlr signals success.
The expression handler will be used.
+
·2
log
x
×
2
·2
y
w1
quadratic
w2
expr log
18/23
State of Development
What exists so far
Constraint Handler (cons expr):
• upgrades from cons quadratic and cons nonlinear
• fundamental callbacks + parse, write, copy, separate, propagate, presolve
• canonicalization: simplify, common subexpressions
• add to NLP relaxation (conversion to classic expressions)
Expression Handler:
• var, value, sum, product, pow, abs, exp, log, cos, sin, entropy
• product: convex hull for moderate number of factors (2–13)
Nonlinearity Handler:
• quadratic: recognize and separate convex quadratic; domain propagation
• convex: recognize some simple general convexities, separate by linearization
• (default: wrap around expression handler)
19/23
What exists so far
Constraint Handler (cons expr):
• upgrades from cons quadratic and cons nonlinear
• fundamental callbacks + parse, write, copy, separate, propagate, presolve
• canonicalization: simplify, common subexpressions
• add to NLP relaxation (conversion to classic expressions)
Expression Handler:
• var, value, sum, product, pow, abs, exp, log, cos, sin, entropy
• product: convex hull for moderate number of factors (2–13)
Nonlinearity Handler:
• quadratic: recognize and separate convex quadratic; domain propagation
• convex: recognize some simple general convexities, separate by linearization
• (default: wrap around expression handler)
19/23
What exists so far
Constraint Handler (cons expr):
• upgrades from cons quadratic and cons nonlinear
• fundamental callbacks + parse, write, copy, separate, propagate, presolve
• canonicalization: simplify, common subexpressions
• add to NLP relaxation (conversion to classic expressions)
Expression Handler:
• var, value, sum, product, pow, abs, exp, log, cos, sin, entropy
• product: convex hull for moderate number of factors (2–13)
Nonlinearity Handler:
• quadratic: recognize and separate convex quadratic; domain propagation
• convex: recognize some simple general convexities, separate by linearization
• (default: wrap around expression handler)
19/23
Motivating example revisited
min z s.t. exp(ln(1000) + 1 + x y) ≤ z , x2 + y 2 ≤ 2
Classic:
presolving (5 rounds: 5 fast, 1 medium, 1 exhaustive):
0 deleted vars, 0 deleted constraints, 1 added constraints,...
0 implications, 0 cliques
presolved problem has 4 variables (0 bin, 0 int, 0 impl, 4 cont)
and 3 constraints
2 constraints of type <quadratic>
1 constraints of type <nonlinear>
[...]
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.08
Solving Nodes : 5
Primal Bound : +9.99999656552062e+02 (3 solutions)
Dual Bound : +9.99999656552062e+02
Gap : 0.00 %
[nonlinear] <e1>: exp((7.90776 + (<x> * <y>)))-1<z>[C] <= 0;
violation: right hand side is violated by 0.000673453314561812
best solution is not feasible in original problem
x -1.00057454873626 (obj:0)
y 0.999425451364613 (obj:0)
z 999.999656552061 (obj:1)
New:
presolving (3 rounds: 3 fast, 1 medium, 1 exhaustive):
0 deleted vars, 0 deleted constraints, 0 added constraints,...
0 implications, 0 cliques
presolved problem has 3 variables (0 bin, 0 int, 0 impl, 3 cont)
and 2 constraints
2 constraints of type <expr>
[...]
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.47
Solving Nodes : 15
Primal Bound : +9.99999949950021e+02 (2 solutions)
Dual Bound : +9.99999949950021e+02
Gap : 0.00 %
x -1.00000002499999 (obj:0)
y 1.00000002499999 (obj:0)
z 999.999949950021 (obj:1)
20/23
Motivating example revisited
min z s.t. exp(ln(1000) + 1 + x y) ≤ z , x2 + y 2 ≤ 2
Classic:
presolving (5 rounds: 5 fast, 1 medium, 1 exhaustive):
0 deleted vars, 0 deleted constraints, 1 added constraints,...
0 implications, 0 cliques
presolved problem has 4 variables (0 bin, 0 int, 0 impl, 4 cont)
and 3 constraints
2 constraints of type <quadratic>
1 constraints of type <nonlinear>
[...]
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.08
Solving Nodes : 5
Primal Bound : +9.99999656552062e+02 (3 solutions)
Dual Bound : +9.99999656552062e+02
Gap : 0.00 %
[nonlinear] <e1>: exp((7.90776 + (<x> * <y>)))-1<z>[C] <= 0;
violation: right hand side is violated by 0.000673453314561812
best solution is not feasible in original problem
x -1.00057454873626 (obj:0)
y 0.999425451364613 (obj:0)
z 999.999656552061 (obj:1)
New:
presolving (3 rounds: 3 fast, 1 medium, 1 exhaustive):
0 deleted vars, 0 deleted constraints, 0 added constraints,...
0 implications, 0 cliques
presolved problem has 3 variables (0 bin, 0 int, 0 impl, 3 cont)
and 2 constraints
2 constraints of type <expr>
[...]
SCIP Status : problem is solved [optimal solution found]
Solving Time (sec) : 0.47
Solving Nodes : 15
Primal Bound : +9.99999949950021e+02 (2 solutions)
Dual Bound : +9.99999949950021e+02
Gap : 0.00 %
x -1.00000002499999 (obj:0)
y 1.00000002499999 (obj:0)
z 999.999949950021 (obj:1)
20/23
The Benchmark Slide
• 1513 instances from MINLPLib (www.minlplib.org)
• 1800s timelimit, 0.0001 gaplimit, LP: CPLEX 12.8.0.0, NLP: Ipopt 3.12.5
• classic = SCIP 6.0.0; new = new framework based on SCIP 6.0.0
# instances1 classic new classic → new
# instances handled2 1513 1453 1507 -1+55
# fail/abort/no-result3 1452 37 22 -31+16
# sol. infeas. > 10−6 1452 114 4 -112+ 2
# solved 1452 816 813 -72+69
time (sh.geom.mean) [s] 744 9.1 12.8 -306+1314
1number of instances considered in this row: 1513 = all
; 1452 = both frameworks can handle (no
sin/cos/min/max/erf); 744 = solved by both frameworks
2handled = only supported expression operators
3no-result = run did not return from cluster4number of instances with ≥ 10% increase/decrease in time
21/23
The Benchmark Slide
• 1513 instances from MINLPLib (www.minlplib.org)
• 1800s timelimit, 0.0001 gaplimit, LP: CPLEX 12.8.0.0, NLP: Ipopt 3.12.5
• classic = SCIP 6.0.0; new = new framework based on SCIP 6.0.0
# instances1 classic new classic → new
# instances handled2 1513 1453 1507 -1+55
# fail/abort/no-result3 1452 37 22 -31+16
# sol. infeas. > 10−6 1452 114 4 -112+ 2
# solved 1452 816 813 -72+69
time (sh.geom.mean) [s] 744 9.1 12.8 -306+1314
1number of instances considered in this row: 1513 = all; 1452 = both frameworks can handle (no
sin/cos/min/max/erf)
; 744 = solved by both frameworks
2handled = only supported expression operators3no-result = run did not return from cluster
4number of instances with ≥ 10% increase/decrease in time
21/23
The Benchmark Slide
• 1513 instances from MINLPLib (www.minlplib.org)
• 1800s timelimit, 0.0001 gaplimit, LP: CPLEX 12.8.0.0, NLP: Ipopt 3.12.5
• classic = SCIP 6.0.0; new = new framework based on SCIP 6.0.0
# instances1 classic new classic → new
# instances handled2 1513 1453 1507 -1+55
# fail/abort/no-result3 1452 37 22 -31+16
# sol. infeas. > 10−6 1452 114 4 -112+ 2
# solved 1452 816 813 -72+69
time (sh.geom.mean) [s] 744 9.1 12.8 -306+1314
1number of instances considered in this row: 1513 = all; 1452 = both frameworks can handle (no
sin/cos/min/max/erf); 744 = solved by both frameworks2handled = only supported expression operators3no-result = run did not return from cluster4number of instances with ≥ 10% increase/decrease in time
21/23
WIP / TODO
• improved separation and propagation for bilinear terms: see next talk
• Reformulation-Linearization Technique (RLT) for quadratics
• more convexity detection, e.g., xex ; f (g(x), y) with only f (·, ·) convex
• separation for more vertex-polyhedral functions, e.g., concave
• handling of second-order cone constraints (!= expressions)
• nonlinearity handler during presolve
• replace classic by new: adapt readers, NLP, primal heuristics, propagators, etc.
• performance tuning and debugging; documentation (?)
• . . .
22/23
End.
Summary:
The handling of nonlinear constraints in SCIP is rewritten.
The new code will be nicer, better, faster, greater:
• less issues with slightly infeasible solutions
• easier to extend by own operators and structure-exploiting algorithms
(see also talk by Felipe tomorrow 15:45, Salle 34)
• understood by 4 (instead of only 1) SCIP developers
• support for sine and cosine
. . . if it ever gets released.
Thank you for your attention!
23/23
End.
Summary:
The handling of nonlinear constraints in SCIP is rewritten.
The new code will be nicer, better, faster, greater:
• less issues with slightly infeasible solutions
• easier to extend by own operators and structure-exploiting algorithms
(see also talk by Felipe tomorrow 15:45, Salle 34)
• understood by 4 (instead of only 1) SCIP developers
• support for sine and cosine
. . . if it ever gets released.
Thank you for your attention!
23/23
top related