welcome to the webinar - gurobi · welcome to the webinar migrating models from other solvers to...

Welcome to the webinarMigrating models from other solvers to use the Gurobi solver

Renan Garcia, Ph.D.

Optimization Support Engineer at Gurobi Optimization

} Ph.D. in Industrial and Systems Engineering, Georgia Tech} Expert in optimization modeling and software development} Over a decade of experience implementing decision support systems

© 2016 Gurobi Optimization

David Nehme, Ph.D.

Principal Consultant at Abremod

} 20 years of software development experience} 15 years implementing CPLEX and Gurobi models} Ph.D. in Operations Research for The University of Texas} Top StackOverflow answerer for both CPLEX and Gurobi tags


Agenda

} Switching is easier than you may think

} Common migration scenarios

} Migrating from OPL

} Migrating from CPLEX Concert API

} Best practices, limitations and considerations



Switching is Easier Than You May Think

Key Migration Issues

} Building the model◦ How do I build my optimization model?◦ Do I build it one constraint at a time, or do I build an entire constraint

matrix?

} Setting solver parameters◦ What solver parameters do I change?◦ What effects are these changes intended to produce?◦ Am I looking for an optimal solution, or just a good feasible solution?

} Computing and extracting the solution◦ How do I extract the solution produced by the solver?


Interfacing with Gurobi

} Command-Line Tool

} Full-featured Interactive Shell

} Matrix-oriented APIs◦ C, MATLAB, R

} Object-oriented APIs◦ C++, Java, .NET, Python

} Modeling systems◦ Commercial: AIMMS, AMPL, Frontline Solvers, GAMS, MPL, …◦ Free: CMPL, JuliaOpt, OSI, PuLP, PYOMO, SolverStudio, YALMIP, …


Do-It-Yourself Resources on www.gurobi.com

} Switching guidance:https://www.gurobi.com/resources/switching-to-gurobi/switching-overview

} Documentation:https://www.gurobi.com/documentation/◦ Quick Start Guides◦ Reference Manual

� APIs� Attributes� Parameters� Tuning� …

◦ Example Tour� 22 functional examples

} Seminars and videos:https://www.gurobi.com/resources/seminars-and-videos/seminars-videos



Common Migration Scenarios

Scenario 1: Migrating with Model Files

} Example: performance testing

} Export model file with virtually no changes needed to existing code ◦ Gurobi supports several file formats (MPS, LP, …)

} Export guidance:https://www.gurobi.com/resources/switching-to-gurobi/exporting-mps-files-from-competing-solvers

} Use Gurobi Command-Line Tool to solve the model ◦ Usage: gurobi_cl [parameters] filename◦ Ex: gurobi_cl TimeLimit=3600 misc07.mps

} "Quick-and-dirty" approach◦ Limited ability to interact with solver (parameters only)◦ For more control, try Interactive Shell


Gurobi Parameters

} Control algorithmic behavior◦ Defaults validated against large

internal test set of models

} Termination criteria◦ Ex. TimeLimit, SolutionLimit

} Tolerances◦ Ex. MIPGap, BarConvTol

} Simplex and barrier◦ Ex. Method, Crossover

} MIP◦ Ex. MIPFocus, ImproveStartTime

} MIP cuts◦ Ex. Cuts, GomoryPasses

} Tuning and distributed algorithms◦ Ex. TuneJobs, WorkerPool

} General◦ Ex. Presolve, OutputFlag


Scenario 2: Using a Modeling System

} Example: model is written in AMPL

} Migrating is extremely easy for solver-independent systems

◦ Obtain license◦ Set solver to Gurobi◦ Convert parameter settings

◦ Ex: in AMPL model file, addoption solver gurobi_ampl;

option gurobi_options 'mipfocus 1';

} Need to migrate your existing model code for single-solver systems◦ OPL, Mosel, …


Gurobi Python Environment

} High-level optimization modeling constructs embedded in Python programming language

} Combines expressiveness of a modeling language with the power and flexibility of a programming language◦ Bring "feel" of a modeling language to the Python interface

} Requires minimal programming skills to get started

} Support all solver and programming needs

} Several seminars on this topic:https://www.gurobi.com/resources/seminars-and-videos/seminars-videos


Interactive Examples


Scenario 3: Porting Matrix-Oriented Code

} Example: C program which calls CPLEX Callable Library, Xpress, …

} Gurobi's C API supports sparse matrix format◦ Standard format used by many solvers

� Simple arrays represent matrix coefficients and their index positions◦ Ex: GRBaddconstrs(), GRBaddvars()◦ Minimal changes required to existing code

} Gurobi also supports advanced features◦ Callbacks◦ Advanced simplex routines (querying tableau rows)◦ …

} Must consider some Gurobi-specific modeling features when porting existing code◦ Gurobi environments◦ Lazy updates◦ Attributes


Gurobi Environments

} Models are built from an environment

} Parameters are set on an environment

} Models get their own copy of the environment ◦ Once a model is created, subsequent parameter changes in parent

environment are not reflected in model environment ◦ Use getEnv() functions to get the environment from model

} Setting parameters in C ◦ Ex: set time limit of 3600 seconds for parent environment

status = GRBsetdblparam(env, "TimeLimit", 3600); ◦ Ex: set presolve level to 2 for model's environment

status = GRBsetintparam(GRBgetenv(model), "Presolve", 2);


Lazy Updates

} Gurobi updates models in batch mode◦ Model creation and updates are efficient

} Must call update() functions to use model elements ◦ Ex: Call after creating a variable before using it in a constraint ◦ May require changes to code for other solvers

} UpdateMode=1 parameter setting allows you to use elements immediately


Attributes

} Unified system to access model elements◦ Attributes work the same across all Gurobi interfaces

} Access via a basic set of get/set functions◦ Attribute name is specified as a parameter◦ Replaces many functions used by other solvers

} Getting/Setting attributes in C◦ Use get/set functions by type (int, double, char, string)◦ Ex: query number of nonzeros in a model

status = GRBgetintattr(model, "NumNZs", &nzs);◦ Ex: query solution vector

double x[NUMVARS];status = GRBgetdblattrarray(model, "X", 0, NUMVARS, x);

◦ Ex: modify constraint RHS to 1status = GRBsetdblattrelement(model, "RHS", cidx, 1.0);


Scenario 4: Porting Object-Oriented Code

} Example: Java program which uses CPLEX Concert Technology

} Gurobi's OO APIs represent models using objects◦ Objects for variables and constraints ◦ Methods are used to create model elements◦ Ex: add simple constraint x + y ≥ 1 in C++

c1 = model.addConstr(x + y >= 1, "c1");

} All Gurobi APIs are just thin layers on top of same native C code

} Must consider same Gurobi-specific modeling features when porting◦ Subsequent parameter changes in parent environment not reflected in

model environment � Java Ex: model.getEnv().set(GRB.IntParam.Presolve, 2);

◦ Must call model's update() method to use elements (unless UpdateMode=1)◦ Use get/set methods on objects to access attributes

� Python Ex: constr.setAttr('RHS', 1.0)


Don't Panic, Your Code Often Looks Similar

} CPLEX Concert C++:IloEnv env; // create empty environmentIloModel model(env); // create empty model

IloNumVar x(env, 0, 10, ILOINT); // add variables

model.add(x);

// ...

model.add(x + 2*y <= 1); // add constraints// ...

} Gurobi C++:GRBEnv env; // create empty environment

env.set(GRB_IntParam_UpdateMode, 1); // enable automatic updates

GRBModel model(env); // create empty model

GRBVar x = model.addVar(0, 10, 0, GRB_INTEGER); // add variables

// ...model.addConstr(x + 2*y <= 1); // add constraints

// ...


Solver-specific Guidance



} Cover two disparate examples◦ OPL ◦ ILOG Concert� Have a C#, and Java Adapters� Working on C++◦ Showing Two approaches� Translating OPL� Using an Adapter for Concert

} General advice on the migrating

Agenda


Switching To Gurobi

} Hurdles◦ Large code base◦ Business Logic embedded in API calls

} Mitigating Factors◦ The time to migrate is not proportional to the size of the

codebase◦ With mps files, you can see how the Gurobi will perform on

your specific models.


Before You Migrate

} Current State◦ Actively adding features?◦ Maintenance only?◦ Do you have regression tests?

} MPS is your friend◦ Evaluate relative performance

} LP is also your friend◦ Name your variables and constraints◦ Testing your code


Migrating From OPL

} OPL includes a domain specific language

} Locked to a Solver (CPLEX)

} Two most likely strategies◦ Move to a solver-agnostic Language� AMPL� AIMMS, GAMS, MPL◦ Migrate to Python


Migrating from OPL to Python

} Why Python

◦ Best of Both worlds� Powerful as a General Purpose Language� As effective as a Domain Specific Language

� Concise� Readable� Learnable

� See Stackoverflow� (http://bit.ly/1QEF0fq)

◦ Huge user base


OPL to Python

} Feature Comparison

OPL PythonTuples, Sets Tuples, SetsRead from Excel openpyxl, xlrdRead from SQL SqlalchemySlicing, Grouping PandasUI Jupyter Notebook.dat format JSONOPLScript Python


} Installing Python◦ Use the Anaconda Distribution� Especially great on Windows� Easy installation for Gurobi Libraries

� http://www.gurobi.com/downloads/get-anaconda

OPL To Python


OPL To Python

} Migrate a complete example

} From a book ◦ Planning and Scheduling in Manufacturing and Services◦ OPL Code is freely available� http://bit.ly/20VpATz◦ Reads from OPL dat file

} Python Code Available on github


} Translate the “.dat” file to JSON

◦ JSON is a standard� Almost all programming languages have readers� OPL reads it� Native format for MongoDB� Many application have JSON exporters

◦ Simple Python script to translate� Most dat files

� without embedded logic� Using PyParsing

� Another library included with Anaconda

Approach


OPL Data

Demand = #[ 2: #[

1: [ 20000 30000 15000 40000 ] 2: [ 0 50000 30000 50000 ]

]#3: #[

1: [ 10000 5000 15000 40000 ] 2: [ 0 10000 0 5000 ] ]#

]#;

"Demand": {"2": {

"Product 1": [ 20000, 30000, 15000, 40000 ],"Product 2": [ 0, 50000, 30000, 50000 ]

},"3": {

"Product 1": [ 10000, 5000, 15000, 40000 ],"Product 2": [ 0, 10000, 0, 5000 ]

}

} Can Translate with Python Script


OPL Model

} Data DeclarationsLang. StatementOPL int RequiredLotSize = ...;

Python RequiredLotSize = d['RequiredLotSize']

OPL float Demand[Stages, Products, Periods] = ...;Python Demand = series_from_json(d['Demand'],

[Stages, Products, Periods])OPL range Products = 1..2;Python Products = ['Product 1', 'Product 2']


OPL Model

} Variable Declarations

dvar float+ x[Factories, Products, Periods];dvar int+ y[Factories, Stages, Products, Periods] in 0..maxint;dvar float+ z[Products, Periods];dvar float+ q2[Products, ZPeriods];dvar float+ v2[Products, Periods];dvar float+ v3[Products, ZPeriods];dvar boolean yb[bnds, Periods];


VariablesOPL dvar float+ x[Factories, Products, Periods];Python x = get_vars('x', Factories, Products, Periods)OPL dvar boolean yb[bnds, Periods];Python yb = get_vars('yb', bnds.index, Periods,

vtype=GRB.BINARY)


Objective function

} OPL} minimize} sum (t in Periods, j in Products, } i in Factories) } ProdCost[i,j]*x[i,j,t]

} Python} model.setObjective(} grb.quicksum([ProdCost[i, j] * x[i, j, t]} for i in Factories} for j in Products} for t in Periods])


Constraints

OPL if (RequiredLotSize > 0)forall (i in Factories, j in Products, t in Periods)

x[i,j,t] == RequiredLotSize*xlots[i,j,t];else

forall (i in Factories, j in Products, t in Periods)xlots[i,j,t] == 0;

Python if RequiredLotSize > 0:[addConstr(x[i, j, t] == RequiredLotSize * xlots[i, j, t])for i in Factories for j in Products for t in Periods]

else:[addConstr(xlots[i, j, t] == 0)for i in Factories for j in Products for t in Periods]


Results From SolverOPL float TotProdCost = sum (t in Periods,

j in Products, i in Factories)

ProdCost[i,j]*x[i,j,t];Python TotProdCost = sum([ProdCost[i, j] * x[i, j, t].x

for t in Periodsfor j in Productsfor i in Factories])


OPL to Python

} Consider moving to Python} If you still like modeling languages, try AMPL


Migration Library for CPLEX Concert API


Migrating From Concert

} Concert◦ C++ API developed in late 90s◦ Java and C# versions followed◦ Good option for using mainstream languages


Migrating from Concert

} Two Strategies◦ Translate code◦ Use an Adapter


} Pattern◦ Object Adapter Pattern

} Physical Examples◦ Electric plugs◦ CO2 Filter on Apollo 13

Adapter


Adapter Approach

} Give applications written against the Concert API, access to the Gurobi solver

} Our Adapter◦ Not a complete implementation of Concert� Most applications use a small subset◦ Enough to make a model run◦ Free to use� Starting point for your application


Adapter Approach

} Working Concert Application

} Unlink CPLEX / Concert libraries

} Add Gurobi Library

} Add adapter library◦ Modify adapter for your code


Example

} Makefile◦ CPPLIB = … -lilo_grb … -lgurobi_c++ -lgurobi65

} Eclipse


Adapter Library

} Available for Java and C#

} Working on C++

} Available on Github◦ https://github.com/abremod/concert2gurobi4cs


Adapter Library

} “Object Adapter”, not a “Class Adapter”} Favor Composition over Inheritance

◦ public class IloCplex extends ilog.concert.Algorithmimplements IloMPModeler {

◦ GRBEnv env;

◦ GRBModel model;◦ private boolean vars_synced = true;◦ private boolean constrs_synced = true;


Adapter Library

} C++ specific

◦ IloXXXX objects are handles� “Pimpl” ◦ Boost shared_ptr<> covers handle functionality◦ class IloRange : public IloExtractable {◦ private:◦ class Impl;

◦ boost::shared_ptr<Impl> _impl;◦ public:◦ IloRange(IloEnv env, double lb, double ub, const char *name=0);

◦ IloColumn operator()(IloNum);

◦ };


Limitations

} Adapter probably won’t work out of the box

} Variables tied to models, not environments◦ No “not extracted exceptions”

} Limited Support for Callbacks

} No support for Goals

So what happens next?

} Request a free evaluation license, if you have not already done so◦ [email protected]◦ [email protected]

} Set up a free consultation with a consultant from [email protected]

} Webinar slides: Will be available in the next day or two

} Abremod tools available from: https://github.com/abremod/ilogrb

} Webinar recording: Will be available next week

welcome to the webinar - gurobi · welcome to the webinar migrating models from other solvers to...

Documents