![Page 1: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/1.jpg)
1
SMU Course EMIS 8381
Nonlinear Programming
January 19, 2008
Hossam Zaki
![Page 2: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/2.jpg)
2
Outline
• Course Scope – Classification of
Optimization Problems
• Course Syllabus– Description– Pre requisites– Text & References– Calendar– Grading
• On Line Resources• Course Topics
• Course Context– DSS Example
– DSS Paradigm
– Career Roles & Questions
![Page 3: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/3.jpg)
3
Scope
![Page 4: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/4.jpg)
4
What is an Optimization Problem?
• Optimization problems involve the selection of values of a number of interrelated variables by focusing attention on one or more selection criteria designed to measure the quality of the selection
![Page 5: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/5.jpg)
5
Optimization Problem Statement
• Select the values of x
• From a set of possible values X – that satisfy a group of algebraic constraints
• In such a way that will optimize the value f(x)
![Page 6: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/6.jpg)
6
Terminology
• Decision Variables = x
• Objective Function(s) = f(x)
• Feasible Region = X– Constraints defining X
• Inequality constraints : g(x)<=0• Equality constraints : h(x)=0
![Page 7: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/7.jpg)
7
Optimization Problems Three Classifiers
Constraints Objective
Decision Variables
OptimizationProblems
![Page 8: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/8.jpg)
8
DV Attributes
Sequential
InfiniteDims
FiniteDims
Discrete
Continuous
DV
![Page 9: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/9.jpg)
9
Objective Attributes
Non Linear
Linear
NonDifferentiable
Differentiable
Closed form
From Simulation
Many
One
Objective
![Page 10: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/10.jpg)
10
Constraints Attributes
Stochastic
Deterministic
Nonlinear
Block Diagonal
Network
GUB
Simple Bounds
Linear
UnConstrained
Constraints
![Page 11: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/11.jpg)
1111
Optimization Problems
Constraints• Unconstrained• Constrained• Linear• Non linear • Simple bounds• GUB• Network• Block diagonal
Decision Variables (DV)• Discrete• Continuous• Finite • Infinite
Objective• One• Many• Differentiable• Non Differentiable• Closed Form• From Simulation• Linear• Non linear• Deterministic• Stochastic
Unconstrained
Continuous DV
Linear Constraints
One ObjDifferentiable Obj
© 2006, 2007 Zilliant, Inc. -- CONFIDENTIAL
Finite DV
Deterministic Obj
![Page 12: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/12.jpg)
12
Course Scope
• Decision Variable– Continuous– Finite dimensions
• Objective– Single– Closed form or From Simulation– Linear or Nonlinear– Deterministic
• Constraints– Deterministic
![Page 13: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/13.jpg)
13
Exercise # 1
1. Linear Programming
2. Goal Programming
3. Network Programming
4. Integer Programming
5. Non differentiable Optimization
6. Global Optimization
7. Dynamic Programming
8. Stochastic Programming
9. Quadratic Programming
10.Fractional Programming
11.Geometric Programming
12.Multi Objective Optimization
• Identify the attributes that define following optimization problems and prove one simple example. Display the results in table format
![Page 14: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/14.jpg)
14
Course Syllabus
![Page 15: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/15.jpg)
15
Course Description
• This course discusses, presents and explains the most important methods and results used to model and solve nonlinear optimization problems.
![Page 16: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/16.jpg)
16
Prerequisites
• Advanced calculus (partial derivatives)
• Linear algebra (vectors and matrices).
• Knowledge on linear programming is helpful but not required (we will cover what we need).
![Page 17: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/17.jpg)
17
Text & References
• Text Book– "Nonlinear Programming: Theory and Algorithms" by, M. Bazaraa,
H. Sherali and C Shetty, 3rd Edition, John Wiley, ISBN
• References– “Handbooks in OR & MS, Volume 1, Optimization”, Nemhauser et al
editors, North Holand, Chapters I and III, 1989
– Dennis & Schnabel, “Numerical Methods for Unconstrained Optimization”, Prentice-Hall, 1983
– Gill, Murray and Wright, “Practical Optimization”, Academic Press, 1981
– D. Luenberger, “Linear and Nonlinear Programming”, 2nd Edition, Addison Wesely, 1984
– D. Bertsekas, “Nonlinear Programming”, 2nd Edition, Athena
Scientific, 1999
![Page 18: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/18.jpg)
18
Class Calendar
• Class (Section001) – Saturday 10 AM-12:50 PM
• 10 Minute break every 50 minutes
– Meets in 203 Junkins
• 14 Class Periods– First Class Period: January 19
– No Class: March 22
– Last Class Period: April 26
![Page 19: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/19.jpg)
19
Grading
• In-class Exams (70%)– Mid Term on 3/8/ 08 = (30%)– Final on 4/26/08 = (40%)
• Term Paper & Presentation (20%)– A nonlinear optimization topic, e.g. algorithm,
application and /or software demo not covered in class
– Due on 3/29/08
• Homework (10%)
![Page 20: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/20.jpg)
20
On Line Resources
![Page 21: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/21.jpg)
21
Mathematical Programming Glossary
http://glossary.computing.society.informs.org/index.php?page=N.html • General Information - A list of dictionaries, suggested methods of
citation, and contribution instructions. • The Nature of Mathematical Programming - See this for basic terms
and a standard form of a mathematical program that is used throughout this glossary.
• Notation - Read this to clarify notation. • Supplements - A list of supplements that are cited by entries. • Myths and Counter Examples - Some common and uncommon
misconceptions. • Tours - Collections of Glossary entries for a particular subject. • Biographies - Some notes on famous mathematicians. • Please remember this is a glossary, not a survey, so no attempt is
made to cite credits.
![Page 22: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/22.jpg)
22
1998 Nonlinear Programming Software Survey
• http://www.lionhrtpub.com/orms/surveys/nlp/nlp.html
• The information in this survey was provided by the vendors in response to a questionnaire developed by Stephen Nash. The survey should not be considered as comprehensive, but rather as a representation of available NLP packages. The listings are limited to products that fit the parameters of the survey as outlined in the accompanying article.
![Page 23: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/23.jpg)
23
Nonlinear ProgrammingFrequently Asked Questions
http://www-unix.mcs.anl.gov/otc/Guide/faq/nonlinear-programming-faq.html
• Q1. "What is Nonlinear Programming?" • Q2. "What software is there for nonlinear
optimization?" • Q4. "What references are there in this
field?" • Q5. "What's available online in this field?"
![Page 24: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/24.jpg)
24
NEOS Guide
http://www-fp.mcs.anl.gov/OTC/Guide/ • The Optimization Tree. Our thumbnail sketch of
optimization (also known as numerical optimization or mathematical programming) and its various sub disciplines.
• The Optimization Software Guide. Information on software packages from the book by Moré and Wright, updated for the NEOS Guide.
• Frequently Asked Questions on Linear and Nonlinear Programming. These are the FAQs initiated by John Gregory, now maintained by Bob Fourer as part of the NEOS Guide.
![Page 25: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/25.jpg)
25
MIT Nonlinear Course
• Prof. Dimitri Bertsekas http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-252JNon-linear-ProgrammingSpring2003/CourseHome/
![Page 26: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/26.jpg)
26
UCLA Nonlinear EE Course
• Prof. Lieven Vandenberghe
http://www.ee.ucla.edu/ee236b/
![Page 27: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/27.jpg)
27
Course Topics
![Page 28: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/28.jpg)
28
Course Topics
• Chapter 1– Intro
• Appendix A– Math Review
• Chapter 2– Convex Sets
• Chapter 3– Convex Functions
• Chapter 4– Optimality Conditions
• Chapter 6– Lagragian Duality
• Chapter 8– Unconstrained Opt
• Chapter 9 – Penalty and Barrier
• Chapter 10– Methods of Feasible
Directions
![Page 29: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/29.jpg)
29
Exercise # 2
Plot and solve graphically:
Maximize x1
Subject to
h1(x1,x2) = x12 – x2 + a = 0
h2(x1,x2) = – x1 + x22 + a = 0
For the following values of a:a = 1, 0.25, 0 and -1
![Page 30: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/30.jpg)
30
Course Context
![Page 31: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/31.jpg)
31
DSS Example
Simulation Model
StochasticShow-up rates
StochasticDaily
Demand
Stochastic Success Rates
Daily Resources
FeedbackCapacity
Configuration
OperationsCharacteristics
Optimization ModelBusiness Rules
Alternative Resources
Existing Capacity
Aggregate DemandProfile
Cost of newResources
Timing Constraints
What if
![Page 32: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/32.jpg)
32
Decision Support System Development Methodology
DV = Decision VariableDB = Database
9. Solve Real Problems
2. Develop DSS High Level Design
3. Formulate Mathematical Model
DV, Objective, Constraints, Goals
5. Estimate and/or Forecast needed data, e.g. demand
6. Develop or Select Solver, prepare input
files, connect to DB
8. Refine Formulation: Aggregate, Decompose,
Transform
7. Prepare & Solve Small Sample Problems
10. Prepare & analyze output.Validate results
12. Estimate Lift (Improvement)
ITERATE
11. Perform What-if
1. Understand Business Context and Formulate Problem Statement in English
4. Receive, Clean and Synthesize Business Data.
Create DB
![Page 33: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/33.jpg)
33
How will the course help u with career questions?
P
SVP
VP
Director
Senior Manager
Manager
Senior Scientist
Scientist, OR Specialist, Consultant
Which problems to invest $ in?
Reformulate?Which solver to use?
How to formulate?
Which parameters?
What is the impact of a data change?
Which is the expected lift?
How many person-months?
How to prioritize tasks?
How to gain tech diff over competitors?
How to validate results?
Should we productize?
Develop in house or buy? How many scientists do we need?
Which algorithm?
Service or license?
![Page 34: 1 SMU Course EMIS 8381 Nonlinear Programming January 19, 2008 Hossam Zaki](https://reader035.vdocuments.site/reader035/viewer/2022062321/56649e2a5503460f94b189d6/html5/thumbnails/34.jpg)
34
Questions?