non linear optimization in eeng lecture_00
DESCRIPTION
Lectures in optimizationTRANSCRIPT
-
Lecture 0 0-1
ECE733
Nonlinear Optimization for Electrical Engineers
Dr. Mohamed Bakr,
905 525 9140 x24079
-
Lecture 0 0-2
Info About Myself
B.Sc. in Electronics and Communication Engineering, Cairo
University, Cairo, Egypt with Distinction (honors), 1992
M.Sc. in Engineering Mathematics (Optimization), Cairo
University, 1996
Ph.D. in Computer Aided Design (CAD) of Microwave Circuits,
McMaster University, 2000
P.Eng., Ontario, 2003
Full professor, 2013
Author/CoAuthor of over 200 journal and conference papers,
one book, two book chapters, and two patents
-
Lecture 0 0-3
Info About Myself (Contd)
Research Areas: Optimization methods, computer-aided design
and modeling of microwave circuits, neural networks
applications, computational electromagnetics, and nanophotonics
Awards/Scholarships:
TRIO Student Internship in OSA, inc. 1997
Ontario Graduate Scholarship (OGS) 1998-2000,
NSERC PostDoctoral Fellowship 2000-2001,
Premiers Research Excellence Award (PREA) 2003-2009
McMaster Tenure 2007
Sabbatical Leave with RIM (2008-2009)
NSERC Accelerator Supplement Award (DAS), 2011
Supervisor/Co-supervisor to a number of graduate students
-
Lecture 0 0-4
Teaching Experience
Teaching Assistant in Engineering Mathematics (Cairo
University), 1992-1996
Teaching Assistant in Electrical Engineering (McMaster
University) 1996-1999
Assistant Professor in the Department of Electrical and
Computer Engineering, McMaster University 2002-2007:
ECE 750 Advanced Engineering Electromagnetics
ECE 2EI4 Electronic Devices and Circuits
ECE 3TP4 Signals and Systems
ECE 757 Numerical Techniques in Electromagnetics
ECE 2EI5 Electronic Devices and Circuits
ECE 3FI4 Theory and Applications in Electromagnetics
-
Lecture 0 0-5
Teaching Experience (Contd)
ECE 2FH3 Electromagnetics I
ECE 2CI5 Introduction To Electrical Engineering
ECE 3FK4 Electromagnetics II
ECE 4OI6 Engineering Design
ECE 718 Nonlinear Optimization
Developer of a number of coursewares for several courses
-
Lecture 0 0-6
Course Overview
1-Introduction To Vector Analysis and Optimization
Introductory mathematical tools
Historical background
Jargon of optimization problems and their classifications
-
Lecture 0 0-7
Course Overview (Contd)
2-Classical Optimization Approaches
Single-variable methods
Multi-variable methods
The KKT conditions for equality and inequality constraints
-
Lecture 0 0-8
Course Overview (Contd)
3-One Dimensional Search Techniques
Why one-dimensional search is so important?
Derivative-free methods
Gradient and Hessian based methods
-
Lecture 0 0-9
Course Overview (Contd)
4-Unconstrained Optimization
Derivative-free approaches
Gradient-based techniques
Second-order methods
-
Lecture 0 0-10
Course Overview (Contd)
5-Constrained Optimization
Quadratic programming
Sequential quadratic programming
Penalty methods
Gradient projection methods
Methods of feasible direction
-
Lecture 0 0-11
Course Overview (Contd)
6. Global Optimization Methods
Old population New population
Simulated annealing
Genetic algorithms
Particle swarm optimization
-
Lecture 0 0-12
Course Overview (Contd)
7. Space Mapping Optimization
Aggressive space mapping
Trust region space mapping
Implicit space mapping
Surrogate-based space mapping
fine
model
space
mapping
responses
surrogate
input
mapping
design
parameters responses
coarse
model
space
mappingspace
mapping
output
mapping
implicit
mapping
-
Lecture 0 0-13
Course Overview (Contd)
8. Adjoint Sensitivities and Their Applications
Using only at most one extra simulation, the sensitivities of the
response with respect to all design parameters are obtained
This makes gradient-based optimization far more efficient
Adjoint
Simulation x R R Original
Simulation
x
-
Lecture 0 0-14
Course Overview (Contd)
Text: Engineering Optimization Theory and Practice, Singiresu
S. Rao, Third Edition
or
Text: Nonlinear Optimization in Electrical Engineering with
Applications in MATLAB, Mohamed H. Bakr, IET Press, 2013
CLASSES: TBD
Course Webpage:
http://www.ece.mcmaster.ca/faculty/bakr/
ECE733/ECE733_Main_2014.htm
4 Matlab assignments and one final project are required
-
Lecture 0 0-15
Detailed Course Outline
Date Lecture Description
0 Course Outline
1 Introduction: Historical Background, statement of
optimization problem
2 Introduction: Classifications of Optimization problems
3 Classical Optimization Methods: single variable
optimization, unconstrained multivariate optimization
4 Equality Constraints: Solution by Direct substitution,
Method of constrained variation
5 Equality Constriants: Method of Lagrange multipliers
6 Inequality constraints: Kuhn-Tucker Conditions,
Constraint qualification
7
One Dimensional Search: why one dimensional search?,
Search with Fixed Step Size, Search with Accelerated Step
size
8
One Dimensional Search: Interval halving Method,
Fibonacci Method, Golden Section Search
-
Lecture 0 0-16
9 One Dimensional Search: Interpolation Methods, Newton
Method
10 One Dimensional Search: Quasi-Newton Method, Secant
Method, Practical Consideration
11 Unconstrained Nonlinear Optimization: Introduction and
basic concepts
12 Direct Search Methods: Random Walks, Grid Search,
Univariate Method, Simplex Method
13 Conjugate Gradient Methods: Powells Method, Conjugate Directions
14 Indirect Methods: Steepest Descent, Conjugate Gradients
15 2nd Order Methods: Newton Method, Marquardt Method,
and Quasi Newton Methods
16 2nd Order Methods (Contd): The DFP formula, the BFGS formula, summary
17 Constrained Nonlinear Optimization: Introduction and
basic concepts
18 Some Constrained Optimization Methods: Zoutendijks method of feasible directions
19 Constrained Optimization (Contd): Rosens Gradient
-
Lecture 0 0-17
projection Method, sequential quadratic programming
20 Constrained Optimization (Contd): Penalty Methods
21 Global Optimization Techniques: Genetic Algorithms
22 Global Optimization Techniques (Contd): Simulated
annealing
23 Global Optimization Techniques(Contd): Particle Swarm Optimization
24
Space Mapping Optimization and Modelling: Basic
Concepts, classical Space Mapping, Aggressive Space
Mapping
25 Space Mapping (Contd): surrogate-based optimization,
Output Space Mapping
26 Adjoint Variable Methods: The Frequency Domain Case
27 Adjoint Variable Methods: The Dynamic Case
28 Areas of Research in Optimization
-
Lecture 0 0-18
General Comments
Lecture is divided into two parts each for about 1.0 Hr to
1.25 Hr. We will have a break in the middle
We will not focus on theorem proving. We will give a proof
as long as it is concise and useful
Engineering Applications will be given as much as possible
We will write all our optimization code. Ready functions in
packages will only be used for comparison
Material will be posted on the course webpage the day
before. Copy only examples not in the slides.