Slide 1

Engineering optimization dilemmaOptimization algorithms developed by mathematicians are normally based on linear and quadratic approximationsUsually have proofs of convergence to local optimum (Karush-Kuhn-Tucker points) Engineers often use approximations motivated by problem-specific knowledgeThey conduct sequential approximate optimizationDefine a box; approximate in the box; optimize based on approximation; move the center of the box to the approximate optimumNo easy way to determine box size, no proofs

Approximation management framework (AMF)John Dennis at Rice University developed methodology for general approximations for unconstrained problemsHis students carried work further for constrained problemsWe use paper by two of them (Natalia Alexandrov of NASA Langley and Michael Lewis of the College of William and Mary)

Trust regionFor approximations, trust region refers to where the approximation is sufficiently accurate.Some approximations (e.g. Taylor series) can be made very accurate if the region is small enough.For optimization, a key measure of the accuracy is the ratio between actual and predicted improvement in the objective.Good improvement ratio means getting the slope approximately right.Example: If range of values in box is only 5%, any approximation is likely to have small error, but not necessary improvement ratio close to 1.

ExampleWe minimize the function f=1-sinx using the (Taylor series) approximation fa=1-x, starting at x=0.If our box is |x|