a study of multi-objective optimization ......a study of multi-objective optimization methods for...

A STUDY OF MULTI-OBJECTIVE OPTIMIZATION METHODS FOR ENGINEERING

APPLICATIONS

by

R. Timothy Marler

A thesis submitted in partial fulfillment of the

requirements for the Doctor of Philosophy degree

in Mechanical Engineering in

the Graduate College of

The University of Iowa

May 2005

Thesis Supervisor: Professor Jasbir S. Arora

Copyright by

R. TIMOTHY MARLER

2005

All Rights Reserved

Graduate College

The University of Iowa

Iowa City, Iowa

CERTIFICATE OF APPROVAL

___________________________

PH.D. THESIS

____________

This is to certify that the Ph.D. thesis of

R. Timothy Marler

has been approved by the Examining Committee for the

thesis requirement for the Doctor of Philosophy degree in

Mechanical Engineering at the May 2005 graduation.

Thesis Committee: _______________________________________

Jasbir S. Arora, Thesis Supervisor

_______________________________________

Karim Abdel-Malek

_______________________________________

M. Asghar Bhatti

_______________________________________

Lea-Der Chen

_______________________________________

Ray P. S. Han

_______________________________________

Colby C. Swan

ii

To my heroes and friends

iii

The best laid schemes o’ mice and men

Gang aft a-gley

And leave us naught but grief and pain

For promised joy.

Robert Burns

iv

ACKNOWLEDGEMENTS

My sincere gratitude goes to my advisor, Professor Jasbir S. Arora, for his

patience and persistence in teaching me to learn and to ask the right questions, and his

insistence on clarity and quality rather than just acceptability. In addition, I thank my

committee members for their advice and direction. I am especially grateful to Professor

Karim Abdel-Malek who acted as a second mentor. I thank Ray Watkins, Dr. Carol

Devore, and Dr. Robert Marler, for their advice and lessons on writing. I greatly

appreciate Brad Parker providing valuable computer support. I thank my peers, in

particular everyone at VSR, for their help and camaraderie. Kim Farrell and Steve Beck

provided substantial contributions to the work with posture prediction. I also thank my

friends and family, all of whom essentially went to graduate school with me. Finally,

above all, I am grateful for my Mom and my Dad, and their confidence and support. This

work was supported financially by the Department of Civil and Environmental

Engineering, the Department of Mechanical and Industrial Engineering, and the Virtual

Soldier Research program.

v

ABSTRACT

Potential applications for even the most fundamental and common multi-objective

optimization (MOO) methods span a variety fields. Consequently, the significance of

any contribution to this process is far reaching, with implications in terms of

computational problem solving and in terms of conceptual decision-making. However,

the study of fundamental methods has been deficient; critical information concerning the

significance of method-parameters, the effect of formulation variations, and guidelines

for effective use has been unrevealed and thus underutilized. Drawbacks of different

methods and their variants have not been explored. We contend that new in-depth

analysis reveals useful insight into how the methods work, fosters new methods, and

provides method augmentation that improves performance. In this dissertation, we

respond to deficiencies on three levels: 1) qualitative assessment of methods, 2) analysis

of methods with consequent enhancements and guidelines, and 3) application of methods

to significant problems. First, the results of an unparalleled evaluation of approximately

60 different methods are summarized. This evaluation pertains to computational

requirements, ease of use, and solution characteristics. Conclusions are drawn

concerning: 1) individual algorithms, 2) broad classes of methods, and 3) MOO as a

whole. Then, new understanding, new approaches, and method enhancements are

presented for function-transformation methods, the weighted sum method, the global

criterion method, the min-max method, and the ε -constraint method. Special attention is

paid to the ability of methods to depict the Pareto optimal set accurately. Finally, MOO

is applied to a system identification problem for crash analysis and to an optimization-

vi

based approach for human posture prediction. These problems are used to test various

methods, and concurrently, methods are used to provide new contributions to the

problems.

vii

TABLE OF CONTENTS

LIST OF TABLES xii

LIST OF FIGURES xiv

LIST OF SYMBOLS xxi

CHAPTER

I. INTRODUCTION 1

1.1 Definition of a Multi-objective Optimization Problem 1

1.2 Overview of Fundamental Concepts 1

1.3 Motivation 2

1.4 Research Objectives 6

1.5 Overview of the Dissertation and Specific Contributions 11

II. FUNDAMENTAL CONCEPTS 15

2.1 Introduction 15

2.2 Definitions 15

2.3 Design Space and Criterion Space 18

2.4 Pareto Optimality 24

2.5 Necessary and Sufficient Conditions 28

2.6 Compromise Solution 29

III. SUMMARY OF THE LITERATURE REVIEW 32

3.1 Introduction 32

3.2 Surveys in the Literature 33

3.3 No Articulation of Preferences 34

3.4 A Priori Articulation of Preferences 34

3.5 A Posteriori Articulation of Preferences 35

3.6 Fuzzy Multi-objective Optimization 35

3.7 Genetic Multi-objective Optimization 36

3.8 Discussion and Conclusions 37

IV. WEIGHTED SUM METHOD 40

4.1 Introduction 40

4.1.1 Overview of the Formulation 41

4.1.2 Review of the Literature and Motivation 43

4.1.2.1 Approaches to Setting Weights 45

viii

4.1.3 Objectives of the Present Study 46

4.2 Interpretation of the Weights 47

4.2.1 Preferences 47

4.2.1.1 Deficiencies in the Method 49

4.2.2 Pareto Optimal Set 52

4.2.3 Objective Functions 56


V. FUNCTION TRANSFORMATIONS 61

5.1 Introduction 61



5.2 Transformation Methods 64

5.2.1 Determining Function-Maxima 66

5.3 Analysis of the Methods 68

5.3.1 Problem Statement 68

5.3.2 Unrestricted Weights 70

5.3.3 Convex Combination of Functions 73

5.3.4 New Transformation Criteria 79

5.4 Example Problems 82

5.4.1 Truss Problem 1 82

5.4.2 Truss Problem 2 87


VI. GLOBAL CRITERION APPROACH 94

6.1 Introduction 94

6.1.1 Overview of the Formulations 95


6.1.2.1 Standard Global Criterion Formulations 100

6.1.2.2 Min-max Formulations 102


6.2 Analysis of the Standard Global Criterion Formulations 104

6.3 Significance of the p-exponent 108

6.3.1 Mathematical Significance 109

6.3.1.1 Curvature 109

6.3.1.2 Approximating Min-max Results 112

6.3.1.3 Kuhn-Tucker Conditions 117

6.3.2 Preferential Significance and the Effects of

Normalization 120

6.3.3 Values of p Less than One 122

6.4 Analysis of the Min-max Formulations 123

6.4.1 Kuhn-Tucker Conditions and Regularity 125

6.4.1.1 Two Active Function-Constraints 128

6.4.2 Normalization 129

ix

6.5 Development of A Modified Global Criterion Approach 130


6.5.2 Results 133

6.5.2.1 Alternate Min-max Formulation 133

6.5.2.2 Standard Global Criterion Formulations 134

6.5.2.3 Modified Function-Normalization Scheme 138

6.5.2.4 Pareto Optimal Design Space 140


VII. WEIGHTED GLOBAL CRITERION APPROACH 146

7.1 Introduction 146

7.1.1 Overview of the Formulations 147



7.2 Analysis of the Weighted Global Criterion Formulations 153

7.2.1 Standard Global Criterion Formulations 153

7.2.2 Min-max Formulations 157

7.3 Development of A Modified Weighted Global Criterion

Approach 158


7.3.2 Results 160


VIII. E-CONSTRAINT METHOD 167


8.1.1 Overview of the Formulation 168

8.1.2 Setting the Constraint Limits 172

8.1.2.1 Primary-Objective Approach 173

8.1.2.2 Pareto-Maximum Approach 173



8.2 Analysis of the Method 176


8.2.2 Representing the Complete Pareto Optimal Set 178

8.2.3 Infeasible Problems and Duplicate Solutions 184

8.2.4 Normalization 187

8.2.5 Alternate Primary Objective 188

8.2.6 Clustering 190


IX. SYSTEM IDENTIFICATION PROBLEM FOR A SIMPLIFIED

CRASH MODEL 195


x

9.1.1 Motivation and Objectives of the Present Study 197

9.2 Overview of Dynamic System Identification Problems 198

9.3 Multi-objective Problem Statement 201

9.3.1 Design Variables and Objective Functions 202

9.3.2 Constraints 203

9.3.3 Multi-objective Optimization Formulations 204

9.4 Independent Function Minima 206

9.5 Results: No Articulation of Preferences 206

9.5.1 Error Results 207

9.5.2 Force Results 211

9.6 Normalization 215

9.7 Results: A Priori Articulation of Preferences 218

9.7.1 Comparison of Preferences 218

9.7.1.1 Error Results 219

9.7.1.2 Force Results 221

9.7.2 Comparison of Methods 223


X. OPTIMIZATION-BASED POSTURE PREDICTION 228



10.1.1.1 Motion Prediction 230

10.1.1.2 Posture Prediction 235

10.1.1.3 Multi-objective Optimization 236


10.2 Overview of the Human Model 238

10.2.1 Denavit-Hartenberg Method 241

10.3 Development of Human Performance Measures 245

10.3.1 Joint Displacement 246

10.3.2 Delta-Potential-Energy 247

10.3.3 Discomfort 251

10.4 Multi-objective Problem Statement 258

10.4.1 Posture Prediction Formulation 258

10.4.2 Multi-objective Optimization Formulations 259

10.5 Independent Function Minima 260

10.6 Results: No Articulation of Preferences 269

10.6.1 Modified Global Criterion Approach 273

10.7 Results: A Posteriori Articulation of Preferences 276

10.7.1 Study of Posture Prediction 276

10.7.2 Study of MOO Methods 284

10.7.2.1 Non-convex Portions of the Pareto Optimal Set 289


xi

XI. SUMMARY AND CONCLUSIONS 297

11.1 Summary 297

11.2 Conclusions 304

11.3 Future Work 307

REFERENCES 312

xii

LIST OF TABLES

Table

5.1 Function-Comparison Matrix for Transformation Study 70

5.2 Ratio Values 81

5.3 Function-Comparison Matrix for Truss Problem 1 84

5.4 Ratio Values for Truss Problem 1 85

5.5 Function-Comparison Matrix for Truss Problem 2 87

5.6 Ratio Values for Truss Problem 2 89

6.1 Function-Comparison Matrix for Global Criterion Study 132

6.2 Min-max Solutions for Global Criterion Study 133

8.1 ε -constraint Limits with 3

F as the Primary Objective 178

9.1 Function-Comparison Matrix for System Identification Problem 206

9.2 Error-Function Results with No Articulation of Preferences 207

9.3 Normalized Error-Function Results with No Articulation of Preferences 217

9.4 Error Results with the Weighted Global Criterion Approach 220

9.5 Error-Function Results with Weighted Methods 223

10.1 Joint Weights for Joint-Displacement 247

10.2 Joint Weights for Discomfort 255

10.3 Target Point Coordinates 261

10.4 Function-Comparison Matrices for Different Target Points 263

10.5 Performance-Measure Results with No Articulation of Preferences 271

xiii

10.6 Computational Performance with No Articulation of Preferences 271

10.7 Performance-Measure Results with the Modified Global Criterion 274

10.8 Computational Performance for the Modified Global Criterion 274

xiv

LIST OF FIGURES

Figure

2.1 Graphical Representation of a Single-Objective Optimization Problem in

the Design Space 20

2.2 Graphical Representation of a MOO Problem in the Design Space 20

2.3 Graphical Representation of a MOO Problem in the Criterion Space 21

2.4 Graphical Representation of a MOO Problem in the Criterion Space –

Alternate View 22

2.5 Illustration of Contact Theorem 26

2.6 Weakly Pareto Optimal Points 28

4.1 Pareto Optimal Set in the Criterion Space 52

5.1 Complete Pareto Optimal Set in the Design Space 69

5.2 Complete Pareto Optimal Set in the Criterion Space 69

5.3 Pareto Optima without Transformation and with Unrestricted Weights –

Criterion Space 71

5.4 Pareto Optima without Transformation and with Unrestricted Weights

(Second Version) – Criterion Space 72

5.5 Pareto Optima without Transformation – Design Space 74

5.6 Pareto Optima without Transformation – Criterion Space 74

5.7 Pareto Optima with Upper-Bound Approach and Absolute Maximum –

Design Space 76

5.8 Pareto Optima with Upper-Bound Approach and Pareto-Maximum –

Design Space 76

5.9 Pareto Optima with Lower-Bound Approach – Design Space 77

xv

5.10 Pareto Optima with Lower-Bound Approach – Criterion Space 78

5.11 Pareto Optima with Upper-Lower-Bound Approach and Pareto-Maximum

– Design Space 79

5.12 Pareto Optima with Upper-Lower-Bound Approach and Pareto-Maximum

– Criterion Space 79

5.13 Three-bar Truss 83

5.14 Truss Problem 1, Pareto Optima with Upper-Bound Approach and Pareto-

Maximum – Design Space 86

5.15 Truss Problem 1, Pareto Optima with Upper-Bound Approach and Pareto-

Maximum – Criterion Space 86

5.16 Truss Problem 2, Pareto Optima with Upper-Bound Approach – Design

Space 88

5.17 Truss Problem 2, Pareto Optima with Upper-Bound Approach – Criterion

Space 89

6.1 Global Criterion Contours in the Criterion Space, with 2p = 112



6.4 Min-max Contours in the Criterion Space 114

6.5 Global Criterion Surface for the Basic Global Criterion with 8p = 116

6.6 Global Criterion Surface for the Root Global Criterion with 8p = 116

6.7 Global Criterion Surface for the Min-max Formulation 117

6.8 Global Criterion Contours in the Criterion Space, with 0.8p = 123

6.9 Global Criterion Solutions for the Basic Global Criterion with Cases One

and Two 134

6.10 Global Criterion Solutions for the Root Global Criterion with Cases One

and Two 135

6.11 Global Criterion Solutions for the Basic Global Criterion with Case Three 135

xvi

6.12 Global Criterion Solutions for the Root Global Criterion with Case Three 136

6.13 Global Criterion Solutions for the Basic Global Criterion using a Modified

Normalization Scheme 139

6.14 Global Criterion Solutions for the Root Global Criterion using a Modified

Normalization Scheme 139

6.15 Solutions for the Basic Global Criterion - Design Space 140

6.16 Solutions for the Root Global Criterion - Design Space 141

7.1 Global Criterion Contours with =w 1 and 7p = 154

7.2 Global Criterion Surface with =w 1 and 7p = 154

7.3 Global Criterion Contours for the Weighted Exponential Approach with

( )0.3,0.7w = and 7p = 155

7.4 Global Criterion Surface for the Weighted Exponential Approach with

( )0.3,0.7w = and 7p = 155

7.5 Global Criterion Contours for the Weighted Function Approach with

( )0.3,0.7w = and 7p = 156

7.6 Global Criterion Surface for the Weighted Function Approach with

( )0.3,0.7w = and 7p = 156

7.7 Pareto Optimal Set using a Weighted Global Criterion with 7p = -

Design Space 160


Design Space 161


Design Space 161


Criterion Space 163

7.11 Pareto Optimal Set using the Weighted Min-max Approach – Design

Space 164

xvii

7.12 Pareto Optimal Set using the Weighted Min-max Approach – Criterion

Space 165

8.1 ε -constraint Method 171

8.2 Pareto-Maximum Approach - 324 Problems – Design Space 179

8.3 Pareto-Maximum Approach - 324 Problems – Criterion Space 180

8.4 Surface for 1F 181

8.5 Surface for 2

F 181

8.6 Surface for 3

F 182



8.9 Pareto-Maximum Approach - 1225 Problems – Criterion Space 184

8.10 Primary-Objective Approach – 625 Problems – Design Space 185

8.11 Primary-Objective Approach – 625 Problems – Criterion Space 186

8.12 Pareto-Maximum Approach and Normalization - 625 Problems – Design

Space 188

8.13 Pareto-Maximum Approach with 1F as the Primary Objective - 1225

Problems – Design Space 189

8.14 Pareto-Maximum Approach with 1F as the Primary Objective - 1225

Problems – Criterion Space 189

9.1 Single-Degree-of-Freedom Dynamic Model 199

9.2 Lumped Mass Representation of the Front End of a Car 201

9.3 Pareto Optimal Curve with High Trade-offs 208

9.4 Force-Time Data 212

9.5 Force-Time Results with the Objective Sum Method 212

9.6 Force-Time Results with the Min-max Method 213

xviii

9.7 Force-Time Results with the Alternate Min-max Method 213

9.8 Force-Time Results with the Alternate Min-max Method and

Normalization 218

9.9 Force-Time Results with a Weighted Global Criterion using 1

w 221

9.10 Force-Time Results with a Weighted Global Criterion using 2

w 222

9.11 Force-Time Results with the Weighted Sum 224

10.1 General Kinematic Model 239

10.2 21-DOF Kinematic Model 240

10.3 SantosTM

241

10.4 DH-Parameters 243

10.5 Reference Frame Transformation 244

10.6 Potential Energy of a Body Segment 249

10.7 Graph of Discomfort Joint-Limit Penalty Term 257

10.8 Target Points 262

10.9 Posture Prediction for Front Right Target Point, using Performance

Measures Independently 265

10.10 Posture Prediction for Front Left Target Point, using Performance


10.11 Posture Prediction for Front Lower Target Point, using Performance


10.12 Posture Prediction for Back Right Target Point, using Performance


10.13 Posture Prediction with MOO and No Articulation of Preferences 272

10.14 Posture Prediction with Weighted Sum Method - View 2 272

10.15 Posture Prediction with the Modified Global Criterion 274

xix

10.16 Pareto Optimal Set using the Modified Weighted Global Criterion Method

- Front Right Target Point 277

10.17 Posture Prediction with the Modified Global Criterion and Heavily

Weighted Discomfort 278

10.18 Pareto Optimal Set Corner using the Modified Weighted Global Criterion

Method - Front Right Target Point 280

10.19 Postures at Different Points in the Criterion Space, using the Modified

Weighted Global Criterion Method - Front Right Target Point 281

10.20 Pareto Optimal Set using the Weighted Global Criterion Method - Front

Left Target Point 283


Lower Target Point 283

10.22 Pareto Optimal Set using the Weighted Global Criterion Method – Back

Right Target Point 284

10.23 Pareto Optimal Set using the Weighted Sum Method - Front Right Target

Point 285



10.25 Pareto Optimal Set using the Weighted Min-max Method - Front Right

Target Point 286

10.26 Pareto Optimal Set using the ε -constraint Method with Discomfort as the

Primary Objective - Front Right Target Point 286

10.27 Pareto Optimal Set using the ε -constraint Method with Energy as the

Primary Objective - Front Right Target Point 287

10.28 Pareto Optimal Set using the Weighted Sum Method with Improved

Normalization - Front Right Target Point 288

10.29 Pareto Optimal Set Corner using the Weighted Sum Method with

Improved Normalization - Front Right Target Point 290

10.30 Pareto Optimal Set Corner using the Global Criterion Method - Front


xx

10.31 Pareto Optimal Set Corner using the Min-max Method with Improved

Normalization - Front Right Target Point 291

10.32 Pareto Optimal Set Corner using the ε -constraint Method with Energy as

the Primary Objective - Front Right Target Point 291

xxi

LIST OF SYMBOLS

E Euclidean space

R Real space

ix Design variables

x Vector of design variables (point in the design space)

X Feasible design space

iF Objective functions

iF

� Minimum objective function values (components of the utopia point)

max

iF Maximum objective function values

trans

iF Transformed objective functions

gF Global criterion function

F Vector of objective functions (point in the criterion space)

pF Pareto optimal point in the criterion space

F� Utopia point/positive ideal

z Aspiration point in the criterion space

U Total utility function

P Preference function

Z Feasible criterion space

jg Inequality constraints

lh Equality constraints

xxii

jq Inequality constraints in the criterion space

n Number of design variables

k Number of objective functions

ak Number of active function constraints

m Number of inequality constraints

am Number of active model constraints

e Number of equality constraints

J Jacobian matrix

iw Weighting coefficients

iµ Lagrange multipliers for inequality constraints

p Exponent for a global criterion functions

λ Min-max parameter

iε Constraint limits for ε -constraint method

DOF Number of degrees-of-freedom

FS Time dependent restoring force

iE Error functions for a restoring force

M Lumped mass

ib Expansion parameters (design variables) for system identification problem

iq Joint angle

T Transformation matrix used with DH-method

f Human performance measure

N

iq Neutral position for a joint

xxiii

L

iq Lower limit on joint angle

U

iq Upper limit on joint angle

1

CHAPTER I

INTRODUCTION

1.1 Definition of a Multi-objective Optimization Problem

The process of optimizing systematically and simultaneously a collection of

objective functions is called multi-objective optimization (MOO), and the general

problem is formulated as follows:

Find: [ ]1 2, , ,

T

nx x x=x � (1.1)

to minimize: ( ) ( ) ( ) ( )1 2, , ,

T

kF F F= F x x x x�

subject to: ( ) 0; 1,2, ,j

g j m≤ =x …

( ) 0; 1, 2, ,lh l e= =x …

where n is the number of design variables, k the number of objective functions, m the

number of inequality constraints, and e the number of equality constraints. n

E∈x is a

vector of scalar design variables ix , and ( ) k

E∈F x is a vector of scalar objective

functions ( ) 1:

n

iF E E→x . Many MOO algorithms involve the use of additional

constraints, so the original constraints indicated in (1.1) are distinguished as model

constraints. The feasible design space is defined as

( ) ( ){ }0, 1, 2,..., ; and 0, 1, 2, ...,j i

g j m h i e= ≤ = = =X x x x .

1.2 Overview of Fundamental Concepts

An overview of fundamental concepts that are discussed in detail later is given

here to provide a foundation for the motivation and objectives of this work. With MOO,

2

an improvement in one objective often results in detriment to another. Consequently, the

idea of a solution is less straightforward than it is with single-objective optimization. The

predominant solution concept is Pareto optimality, and a point is Pareto optimal if and

only if it is impossible to move from that point and reduce at least one objective function

without increasing (i.e., detrimentally affecting) any other function.

There are typically infinitely many Pareto optimal points for a problem, and to

settle on one point requires the decision-maker to somehow articulate preferences.

Methods with a priori articulation of preferences require the user to specify preferences

in terms of the relative importance of the objectives or in terms of goals, before the

optimization algorithm runs. A posteriori articulation of preferences involves selecting a

solution from a palette of possible solutions, presumably Pareto optimal solutions, after

the algorithm runs.

Approaches to MOO that entail combining all of the objective functions into a

single scalar function, are called scalarization methods. When such approaches are used

with a priori articulation of preferences, preferences are modeled as components of the

scalar function, such as weights in a weighted sum or an exponent in a p-norm. These

components are called method-parameters.

1.3 Motivation

The hypothesis for this research is that the study and evaluation of fundamental

MOO methods has been largely deficient; critical information concerning the significance

of method-parameters, the effect of formulation variations, and guidelines for effective

use of methods in general has been unavailable and unpublished. Although they have

been discussed minimally in the literature, fundamental methods still provide fertile

3

ground for significant research. Drawbacks of different methods and method-

modifications have not been explored. Deficiencies in the literature, such as those

detailed by Marler and Arora (2003) and summarized in Chapter 3, suggest that

researchers have neglected theoretical and practical analysis of methods as well as

significant applications. We contend that new in-depth analysis can reveal unprecedented

insight into how methods work, can foster new methods and methodologies, and can

contribute significantly to practical engineering problems.

The specific motivation for this research is detailed as follows:

1) Although a substantial number of MOO methods have accumulated over the past

twenty-five years, compilations of these methods are incomplete in terms of

comprehensive coverage for engineering problems. Specific methods and different

classes of methods have not been studied in terms of their computational cost, ease of

use, and capacity for providing acceptable (Pareto optimal) solutions. This leaves the

task of assessing methods and selecting a single method for a particular problem

intimidating and sometimes random. Furthermore, the current methods need to be

consolidated with a unifying typology and vernacular. Because of the relative youth

of MOO and the potential for its application to an infinite range of problems, there

has been a duplication of effort on two fronts. First, algorithm development in

different fields breeds unique terms associated with common ideas. Second, many

methods are discussed or referenced as being unique when in fact they are derivatives

of other, more general approaches.

2) Little attention has been paid to how one should interpret and use various method-

parameters to maximize the effectiveness of a specific approach, avoiding potential

4

pitfalls in performance. There is insufficient analysis of the mathematical and

physical significance of the parameters. In addition, the relationship between

different types of parameters incorporated in a single formulation is not well

understood. Consequently, it is unclear how to use many of the most fundamental

methods effectively.

3) The computational differences between alternative formulations for a single approach

have not been scrutinized. In fact, they are often erroneously considered equivalent.

Similarly, the effects of different function-transformation schemes, which constitute

variations on a formulation and that can have a significant effect on performance,

have not been studied.

4) How well consistent variation in parameters yields an even spread of solution points

that accurately depict the complete Pareto set has received insufficient research

attention. In this case, even implies a consistent Euclidean distance between

consecutive solution points and is significant in that an even spread of points more

thoroughly represents the Pareto optimal set, which can be used for a posteriori

articulation of preferences. A recent increase in computer speed and capability has

fostered an interest in depicting the Pareto optimal set for a posteriori articulation of

preferences, but the basic, relatively simple methods have not been evaluated in this

capacity.

5) Given substantial theoretical analysis of methods, it is necessary to consider

significant problems, suitable for evaluating different MOO methods. However,

example problems in the literature tend to be limited with few design variables and

objective functions. When MOO is in fact used with substantial applications, the

5

results are not scrutinized with respect to MOO theory; MOO is not fully leveraged.

Thus, the full implications of the results are rarely appreciated. In addition, applying

MOO to problems previously solved with single-objective optimization can yield

substantial benefits.

6) In response to the above-mentioned deficiency, two complex practical problems are

considered. The application of MOO to these problems provides three benefits: 1)

verification of method analysis and development, 2) new information concerning

MOO methods, and 2) new contributions to the problems themselves. First, a

simplified dynamic automotive model is considered. Dynamic automotive crash

simulations are often highly nonlinear, and the analysis of a model of an entire

vehicle can be expensive, computationally. Consequently, a simplified, relatively

fast, analysis model can provide useful upstream design information. One approach

to developing such a model entails representing the vehicle with a lumped-mass

system in which connections between the masses are represented by restoring forces

that are approximated with a series of basis functions and associated coefficients.

Kim et al (2001) and Kim and Arora (2003a) describe a method in which one

minimizes the error between the approximate restoring forces and actual force-data,

in order to determine optimal values for the coefficients and a consequent. The

problem of determining components of an approximation model that minimize error

is called a system identification problem, and although such problems have received

substantial attention, the use of different MOO methods has not been explored.

Evaluating different potential solutions and user-preferences with regard to the

6

different error functions (one for each restoring force) can provide additional insight

into the problem.

7) The second application involves the development of virtual humans. The demand for

realistic autonomous virtual humans is increasing, with potential application to

prototype design and analysis, for a reduction in design cycle time and cost. In

addition, virtual humans that function independently, without input from a user or a

database of animations, provide a convenient tool for biomechanical studies.

However, development of such avatars is limited. Recently, an optimization-based

approach has been developed (Riffard and Chedmail, 1996; Yu, 2001; Mi et al,

(2002b). This approach entails optimizing objective functions that represent human

performance measures, such as joint displacement, discomfort, etc. These

performance measures govern how the avatar moves. The optimization-based

approach ensures autonomous movement regardless of the scenario, and it can be

implemented in real time. However, despite its advantages, development of the

optimization-based approach is in its infancy, used primarily with robotics. The

advantages of MOO have not yet been fully exploited. MOO can be used to develop

new human performance measures and to combine the different measures in order to

model more accurately, how humans move.

1.4 Research Objectives

In response to the motivation discussed above, the primary goals of this work are

to scrutinize, enhance, and create MOO methods and methodologies, to provide new

understanding of the advantages and disadvantages of the methods, to reveal new

significant characteristics of the methods, and to discover new aspects of and solutions to

7

complex engineering problems. In general, we increase the often-unrealized

effectiveness of common MOO methods. We investigate the physical and mathematical

significance of method-parameters. Various formulations for approaches are studied, and

advantageous modifications are introduced. New approaches are developed. The ability

of methods to yield a Pareto optimal solution consistently and to provide an accurate

representation of the Pareto optimal set is investigated, and techniques for maximizing

effectiveness in this capacity are presented.

The following three primary facets of the research objectives are discussed: 1) an

unprecedented qualitative study of methods, 2) a detailed analysis and extension of

methods, and 3) a new application of methods to a system identification problem and a

human posture-prediction model. Specific objectives corresponding to these facets are

given as follows, and details concerning what is contributed with this dissertation are

listed in the next section:

1) Study the ease-of-use, computational cost, and solution characteristics of methods and

classes of methods. Provide a comprehensive qualitative examination of continuous,

nonlinear MOO concepts, methods, and literature with an eye towards engineering

applications, thus responding to current deficiencies in the literature. Consolidate

seemingly different concepts, methods, and terminology, all stemming from varied

applications and a duplication of effort in the current literature.

2) Examine the following MOO methods in detail and use the resulting studies to

provide new directions and guidelines for improved performance:

a) Weighted sum

b) Weighted global criterion

8

c) Weighted min-max

d) ε-constraint

Analyze these methods with regards to the following issues:

a) Physical and mathematical significance of method-parameters

b) Effectiveness of different formulations for the same general approach

c) Effect of various function transformations

d) Effectiveness in providing an evenly spaced set of solution points that

accurately represents the complete Pareto optimal set

e) Inability to guarantee a Pareto optimal solution and/or inability to provide

all Pareto optimal points with consistent variation in method-parameters

f) Potential numerical difficulties

3a) Demonstrate the importance of the work discussed above by applying various MOO

methods to the identification of a simplified model for a multiple-degree-of-freedom

crash simulation. Use MOO with no articulation of preferences to provide

benchmark results and to study and resolve previously unseen computational issues

that arise with standard normalization techniques. Use methods with a priori

articulation of preferences to study the manifestation of preferences and to contrast

the computational performance of different methods. Reveal and take advantage of

the special form of the Pareto optimal set in order to determine which method(s) is

(are) most effective for further analyses of the problem.

3b) Use MOO to develop novel human performance measures for a human posture-

prediction model. Use the above-mentioned methods to aggregate multiple

performance measures. Again, use methods with no articulation of preferences to

9

provide benchmark results. Resolve issues concerning normalization. Demonstrate

the advantages of the new modified global criterion method. Study the different

methods in terms of their ability to depict Pareto optimal sets and in terms of their

ability to capture points on non-convex portions of the sets. Determine how the new

performance measures, discomfort and delta-potential-energy, should be aggregated

to yield the most realistic results.

An explanation of why the particular methods in 2) are discussed is warranted. In

general, these are fundamental easy-to-use approaches, and this study is a natural

extension of the qualitative evaluation outlined in 1). Deficiencies discovered in these

approaches and their variants are addressed.

The weighted sum often serves as the default approach to MOO. However, while

some attention has been paid to identifying practical difficulties, inherent deficiencies

that cause these difficulties have not been explored. In addition, there is no indication as

to how one might minimize the effect of pitfalls and maximize the effectiveness of the

method. The difference between selecting weights with this method for a priori

articulation and for a posteriori articulation is critical but has not been discussed in the

literature.

The weighted global criterion approach is the most general scalarization approach

and yields other common methods as special cases. However, literature that incorporates

this approach tends to focus on applications rather than on the nature of the approach.

This approach involves three different types of method-parameters. In addition, it

involves a variety of potential formulation-variations with distinctly different sets of

advantages and disadvantages. However, the relationships between the parameters have

10

not yet been explored, and the various formulations have not been contrasted. This

leaves the user with a potentially confusing set of options that needs to be clarified.

Thorough understanding of the weighted criterion method provides a unified

understanding of many other scalarization methods.

The weighted min-max method is a special case of the weighted global criterion

method and is often used as an alternative to or a comparison for the weighted sum

method. However, the most common formulation for this min-max method requires

additional constraints and an additional design variable, and it may lend itself to

irregularity. Conditions for such irregularity have not been investigated. This

formulation is especially susceptible to the effects of function transformation or the lack

thereof, but this condition has not yet been discussed. Furthermore, it may be possible to

use the standard global criterion method with a large exponent to approximate the results

of the min-max method. However, this approach has not been presented in the literature,

and subtle but significant modifications to the formulation are presented in this work that

help avoid computational difficulties.

Finally, the ε-constraint method is a common approach with a format slightly

different from the above-mentioned scalarization methods. The decision-maker sets

limits for all but one of the objective functions rather than setting weights that

presumably indicate the importance of each objective function. Then, one function is

selected as the primary function and is minimized subject to the limits. However,

improperly setting the limits can result in an infeasible problem and/or duplicate solution

points. These limits have not been viewed as mathematical method-parameters, and their

significance in such a capacity has not been addressed. The mathematical significance of

11

using a particular function as the primary objective is also unclear. To date, the method

has been used primarily with bi-criterion problems, and its effectiveness with more

general problems has not been evaluated. Studying this method can provide decision-

makers with information enabling them to set limits with less arbitrariness, possibly

allowing users to depict more accurately the Pareto optimal set with consistent variation

in the parameters.

1.5 Overview of the Dissertation and Specific Contributions

As a foundation for the various studies in this dissertation, first, MOO concepts

are reviewed in Chapter 2. Then, Chapter 3 summarizes the results of a comprehensive

review of the literature and qualitative study of methods for continuous, nonlinear MOO.

Deficiencies in the literature that have motivated this work are highlighted.

Chapters 4 through 8 involve detailed evaluations and extensions of key MOO

methods. Chapter 4 provides an analysis of the weighted sum method as a test bed for

different function-transformation schemes. The significance of the weights is determined

and scrutinized on a fundamental level in terms of preferences, the Pareto optimal set,

and objective function values. We discover which factors govern what solution point

results from a particular set of weights. Inherent unavoidable deficiencies in the method,

which were previously unidentified, are discussed, and unifying guidelines are provided

for maximizing the effectiveness of the weights in terms of a priori articulation of

preferences. In Chapter 5, we present a unique study of transformation methods in terms

of potential numerical difficulties and in terms of imposed limits on function-values.

Methods are tested computationally with respect to their ability to generate an accurate

representation of the Pareto optimal set. New criteria are provided to predict the

12

performance of different methods. It is shown that one method for normalization

consistently outperforms the other function-transformation approaches. Chapter 6

discusses the global criterion method in detail. Potential confusion surrounding the

concurrent use of various method-parameters and formulation-variations is resolved. We

present new analysis of potential formulations. The physical and mathematical

significance of the exponent parameter and the effects of function transformation are

investigated. We introduce new conditions for irregularity and unique benefits of

normalization with the min-max formulation, which is a special form of the global

criterion. A modified global criterion is proposed and tested. This new criterion

alleviates computational difficulties while approximating the results obtained with the

min-max method but without the additional constraints or design variables. This method

also allows the user to alter the exponent continuously to represent preferences more

effectively. Chapter 7 is an extension of Chapter 6 and provides a study of the weighted

global criterion and the weighted min-max methods. Two approaches for incorporating

weights, which are often erroneously considered synonymous, are assessed, and the

effectiveness of each approach in depicting the Pareto optimal set is evaluated. The

modified global criterion is expanded to incorporate weights. Chapter 8 presents a

singular study of the ε -constraint method in the context of problems with more than just

two objectives. Two methods for setting method-parameters are contrasted analytically

and experimentally. The potential occurrence of infeasible problems and duplicate

solution points is demonstrated and studied. In addition, we find no added utility when

normalization is used with this method, and we discover the potential tendency of

solution points to cluster around the minimum of the primary objective function. The

13

results in this section are compared with those obtained using weighted methods.

Although Chapters 4 through 7 involve illustrative example problems, these problems are

not intended to represent significant applications. Rather, they are used as analysis tools

for evaluating and studying methods.

Alternatively, Chapters 9 and 10 each involve complex applications for the

methods discussed above. Findings from previous chapters are implemented and

evaluated further in the context of significant practical problems. With these problems,

MOO is used not only to determine solutions but also to analyze solutions and extract

useful information providing additional problem insight.

Chapter 9 discusses the application of MOO methods to a system identification

problem for a simplified crash model involving a three degree-of-freedom dynamic

system with six restoring forces (and six consequent objective functions) and 126 design

variables. Methods are used with no articulation of preferences and with a priori

articulation. Results are presented and discussed in terms of error-function values and

time histories for each restoring force. A special form for the Pareto optimal set is

identified and exploited, and it is shown that articulating preferences is not advantageous

with this problem. Special computational issues concerning normalization, weight-

values, and the min-max formulations are discussed.

In Chapter 10, various methods are used for the development of a thirty degree-

of-freedom human posture prediction model, with which unknown joint angles provide

30 design variables. MOO is used in two capacities: to develop two new human

performance measures, which act as objective functions and govern posture, and to

combine multiple performance measures. With respect to combining performance

14

measures, MOO methods are used with no articulation of preferences and with a

posteriori articulation. Results are given in terms of performance-measure values and in

terms of posture illustrations. The modified weighted global criterion is tested, and

additional benefits are identified. By studying the Pareto optimal sets, we determine the

most advantageous approach to combining the newly developed performance measures.

Finally, in Chapter 11, the results of this dissertation are summarized, significant

findings are reiterated that relate directly to the research hypothesis, and future research is

discussed.

15

CHAPTER II

FUNDAMENTAL CONCEPTS

2.1 Introduction

Much of the fundamental theory concerning multi-objective optimization (MOO)

veers from the more familiar paradigms of single-objective optimization. Thus, this

chapter addresses basic concepts that are necessary for method analysis and development.

The topics of section 1.2 are extended. First, general ideas concerning preferences and

method-types are explained. Then, the two vector spaces that are used to represent MOO

problems are illustrated. Pareto optimality is discussed in detail, and variations on this

concept are defined. Finally, compromise programming is presented as an alternative

optimality concept.

2.2 Definitions

MOO originally grew out of three areas: economic equilibrium and welfare

theories, game theory, and pure mathematics. Consequently, many terms and

fundamental ideas stem from these fields, and the reader is referred to Marler and Arora

(2003) for in-depth discussions of these topics. For the sake of brevity, only critical

terms are defined below. Many of these terms have multiple definitions in the literature

stemming from the differences between engineering and economic jargon, and in such

cases, the most common and most appropriate definitions are used.

Preferences refer to a decision-maker’s opinions concerning points in the criterion

space, and a fundamental goal of MOO is to mathematically model the decision-maker’s

preferences. Therefore, it makes sense to involve the decision-maker in the optimization

16

algorithm. Accordingly, methods can be categorized depending on how the decision-

maker articulates or incorporates preferences.

A priori articulation of preferences implies that the user indicates the relative

significance of the objective functions or indicates desired goals before running the

optimization algorithm; the user quantifies opinions before actually viewing points in the

criterion space. In this sense, the term preference is often used in relation to the

importance of different objective. However, this articulation of preferences is

fundamentally based on opinions concerning anticipated points in the criterion space.

A posteriori articulation of preferences entails selecting a solution from a group

of mathematically equivalent solutions after the algorithm has run; the decision-maker

imposes preferences directly on a set of potential solution points.

Progressive articulation of preferences requires that the decision-maker

continually provide input during the running of an algorithm. Using this approach can be

relatively efficient (in terms of computational effort), since is strives to produce only a

subset of the complete set of potential solutions, governed by the user’s preferences

(Koski, 1984). However, whereas such approaches accurately reflect the user’s

intentions, they cannot function independently; the user must attend to the algorithm

while it runs. Consequently, such methods are not well suited to repetitive use, and are

not considered in this study.

A preference function ( )P F x is an abstract function (of points in the criterion

space) in the mind of the decision-maker, which perfectly incorporates his/her

preferences. With methods that involve a priori articulation of preferences, the physical

significance of each objective provides input to the preference function. Then, the

17

decision-maker provides preference-function output in the form of method-parameters,

which are coefficients, exponents, factors, etc. that can either be set to reflect decision-

maker preferences, or continuously altered in an effort to represent the complete Pareto

optimal set. Alternatively, the decision-maker may provide output in the form of goals,

or an ordering of the objectives. However, once this information is integrated into a

multi-objective formulation, the final solution may not be acceptable. On the other hand,

with methods that involve a posteriori articulation of preferences, the decision-maker

imposes preferences directly on a set of potential solution points (in the design space or

in the criterion space). Thus, theoretically, the final solution accurately reflects the

decision-maker’s preferences.

In the context of economics, utility, which is modeled with a utility function,

represents an individual’s or group’s degree of contentment (Mansfield, 1985). This

usage of “utility” is slightly different from its usual meaning as usefulness or having

worth. Instead, in this case utility emphasizes a decision-maker’s satisfaction. In terms

of MOO, an individual utility function is defined for each objective and represents the

relative importance of each objective in terms of its ability to represent preferences. The

utility function 1:

kU E E→ is an amalgamation of the individual utility functions and is

an explicit mathematical expression that attempts to model the decision-maker’s

preferences. It is used to approximate the preference function, which typically cannot be

expressed in mathematical form.

A global criterion is a scalar function that mathematically combines multiple

objective functions; it does not necessarily involve utility or preference.

18

One of the predominant classifications of multi-objective approaches is

scalarization methods and vector optimization methods. With scalarization methods,

given a vector of objective functions, the components of this vector are combined to form

a single objective function. Then, one can use standard single-objective methods to

optimize the resulting scalar function. Alternatively, although few authors make the

distinction, the term vector optimization loosely implies independent treatment of each

objective function. Both approaches are discussed.

Most approaches to MOO involve a reformulation of the problem in (1.1).

Different algorithms simply respond in different ways to the question of how to

incorporate multiple objective functions. However, no matter how one prioritizes or

combines the multiple objective functions, ultimately one usually depends on a single-

objective, constrained optimization routine to solve the MOO problem. This single-

objective optimization approach is termed the optimization engine. Different

optimization engines are not evaluated, only the various methods for handling multiple

objective functions are.

2.3 Design Space and Criterion Space

From a classical standpoint, optimizing a single function simply entails

determining a set of stationary points, identifying a local maximum or minimum, and

possibly finding the global optimum. In contrast, the process of determining a solution

for a MOO problem is slightly more complex and less definite than that for a single-

objective problem. In fact, a solution to a multi-objective problem is more a concept than

a definition. It is not intuitively clear what is meant by the minimum of multiple

functions that may have opposing characteristics, since what decreases the value of one

19

of the functions may increase the value of another. In an effort to better understand the

difficulty in defining an optimum for a multi-objective problem, it can be illustrative to

consider a simple graphical problem.

To begin with, consider the basic, single-objective minimization problem

formulated as follows and illustrated in Figure 2.1:

Find: [ ]1 2,

T

x x=x (2.1)

to minimize: ( ) ( ) ( )2 2

1 23 5F x x= − + −x

subject to: 1 2 1

90 10 9 0g x x= − − ≤

2 2 13 2 9 0g x x= − − ≤

Figure 2.1 shows the optimization problem in the design space. That is, the constraints,

1g and

2g , and the objective function contours are shown in terms of the design variables

1x and

2x . Note that the problem has a distinct minimum at Point A (3.8298, 5.5532)

with an objective function value of 0.9946.

A MOO problem can be represented in similar terms, as shown in Figure 2.2, for

the following two-objective problem:

Find: [ ]1 2,

T

x x=x (2.2)

to minimize: ( ) ( ) ( )2 2

1 1 23 5F x x= − + −x

( ) ( ) ( )2 2

2 1 25 7F x x= − + −x

subject to: 1 2 1

90 10 9 0g x x= − − ≤

2 2 13 2 9 0g x x= − − ≤

20

x1

g2

0 2 4 6 8 10 0

2

4

6

8

10

Point A

x2

g1

F=2.0

F=3.5

F=5.0 F=7.2

F=0.5

X

Figure 2.1: Graphical Representation of a Single-Objective Optimization Problem in the

Design Space

x1

g2

0 2 4 6 8 100

2

4

6

8

10

Point A

x2

g1

F1=2.0

F1=3.5

F1=5.0F1=7.2

F1=0.5

X Point B

Point C

Point D

Point E

F2=2.0

F2=3.5F2=5.0

F2=7.2

F2=0.5

Figure 2.2: Graphical Representation of a MOO Problem in the Design Space

21

In Figure 2.2, Point A represents the minimum for 1F , and Point B (5.3077, 6.5385)

represents the minimum for 2

F

with an objective function value of 0.3077. However, if

one wishes to minimize 1F and

2F simultaneously, then pinpointing an optimum point is

not straightforward, hence, the difficulty of defining a solution for MOO. Other points in

Figure 2.2 are referenced later in this section.

A MOO problem may also be depicted in the criterion space Z, where the axes

represent the different objective functions, as shown in Figures 2.3 and 2.4.

10 20 30 40 50 60 70 F1

20

30

40

50

60

70

F2

Z

q1

q2

10

q1

q2

Figure 2.3: Graphical Representation of a MOO Problem in the Criterion Space

22

1 2 3 4 5 6 7 8

1

2

3

4

5

6

7

8

F2

Z

q1

q2

Pareto Optimal Set

Utopia Point

Point A

Point B

Point C

F2

Point D

q1

q2

Figure 2.4: Graphical Representation of a MOO Problem in the Criterion Space –

Alternate View

1q represents

1g , and

2q represents

2g . In general, a curve in the design space, in the

form ( ) 0j

g =x , is mapped onto a curve in the criterion space, j

q , as follows:

( ) ( ){ }0j jq gF x x =� (2.3)

Although j

q represent the mapping of j

g onto the criterion space, they do not

necessarily represent the boundaries of the feasible criterion space (also called the

attainable set), which is defined as { }( )Z F x x X= ∈ . The inequality portion of a

constraint does not translate directly between the feasible design space and the feasible

criterion space. This is seen in Figure 2.3, where all portions of the curves 1q and

2q do

not necessarily form boundaries of the feasible criterion space. For instance, in Figure

23

2.2, Point C is infeasible, but in Figure 2.4, this point falls in between the two parts of the

curve 1q . Each ∈x X may be represented in k

E by a point with coordinates

( ) ( ) ( )1 2

, , ,k

F F Fx x x� . Thus, the set of points defined by Z is the image of X in the

criterion space. Although Z is indicated in Figures 2.3 and 2.4, which depict a relatively

simple problem, generally, an explicit mapping of the feasible design space to the

feasible criterion space is not readily available.

Although the terms feasible criterion space and attainable set are each used in the

literature to describe Z, there is a subtle distinction between the ideas of feasibility and

attainability. Feasibility implies that no constraint is violated. Attainability implies that a

point in the criterion space exists in or maps to a point in the design space. Each point in

the design space maps to a point in the criterion space, but the reverse may not be true;

every point in the criterion space does not necessarily correspond to a single point ∈x X .

Consequently, even with an unconstrained problem, only certain points in the criterion

space are attainable. In this study, Z is used to indicate points in the criterion space that

are feasible and attainable.

Note that for a given point in the feasible criterion space, there can be feasible or

infeasible corresponding points in the design space. For instance, Points D and E in

Figure 2.2 both result in a value for F1 of 7.2 and a value for F2 of 3.5. Consequently,

although Point D is feasible and point E is infeasible, both points correspond to Point D

in Figure 2.4.

The products of an optimization problem, in terms of practical application, are

usually the optimum values of the design variables, not the objective function values.

Consequently, an engineer or decision-maker typically is interested in results in terms of

24

the design space. However, decisions usually are made in terms of the criterion space,

based on the values of different objectives. Diagrams of the criterion space are

particularly helpful when multiple objectives are combined into a single, composite

objective. The composite objective can then be graphed as a function of the initial

objectives. Studying the criterion space is most beneficial when there are a relatively

small number of objective functions, regardless of the number of design variables.

Alternatively, depicting a problem in the design space can be advantageous from a visual

perspective when the number of objective functions is relatively large.

2.4 Pareto Optimality

In contrast to single-objective optimization, a solution to a multi-objective

problem is more a concept than a definition. Typically, there is no single, global

solution, and often, it is necessary to determine a set of points that all fit a predetermined

definition for an optimum. The predominant concept in defining an optimal point is that

of Pareto optimality (Pareto, 1906). Many provide a formal definition for a Pareto

optimal point in terms of the design space as follows (Vincent and Grantham, 1981;

Eschenauer et al, 1990; Miettinen, 1999; and others):

Definition 2.1 Pareto Optimal Point: A point, *∈x X , is Pareto optimal if

and only if there does not exist another point, ∈x X , such that

( ) ( )*≤F x F x , with at least one ( ) ( )*i iF F<x x .

Any comparison ( , , )etc≤ ≥ between vectors applies to each pair of vector components.

A point is Pareto optimal if there is no other point that improves at least one objective

25

function without detriment to any other function. In other words, there is no way to

improve upon a Pareto optimal point without increasing the value of at least one of the

objective functions. Typically, there are infinitely many possible optimum points in a

MOO problem, and the set of all Pareto optimal solutions is called the Pareto optimal set.

Vincent and Grantham (1981) provide the following common theorem in terms of

the criterion space:

Theorem 2.1 Contact Theorem: A point, *∈F Z , is Pareto optimal if and

only if ( ) { }* *∩ϒ =Z F F , where ( ) { }* *k

Eϒ ≡ ∈ = + ≤F F F F δ,δ 0 .

In order to simplify the notation, ( )≡F F x and ( )* *≡F F x . As demonstrated in Figure

2.5, Theorem 2.1 suggests that if a point *F in the criterion space is Pareto optimal, then

it is the only point in the set ( )*ϒ F that is also in the feasible criterion space Z. One can

think of the set ( )*ϒ F as a portion of a hypercube, with *F located at the corner. The

only way to improve upon (simultaneously reduce all components of) a Pareto optimal

point in the criterion space is to violate constraints. Similarly, the theorem implies that

all Pareto optimal points lie on the boundary of the feasible criterion space Z, a

characteristic that is confirmed in the literature and illustrated in Figures 2.3 and 2.4 (Lin,

1976; Athan and Papalambros, 1996; Chen et al, 2000).

26

1 2 3 4 5 6 7

1

2

3

4

5

6

7

F1

F2

Z

*F

( )*ϒ F

Figure 2.5: Illustration of Contact Theorem

All Pareto optimal points may be categorized as being either proper or improper.

The idea of proper Pareto optimality (Geoffrion, 1968) and its relevance to certain

algorithms is discussed by Yu (1985); Steuer (1989); and Miettinen (1999) and is defined

as follows:

Definition 2.2 Properly Pareto Optimal: A point, *∈x X , is properly

Pareto optimal (in the sense of Geoffrion, 1968) if it is Pareto optimal and

there is some real number 0M > such that for each ( )iF x and each

∈x X satisfying ( ) ( )*i iF F<x x , there exists at least one ( )j

F x such that

( ) ( )*j j

F F<x x and ( ) ( )

( ) ( )

*

*

i i

j j

F FM

F F

−≤

−

x x

x x

. If a Pareto optimal point is

not proper, it is improper.

27

The quotient is referred to as a trade-off, and it represents the increment in objective

function-j resulting from a decrement in objective function-i. Definition 2.2 requires that

the trade-off between each function and at least one other function be bounded.

Some algorithms provide solutions that may not be Pareto optimal but may satisfy

other criteria, making them significant for practical applications. For instance, weakly

Pareto optimal is defined as follows (Koski, 1984; Ruiz-Canales and Rufian-Lizana,

1995; Luc and Schaible, 1997; Miettinen, 1999; and others):

Definition 2.3 Weakly Pareto Optimal Point: A point, *∈x X , is weakly

Pareto optimal if and only if there does not exist another point, ∈x X ,

such that ( ) ( )*<F x F x .

Figure 2.6 illustrates a conceptual example of a Pareto optimal set and weakly Pareto

optimal points. The Pareto optimal set and the line segment AB represent the boundary

of Z. If a point is weakly Pareto optimal, such as those points on line segment AB, then

there is no other point that improves all of the objective functions. It may be possible to

move from a point and improve some objective functions while other objective functions

remain constant. In contrast, a point is Pareto optimal if there is no other point that

improves at least one objective function without detriment to another function. Pareto

optimal points are weakly Pareto optimal, but weakly Pareto optimal points are not Pareto

optimal. Weak Pareto optimality is not as strict a condition as standard Pareto optimality;

it is a necessary condition for Pareto optimality.

28

1 2 3 4 5 6 7

1

2

3

4

5

6

7

F1

F2

A B

Pareto Optimal

Set

Z

Figure 2.6: Weakly Pareto Optimal Points

2.5 Necessary and Sufficient Conditions

Central to the performance of a particular MOO formulation is whether solving it

serves as a necessary and/or sufficient condition for Pareto optimality. However, these

characterizations may veer slightly from their meaning in terms of single-objective

optimization. If a formulation provides a necessary condition, then for a point to be

Pareto optimal, it must be a solution to that formulation. Consequently, the method can

yield every Pareto optimal point with adjustments in method-parameters. If a point is

obtainable as a solution for a particular formulation, the formulation is said to capture

that point. However, some solutions to a formulation that provides a necessary condition

may not be Pareto optimal.

On the other hand, if a formulation provides a sufficient condition, then its

solution is always Pareto optimal, although it may not be able to capture certain Pareto

optimal points. Many authors discuss theoretical necessary and sufficient conditions as a

29

means of qualifying Pareto optimality, and such conditions are discussed in Marler and

Arora (2003). However, in this dissertation, the terms necessary and sufficient are used

in the more practical sense to describe the ability of a method to provide Pareto optimal

solutions.

In terms of a global criterion ( )gF F , Stadler (1988) presents the following

practical sufficiency condition for a Pareto optimal point, and it is useful for evaluating

the effectiveness of scalarization methods:

Theorem 2.2 Let ∈F Z , *∈x X , and ( )* *≡F F x . Let ( ) 1: R

gF →F Z

be a scalar global criterion that is differentiable with

( ) g

F∇ > ∀ ∈F

F 0 F Z . Assume ( ) ( ){ }* ming g

F F= ∈F F F Z . Then,

*x is Pareto optimal.

Theorem 2.2 suggests that minimizing a global function ( )gF F is sufficient for Pareto

optimality if ( )gF F increases monotonically with respect to each objective.

Furthermore, if *x is a Pareto optimal point, then there exists a function ( )gF F that

satisfies Theorem 2.2 and captures *x (Messac et al, 2000a).

2.6 Compromise Solution

An alternative to the idea of Pareto optimality is the idea of a compromise

solution (Salukvadze, 1971a, 1971b; Yu, 1973; Zeleny, 1973). This concept provides a

single solution point rather than a set of points. It entails minimizing the difference

30

between the potential optimal point and what is called the utopia point. The utopia point

is defined by Vincent and Grantham (1981) as follows:

Definition 2.4 Utopia Point: A point kE∈F

� , in the criterion space, is a

utopia point if and only if for each 1,2, ,i k= … ,

( ){ }minimumi i

F F= ∈

x

x x X� .

In general, F� is unattainable; it is rarely possible to fully optimize each individual

objective function independently and simultaneously, whether the problem is constrained

or not. The utopia point for the problem in (2.2) is shown in Figure 2.4 approximately at

the point (1.0, 0.3). Note that it is not in Z. The next best thing is a solution that is as

close as possible to the utopia point. Such a solution is called a compromise solution.

Although the ideas of a utopia point and a compromise solution occasionally are used in

reference to solution concepts, as is the case here, they are more prevalent as facets of

specific methods that provide Pareto optimal solutions. In fact, a compromise solution is

Pareto optimal.

A difficulty with the idea of a compromise solution is the definition of the word

close. The term close usually implies that one minimizes the Euclidean distance ( )N x ,

which is defined as follows:

( ) ( ) ( )

1

22

1

k

i iN F F

= − = −

∑x F x F x

� � (2.4)

However, there is no reason for restricting closeness to the case of a Euclidean norm

(Vincent, 1983). Furthermore, if different objective functions have different units, the

31

Euclidean norm or a norm of any degree becomes insufficient to represent closeness

mathematically. Consequently, the objective functions should be transformed such that

they are dimensionless; each component of the summation argument in ( )N x should be

divided by i

F� , yielding what is called the relative deviation of each function

(Salukvadze, 1979).

32

CHAPTER III

SUMMARY OF THE LITERATURE REVIEW

3.1 Introduction

Much of the fundamental motivation for this dissertation stems from a monograph

that is provided by Marler and Arora (2003) and is abbreviated in the survey by Marler

and Arora (2004b). This work serves not only as a synopsis of current literature on

research and theory, but also as a qualitative analysis of multi-objective optimization

(MOO) methods. Methods are evaluated qualitatively in terms of their computational

requirements, ease of use, and the characteristics of the solutions they provide.

Consequently, this work satisfies Objective 1 in section 1.4. The predominant MOO

methods and their variants are studied. Approximately 60 methods are analyzed, and an

extensive list of references is provided. However, despite the extensive scope of the

review, its intent is not to present every available method, but to provide a thorough state-

of-the-art review of the most common current methods in engineering, with a concise

discussion of each method and of each class of methods. Fundamental concepts are

explained thoroughly. Necessary and sufficient optimality conditions are described.

Explicit algorithms are presented in a ready-to-use format.

Significant characteristics of the methods are presented in a comprehensive

comparison matrix, which provides a new and useful tool when it comes to selecting an

appropriate method for a particular problem. The matrix indicates whether or not each

method is a scalarization method and whether or not each method requires a utopia point.

Also noted in the matrix is whether each method provides a necessary and/or sufficient

33

condition for Pareto optimality. The programming complexity, software use complexity,

and computational complexity are rated on a scale of zero to ten. It is indicated where

there is a potential for future work, when classification of a method may not readily be

interpreted or available in the current literature.

In this section, the work of Marler and Arora (2003) is summarized. The broad

classifications of methods and the most common methods themselves are briefly

described; and key findings are reiterated. Conclusions concerning future research,

which provide the motivation for much of the work in this dissertation, are summarized.

In addition to the expanse of literature in the general review, specific literature reviews

are provided for the weighted sum method, function transformations, the weighted global

criterion approach, the weighted min-max approach, and the ε -constraint method in their

respective chapters. These individual reviews argue the necessity and significance of the

studies in each chapter.

3.2 Surveys in the Literature

Although surveys of MOO methods are common, often they are incomplete in

terms of comprehensive coverage, algorithm presentation, and general applicability to

engineering design. For example, many papers only target specific applications, while

others only address the critical literature in the field. As opposed to presenting

computational methods, some surveys focus on historical aspects. At the opposite end of

the spectrum are discussions that focus on mathematics. Surveys that do explicitly

present a variety of algorithms tend to incorporate a minimal discussion of their

advantages and disadvantages. Furthermore, most surveys only target a limited number

of approaches. Textbooks can be more complete in terms of the treatment of a specific

34

topic. However, if one simply wants to extract explicit algorithms, a book, as opposed to

a review paper, can inundate the engineer with too much information. These deficiencies

in the literature are addressed in Marler and Arora (2003).

3.3 No Articulation of Preferences

MOO methods can be grouped into categories with similar characteristics. For

instance, the primary classification for standard methods concerns articulation of

preferences. The most general approach to problems with no articulation of preferences

is to combine the objective functions in a scalar aggregate function called a global

criterion. This method can be distilled further into different classifications such as

distance function methods, which entail minimizing the distance between the utopia point

(or an approximation of the utopia point) and the final solution. Compromise

programming is another subtly distinguishable subset of the global criterion method,

which implies the use of an exponent as a variable method-parameter. The exponential

sum method entails adding all of the objective functions together, with each raised to an

exponent. The objective sum method simply entails adding all of the objective functions

together; it tends to be a default approach to MOO. The min-max method involves

minimizing the objective with the highest value (at a current iteration). The Nash

arbitration scheme is derived from game theory and essentially involves minimizing the

product of scaled objective functions, whereas the more fundamental objective product

method excludes the scaling factor.

3.4 A Priori Articulation of Preferences

Methods with a priori articulation of preferences constitute the most common

approaches and form the most extensive category. In many cases, these methods are

35

developed by introducing method-parameters to the formulations for methods with no

articulation of preferences. The weighted global criterion and weighted min-max

methods, and the weighted sum method are the most common approaches.

Lexicographic and hierarchical methods involve ordering objectives according to relative

importance and then minimizing one objective at a time. The weighted product method

entails using exponents as weights with the objective product method. Goal

programming methods allow one to express preferences by setting goals for each

objective function and then minimizing the deviation from those goals. Bounded

objective function methods, such as the ε -constraint approach, minimize a single

objective function while constraints are placed on all other objectives. Physical

programming is a relatively new approach that lets the user design the individual utility

functions and combine them into a single utility function.

3.5 A Posteriori Articulation of Preferences

When method-parameters are varied consistently as purely mathematical

parameters, rather than as indications of relative objective-importance, many of the

above-mentioned methods can be used for a posteriori articulation of preferences. In

addition, some approaches are designed specifically for a posteriori articulation, such as

the normal boundary intersection (NBI) method, the normal constraint (NC) methods, the

adaptive search method, and the projection method.

3.6 Fuzzy Multi-objective Optimization

Often, constraint limits and objectives are not in absolute terms; they may be

fuzzy rather than crisp, and such constraints and objectives may be modeled

mathematically using fuzzy theory. Fuzzy theory is a field of mathematics that enables

36

one to model systems that involve non-quantitative human reason, perception, and

interpretation. With fuzzy optimization, constraints and objectives are treated

equivalently, a condition that lends itself to use with multiple objectives. Each constraint

and objective is modeled with what is called a membership function, which is developed

based on the insight and experience of the user. The literature on general fuzzy

optimization and fuzzy MOO is extensive. In addition, fuzzy MOO is incorporated into

weighted methods, goal programming, and global criterion methods.

3.7 Genetic Multi-objective Optimization

There are many approaches to solving MOO problems by using global methods.

However, genetic algorithms are by far the most common global optimization technique

that is used with multi-objective problems. Consequently, the amount of literature

concerning genetic algorithms for MOO and genetic algorithms, in general, is vast. The

algorithms are loosely based on Darwin’s theory of natural selection, although

deficiencies in the comparison between biology and optimization algorithms are

highlighted in the review. In addition, the basic components of general genetic

algorithms are outlined. Genetic algorithms work with a population of points and

consequently can converge on the Pareto optimal set as a whole rather than finding one

point at a time. In this respect, they are most appropriate as a method for a posteriori

articulation of preferences.

3.8 Discussion and Conclusions

Evaluations of specific methods are presented throughout the review, and

conclusions concerning different classes of methods are stated at the end of each section.

It is found that no single approach is superior; rather, the selection of a specific method

37

depends on the type of information provided in the problem, user preferences, solution

requirements, and availability of software. Consequently, the comparison matrix

discussed above provides an extremely useful tool that was previously unavailable.

Some of the conclusions most relevant to this work are the following:

1) In general, MOO requires more computational effort than single-objective

optimization. Unless preferences are irrelevant or completely understood prior to the

implementation of an algorithm, the solution of several single-objective optimization

problems may be necessary to obtain an acceptable final solution.

2) Most MOO methods are essentially a means to approximate the decision-maker’s

preferences. In fact, one of the primary advantages of using MOO is its ability to

incorporate such preferences in recognition of multiple potential solutions. Thus,

selection of a method depends on how a decision-maker wishes to articulate

preferences, and this in turn depends on the type of information provided in a

problem.

3) Solutions obtained with no articulation of preferences are arbitrary, relative to the

Pareto optimal set. In this class of methods, the objective sum method is one of the

most computationally efficient, easy-to-use, and common approaches. Consequently,

it provides a benchmark approach to MOO.

4) Methods with a priori articulation of preferences require the user to articulate

preferences only in terms of objective functions in the criterion space. Alternatively,

methods with a posteriori articulation allow the user to view potential solutions in the

criterion space and/or in the design space, and select an acceptable solution

accordingly.

38

5) Selection of a specific scalarization method for a priori articulation of preferences,

which allows one to design a utility function, depends on the type of preferences that

the decision-maker wishes to articulate and on the amount of preference-information

that the decision-maker has.

6) The results of methods with a posteriori articulation of preferences, i.e. the Pareto

optimal set, are depicted typically in the criterion space. Consequently, much

literature addresses the issue of providing an even spread of points in the criterion

space. This is a consequence of the tendency to make decisions based on objectives

or criteria. However, similar points in the criterion space may correspond to

distinctly different points in the design space. Thus, when possible, solutions in the

design space should be used to complement solutions in the criterion space.

7) Methods with a posteriori articulation of preferences reflect user preferences

accurately, but they are suitable only for problems with relatively few objective

functions. Out of this class of methods, the physical programming approach and the

normal boundary intersection method are relatively effective in terms of depicting the

Pareto optimal set.

8) In terms of CPU time, methods with a posteriori articulation of preferences are less

efficient than methods with a priori articulation of preferences. Since only one

solution is selected, the time spent determining other Pareto optimal points, is wasted.

In addition, regardless of the specific method being used, presenting the Pareto

optimal set clearly can be a formidable task.

39

9) Even an approximate determination of utopia points can be expensive, particularly

when there are many objective functions. If CPU time is an issue, utopia points

should be avoided. Unattainable aspiration points provide a reasonable substitution.

10) Vector methods, such as the lexicographic method, require the solution of a sequence

of problems and tend to require more CPU time than scalarization methods.

In addition to the above-mentioned conclusions, areas for significant future

investigation are revealed, and these findings provide motivation for much of the work in

this dissertation. For instance, it is found that insufficient effort has been applied to

depicting the Pareto optimal set because of restrictions in computational capabilities.

Now that such restrictions have become less substantial, basic scalarization methods can

be considered for a posteriori articulation of preferences, and their performance in this

capacity should be tested. A series of problems can be solved with method-parameters

being varied consistently as mathematical parameters rather than fixed as indications of

preference. It is also found that although some basic material is presented, the

significance of and relationship between different method-parameters used with

fundamental weighted methods are not well understood. Finally, although significant

applications for MOO are becoming more common, much of the literature involves only

simple illustrative problems. Performance with practical problems that have a relatively

high number of objective functions and/or design variables needs to be considered.

40

CHAPTER IV

WEIGHTED SUM METHOD

4.1 Introduction

The weighted sum method is one of the most common methods for multi-

objective optimization (MOO). However, although some literature discusses the pitfalls

of this method, the fundamental significance of the weights has not been addressed

thoroughly. Thus, in this chapter, new insights into the significance of the weights are

provided with respect to 1) preferences, 2) the Pareto optimal set, and 3) objective-

function values.

There are many different approaches for determining the weights (Marler and

Arora, 2003). Ultimately, however, these are all different methods for organizing one’s

preferences. Instead of proposing yet, another algorithm for translating preferences into

weights, in this chapter, we focus on the mathematical characteristics of the solution to

the weighted sum and on the fundamental meaning of the weights. We determine the

factors that dictate which Pareto optimal solution point results from a particular set of

weights. New understanding of the fundamentals, which is presented in this chapter, can

be useful in setting the weights directly (without the use of an additional algorithm). In

addition, understanding inadequacies that are inherent in the method is instructive. In

this vein, we present fundamental deficiencies that cannot be avoided and that have not

been discussed in the current literature.

The weighted sum method may be used to incorporate preferences in two ways.

One may use the method for a priori articulation of preferences, which implies that the

41

weights are assigned to indicate the relative importance of the objective functions before

the problem is solved. Alternatively, one may use the method for a posteriori articulation

of preferences, which entails selecting a single solution from a set of Pareto optimal

solutions. In the latter case, a representation of the Pareto optimal set is determined by

solving a sequence of weighted sum problems, each with a different set of weights. This

chapter discusses the weighted sum method in terms of a priori articulation, whereas

Chapter 5 involves the use of the weighted sum method for depicting the Pareto optimal

set (a posteriori articulation).

Setting weights is just one approach to articulating preferences, and is applicable

to many different methods. The weighted sum method is the most basic of such

approaches. Consequently, understanding how weights affect the solution with this

method has implications concerning other approaches that involve similar method-

parameters. As a simple approach that is sufficient for Pareto optimality, the weighted

sum method also provides a convenient test bed for transformation methods, which are

discussed in Chapter 5. In conjunction with Chapters 5, this chapter responds to

objective 2 in section 1.4, which concerns the thorough study of method-parameters and

formulation variations.

4.1.1 Overview of the Formulation

The default approach for MOO often entails simply adding the objective functions

together, forming a single, composite function. Incorporating weights for different

objectives can increase the effectiveness of this approach. The result is the following

utility function, which is minimized:

( )1

k

i i

i

U wF

=

=∑ x (4.1)

42

If all of the weights are positive, as assumed in this study, then minimizing (4.1) provides

a sufficient condition for Pareto optimality, which means the minimum of (4.1) is always

Pareto optimal (Zadeh, 1963; Goicoechea et al, 1982). Although some literature

indicates that 1

1k

iiw

=

=∑ and ≥w 0 , if any one of the weights is zero, there is a potential

for the solution to be only weakly Pareto optimal.

Minimizing the weighted sum provides a necessary condition if the multi-

objective problem is convex, which means the feasible design space and all of the

objective functions are convex (Geoffrion, 1968; Koski, 1985; Miettinen, 1999).

Because the Pareto optimal set is always on the boundary of the feasible criterion space

Z, some studies suggest that Z must be convex for the weighted sum method to yield all

of the Pareto optimal points (Stadler, 1995; Athan and Paplambros, 1996). However, Lin

(1975) demonstrates that, in fact, if Z is only p-directionally convex (Holtzman and

Halkin, 1966) for any definitely negative vector p, then the weighted sum approach

provides a necessary condition. The term definitely negative implies <p 0 . p-

directionally convex is defined as follows:

Definition 4.1 p-Directionally Convex: Given a nonzero vector kE∈p ,

kE⊂Z is said to be p-directionally convex if given any two different

points in Z, 1 2 and F F , and any two positive scalars,

1 2and w w , with

1 21w w+ = , there is a positive number β such that

1 1 2 2w w β+ + ∈F F p Z .

This means that only the portion of the feasible criterion space that is visible when

looking in the direction of p from within the feasible space needs to be convex.

43

Therefore, only the Pareto optimal hypersurface needs to be convex for the weighted sum

method to provide a necessary condition for Pareto optimality.

4.1.2 Review of the Literature and Motivation

Following the introduction of the weighted sum method by Zadeh (1963), the

method has been mentioned prominently in the literature (Goicoechea et al, 1982; Steuer,

1989; Eschenauer et al, 1990; Yoon and Hwang, 1995; and others). In fact, any

discussion of MOO usually includes some reference to this method. However, most

general treatments of MOO simply outline the weighted sum approach and indicate that it

provides a Pareto optimal solution. Intricacies of the method and of the solutions that it

yields are not discussed. In particular, the significance of the weights is not thoroughly

explored, and thus, despite the presence of many algorithms for determining weight

values, no basal guidelines have been presented for selecting weights.

The weighted sum method is often presented strictly as a tool, with a focus on the

application, but the problems tend to be limited to those with just two objective functions.

For instance, as development for a new approach, Koski and Silvennoinen (1987) use the

weighted sum method to minimize the volume and the nodal displacement of a four-bar

space truss. Kassaimah et al (1995) use the method for the two-objective optimization of

laminated plates, where critical buckling shear stress is maximized and deflection is

minimized. Proos et al (2001) apply the method to topology optimization, minimizing

compliance and maximizing the first mode of the natural frequency, for two-dimensional

plane stress problems.

Some literature provides a more substantial look at the method. Koski (1985)

demonstrates the method’s inability to capture Pareto optimal points that lie on non-

44

convex portions of the Pareto optimal curve, using basic two-objective structural

problems. The work of Stadler and Dauer (1992), and Stadler (1995) points out the same

deficiency using the problem of maximizing the volume and minimizing the surface area

of a closed cylindrical container. Chen et al (1999) illustrate this difficulty using a two-

objective mathematical example and a two-objective problem concerning the robust

design of a two-bar truss. Athan and Papalambros (1996) also discuss this deficiency but

only within the context of preparing new methods to rectify it.

In reference to a posteriori articulation of preferences, Das and Dennis (1997)

evaluate why the weighted sum may not provide an even distribution of Pareto optimal

points. For a two-objective problem, they derive the explicit form of the unique Pareto

optimal curve for which an even distribution of weights yields an even distribution of

Pareto solutions. In general, however, although varying the weights yields different

Pareto optimal solutions, an even distribution of weights does not necessarily result in an

even distribution of points that accurately represents the Pareto optimal set. Instead, the

points that result from a systematic selection of weights may cluster together. As

suggested in section 1.3, even implies a consistent Euclidean distance between

consecutive solution points. This fault is often answered with alternative methods for

representing a Pareto optimal set (Hwang and Md. Musad, 1979; Das and Dennis, 1998;

Das, 1999; Messac and Mattson, 2002; Tappeta et al, 2000; Messac et al, 2001). Such

alternatives can be effective and valuable, but they are usually independent of the

weighted sum method. Although pitfalls of the weighted sum method are occasionally

addressed in the literature, few suggestions for improvement are proposed, analysis of the

45

method is incomplete, and criteria for maximizing its effectiveness are not available.

Essentially, no fundamental unifying analysis is provided.

4.1.2.1 Approaches to Setting Weights

Misinterpretation of the theoretical and practical meaning of the weights can

make the process of intuitively selecting non-arbitrary weights an inefficient chore.

Consequently, many researchers have developed systematic approaches to selecting

weights, surveys of which are provided by Eckenrode (1965), Hobbs (1980), Hwang and

Yoon (1981), and Voogd (1983). We identify two broad classes approaches, but as we

will show, with these approaches, weights may still be set ineffectively. Here, we briefly

describe the basic general approaches. With rating methods, which constitute the most

common approach, the decision-maker assigns independent values of relative importance

to each objective function. With ranking methods (Yoon and Hwang, 1995), which can

be viewed as a subset of rating methods, the different objective functions are ordered by

importance. Then, the least important objective receives a weight of one, and integer

weights with consistent increments are assigned to objectives that are more important.

The same approach is used with categorization methods, in which different objectives are

grouped in broad categories such as highly important, and moderately important. Ratio

questioning or paired comparison methods are also common and provide systematic

means to rate objective functions by comparing two objectives at a time. In this vein,

Saaty (1977, 1998, 2003) provides an eigenvalue method of determining weights, which

involves ( )1 2k k − pair-wise comparisons between objective functions. This yields a

comparison matrix, and the eigenvalues of the matrix are used as the weights. Wierzbicki

(1986) provides an algorithm that calculates weights based on the aspiration point and the

46

utopia point. The closer an aspiration is to the utopia point, the higher the value of the

corresponding weight is. Rao and Roy (1989) discuss a method for determining weights

based on fuzzy set theory. For the most part, algorithms for determining weights either

fall into the category of rating methods or paired comparison methods.

Although the above-mentioned algorithms provide valid processes for

determining the weights, there is little rigorous investigation as to what the weights

actually represent. The mathematical relationship between weight values and objective-

function values is not investigated sufficiently. By studying the fundamental significance

of the weights from different viewpoints, intrinsic flaws in the rating methods and paired

comparison methods are uncovered. In addition, we explain why even with an effective

algorithm that helps one decide on values for the weights, a weighted sum is not

necessarily capable of incorporating preferences completely. The method is simplistic

and only provides a basic means for expressing one’s preferences; it has a low capacity

for preference information as suggested by Marler and Arora (2004a).

4.1.3 Objectives of the Present Study

Based on the motivation discussed above, the following specific objectives are

pursued in this chapter:

1) Investigate the fundamental significance of the weights in terms of preferences, the

Pareto optimal set, and objective-function values.

2) Identify fundamental deficiencies in the method, in terms of a priori articulation of

preferences.

3) Determine the factors that dictate which Pareto optimal solution point results from a

particular set of weights.

47

4) Provide guidelines that help avoid blind use of the weighted sum method.

5) Provide a test bed for the study of function-transformation methods, which is

presented in Chapter 5.

4.2 Interpretation of the Weights

One often views the weights as general gauges of relative importance for each

objective function. However, selecting a set of weights that reflects preference towards

one objective or another can be difficult, because preferences tend to be indistinct.

Although it is certainly possible to organize or categorize a set of discrete options by

relative importance as with ranking methods, quantifying preference usually involves

some degree of ambiguity. In addition, even with full knowledge of the objectives and

satisfactory selection of weights, the final solution may not necessarily reflect the

intended preferences that are supposedly incorporated in the weights. Consequently, it is

often necessary to resolve the weighted sum problem with alterations in the weights

based on trial and error. Nonetheless, the specific Pareto optimal point that is provided as

the solution depends on which weights are used. It is important to determine how the

weights relate to 1) preferences, 2) the Pareto optimal set, and 3) the individual objective

functions. These relationships are explored in this section.

4.2.1 Preferences

First, we provide an analysis of the relation between preferences and weights.

Concurrently, we study the theoretical foundation of rating methods, which are

mentioned in section 4.1.2. We discover that such methods allow one to inadvertently set

the weights according to the relative magnitude of the objective functions rather than the

relative importance of the objectives. In this way, we demonstrate how function-

48

transformation can be advantageous with a priori articulation of preferences. In addition,

we reveal inherent deficiencies in the weighted sum method with respect to its capacity

for incorporating preferences.

The gradients of the preference function ( )P F x and the utility function in (4.1)

are given respectively as follows:

( ) ( )1

k

i

i i

PP F

F=

∂∇ = ∇ ∂

∑x xF x x (4.2)

( )1

k

i i

i

U w F

=

∇ = ∇∑x xx (4.3)

Each component of the gradient P∇x

qualitatively represents how the decision-maker’s

satisfaction changes with a change in the design point and a consequent change in

function values. Comparing (4.2) and (4.3) suggests that if the weights are selected

properly, then the utility function can have a gradient that is parallel to the gradient of

the preference function. This is significant because the purpose of any utility function is

to approximate the preference function, so imposing similarities between these two

functions such as parallel gradients, can result in a more accurate representation of one’s

preferences. If the utility function should ideally be the same as the preference function,

then certainly the gradients of the two functions should be the same as well.

The relationships above indicate that i

w represents i

P F∂ ∂ . Conceptually,

iP F∂ ∂ is the approximate change in the preference-function value (change in a decision-

maker’s satisfaction) that results from a change in the objective-function value for i

F . A

weight is commonly thought to represent the importance of its corresponding objective

function, but importance is a vague concept. Although i

P F∂ ∂ provides a definition for

49

the importance of i

F , it only makes sense to consider the importance of an objective or

change in preference-function value, in relative terms. Thus, the value of a weight is

significant relative to the values of other weights; the independent absolute magnitude of

a weight is irrelevant in terms of preferences.

4.2.1.1 Deficiencies in the Method

Although (4.2) and (4.3) provide a general, conceptual idea of what the weights

represent in terms of preferences, using them to set precise values for weights can be

difficult. In fact, a fundamental deficiency in the weighted sum method is that it can be

difficult to discern between setting weights to compensate for differences in objective-

function magnitudes and setting weights to indicate the relative importance of an

objective as is done with rating methods. For practical purposes, i

P F∂ ∂ can be

approximated as i

P F∆ ∆ , and when objective functions have different ranges and orders

of magnitude, an appropriate value for i

F∆ may not be apparent. For example, consider

two objective functions to be minimized. Function-one is stress, which for the sake of

argument ranges between 10,000 psi and 40,000 psi. Function-two is displacement,

which ranges between 1cm and 2cm. If displacement were to decrease by one unit, the

consequence would be a significant improvement; 2

P F∆ ∆ would be relatively high.

Concurrently, if stress were to decrease by a unit of one, the consequence would be

negligible; 1

P F∆ ∆ would be insignificant. The result would be a relatively large value

for 2

w , compared to 1

w . However, this result does not necessarily reflect the relative

importance of the objectives; the values for the weights have been dictated purely by the

relative magnitudes of the objective functions.

50

Alternatively, one could consider percentage changes in the objective functions

rather than an absolute change of unity as discussed above. However, then the question

arises as to what one should consider when evaluating percentages. Should i

F∆ be a

percentage of the average value for i

F , the maximum value for i

F , the minimum value

for i

F , or some factor of the range of values for i

F ? The choice is arbitrary.

Consequently, one may set the weights to reflect objective-importance directly,

but this approach can also lead to difficulties. With the previous example, suppose

displacement is twice as important as stress. Considering the relative value of the

weights, if 1

1w = , then 2

2w = . However, given the relative magnitudes of the objective

functions, stress would still dominate the weighted sum, and the use of 2

2w = would be

irrelevant. Again, the magnitudes of the objective functions affect how the weights are

set, and the articulation of preferences becomes blurred at best.

This difficulty arises because what dictate the solution to the weighted sum are

the relative magnitudes of the terms i i

w f , in (4.1), not just the magnitudes of the

functions or weights alone. The value of a weight is significant not only relative to other

weights but also relative to its corresponding objective function. This is a critical idea,

although it is often overlooked. With many weighted methods, including the weighted

sum method, preferences are articulated a priori essentially by using weights to induce

the dominance of a particular term in a formulation (and the objective function associated

with the term). However, the process of selecting weights and thus indicating

preferences is complicated if this dominance is instead, induced by relative function-

magnitude. Thus, transforming functions so that they all have similar magnitudes and do

not naturally dominate a formulation, can help one set parameters to reflect preferences

51

more accurately, when using weights to represent the relative importance of the

objectives.

Further study of (4.1), (4.2), and (4.3) reveals another fundamental deficiency in

the weighted sum method: the weighted sum utility function is only a linear

approximation (in the criterion space) of the preference function. (4.1) is a linear

function of the objective functions; it is the most basic approximation of one’s preference

function. The consequences of this characteristic are explained as follows. i

P F∂ ∂ may

actually be nonlinear, changing from point to point. However, using a weighted sum as a

utility function to approximate the preference function, inherently assumes that this term

is constant, represented by a scalar weight. Thus, values that are selected for the weights

are only locally relevant. That is, they are not necessarily appropriate for the complete

criterion space, only for the point at which they are determined, presumably the starting

point (initial guess) for the optimization engine. As the design point changes with an

iterative optimization process, the values of the objective functions change, and

consequently, preferences (between various points in the criterion space) may change in a

nonlinear fashion. Anticipating such change is impractical, so decision-makers implicitly

assume that i

P F∂ ∂ is constant, when they determine a set of weights to reflect the

relative importance of the objectives. This means that one assumes that the gradient of

the preference function (in the criterion space) is constant, which is not necessarily the

case. This is why even with a process that enables one to determine acceptable values for

the weights, the final solution to the weighted sum problem may not accurately reflect

initial preferences that were supposedly incorporated in the weights.

52

4.2.2 Pareto Optimal Set

In this section and the next, factors that affect the solution to the weighted sum

method are determined. In addition, fundamental characteristics of the weights are

presented with regard to the Pareto optimal set, whereas the previous section concerned

fundamental characteristics with respect to preferences. The characteristics discussed

here provide theoretical foundation for the paired comparison methods discussed in

section 4.1.2. We point out a fundamental deficiency with such methods, and we

discover that function transformations can be detrimental with these methods.

Figure 4.1 depicts a general illustrative model of a Pareto optimal set in the

criterion space. Note that this figure could represent either a constrained or an

unconstrained problem, because in the criterion space, constraints are naturally

incorporated in the Pareto optimal set. This idea is demonstrated with Figures 2.3 and

2.4.

Z

F1

F2

Pareto

Optimal

Set Direction of

Decreasing

U-values

( U−∇F

)

Utility Function Contours

( ) ( )1 1 2 2

U w F w F= +x x

Figure 4.1: Pareto Optimal Set in the Criterion Space

53

The weights represent the gradient of U in (4.1) with respect to the vector function ( )F x ,

shown as follows:

1 1

2

2

U

F wU

wU

F

∂ ∂

∇ = = ∂

∂

F (4.4)

This reinforces the idea discussed in section 4.2.1. The Pareto optimal solution (for a

given set of weights) is found by determining where the U-contour with the lowest

possible value intersects the boundary of the feasible criterion space. Recall that the

Pareto optimal set is on the boundary of the feasible criterion space. The U-contours are

linear and are thus tangent to the Pareto optimal curve at the solution point. Changes in

the relative values of the weights result in changes in the orientation of the U-contours.

The subsequent change in the solution depends on the shape of the Pareto optimal curve

in the criterion space. This has significant repercussions when trying to provide an even

spread of Pareto optimal points by solving a series of weighted sum problems, for a

posteriori articulation of preferences. In fact, some more complicated methods have been

developed so that the shape of the Pareto optimal hypersurface has a minimal affect the

accuracy with which it is depicted (Marler and Arora, 2003).

In terms of higher dimensions (more than two objective functions), the minimum

value of (4.1) for a fixed set of weights determines a supporting hyperplane to Z (i.e.

( ) constantU =F ) (Zadeh, 1963; Gembicki, 1974), and the normal of the hyperplane is

the vector of weights. The Pareto optimal hypersurface is always on the boundary of Z,

54

so the supporting hyperplane is tangent to the hypersurface at the point that minimizes the

weighted sum.

Steuer (1989) uses such a hyperplane to provide an interpretation of the weights,

which we suggest forms the basis for paired comparison methods. This interpretation is

explained as follows. 1

1w = is considered a reference weight, and 1F is a reference

function (theoretically, any function can be used as a reference). Then, i

∆ represents the

amount by which i

F must increase in order to compensate for a decrease (improvement)

of 1

∆ in 1F , while remaining on the hyperplane. Each weight is then defined as follows:

1

1

0

limi

i

w

∆ →

∆=

∆ (4.5)

If one assumes 1

1∆ = , then each weight is approximated as

1

i

i

w ≈∆

(4.6)

Thus, i

w represents the trade-off between i

F and the reference function, at the solution

point to the weighted sum problem. This is the idea behind paired comparison methods

for setting weights.

To clarify this idea, note that one can draw the same conclusion using Figure 4.1

for the case of two objective functions. Considering that the solution to the weighted sum

problem is always Pareto optimal, the slope of the Pareto optimal curve in Figure 4.1 is

determined as follows:

2 1

1 2

dF w

dF w= − (4.7)

55

The left side of (4.7) can be approximated as 2 1

F F∆ ∆ . Then, assuming 1

1w = is again a

reference weight and 1

1F∆ = (comparable to using 1

1∆ = as above), (4.7) is reduced to

2

2

1w

F≈ −

∆ (4.8)

There is no negative sign in (4.6), because the numerator is assumed to represent a

decrease of one.

Using (4.5) involves comparing the different objective functions and evaluating

trade-offs (how much of a loss in an objective one is willing to sacrifice for a gain in

another objective). In contrast to the idea of evaluating relative importance, which was

discussed with regard to rating methods, assigning values to the weights based on trade-

offs can be hampered by function transformations. Trade-off comparisons between

functions are easier when the functions retain their original physical significance and

original units. However, most transformation methods yield unitless functions, the values

of which have little or no physical significance.

Although (4.5) is derived based on the mathematical characteristics of (4.1), it

provides another general means for articulating preferences. However, here we point out

an inherent deficiency in such methods. Note that (4.5) is a relationship that necessarily

exists at the solution point for the weighted sum. However, a user’s decisions concerning

trade-offs can change depending on the point at which the different objective functions

are evaluated. One may be forced to determine trade-offs based on values for the

different functions when the functions are evaluated at a point other than the solution

point. Therefore, as with (4.2) and (4.3), (4.5) only provides an approximate means for

56

incorporating preferences in the weighted sum utility function a priori (before the

solution point is actually determined).

Figure 4.1 and (4.4) provide reinforcement of the conclusions drawn in section

4.2.1 concerning the significance of the weights. For instance, note that the direction of

the gradient of U in the criterion space, and thus the orientation of the U-contours and

corresponding solution point, depends on the relative values (not the absolute values) of

the weights. In addition, the orientation of the U-contours relative to the Pareto optimal

set also depends on the range of objective-function values for points in the set. That is,

for a given set of weights (a given gradient vector for the weighted sum), scaling an

objective function changes the Pareto optimal solution point that the weighted sum

method provides. Thus, as suggested earlier, the value of a weight is significant relative

to the values of other weights and relative to the value of its corresponding objective

function.

4.2.3 Objective Functions

Whereas the discussion above considers the significance of weights in terms of

the Pareto optimal set, this section focuses on the relationship between a set of weights

and the objective function values. First, consider an illustrative problem with two

unconstrained objective functions:

( ) ( )1 1 2 2U w F w F= +x x (4.9)

For U to have a minimum, it is necessary that the gradient of U be equal to zero, as

follows:

1 1 2 2

U w F w F∇ = ∇ + ∇ =x x x

0 (4.10)

57

Assuming the weights are positive and noting that minimizing a weighted sum always

provides a Pareto optimal solution, (4.10) indicates that at a Pareto optimal point, the

gradients of the two objective functions are co-linear and point in opposite directions.

Essentially, the linear combination of the gradients equals zero. In compliance with the

definition of Pareto optimality, this suggests that moving from a solution that satisfies

(4.10), in order to improve a function, is detrimental to at least one other function.

General constrained problems are considered using Kuhn-Tucker necessary

conditions, which are stated in Marler and Arora (2003). These conditions are simplified

as follows:

1 1

amk

i i i i

i i

w F gµ

= =

∇ = − ∇∑ ∑x x (4.11)

where a

m is the number of active constraints and i

µ is the Lagrange multiplier for

( )i

g x . (4.11) indicates that a linear combination of the objective function gradients is

equal and opposite to a linear combination of the constraint gradients (for active

constraints), at a Pareto optimal point. If a constraint is active at the solution point for a

weighted sum, which necessarily provides a Pareto optimal point, then that constraint

forms part of the Pareto optimal set. (4.10) and (4.11) suggest that the Pareto optimal

point resulting from the use of a specific set of weights depends on active constraints that

form part of the Pareto optimal set and on the relationship between the gradients of the

different objective functions.


In this chapter, we have provided new insight into how the weighted sum method

works. We explore the significance of the weights with respect to preferences, the Pareto

58

optimal set, and the objective-function values. We show that although the weighted sum

method is easy to use, it provides only a basic approximation of the preference function.

Thus, the solution may not represent one’s initial preferences no matter how the weights

are set, a crucial idea that is consistently overlooked in the literature. We determine that

the solution depends on multiple factors, one of which is the relative magnitude of the

objective functions. However, when setting the weights, only the relative importance of

the objectives should be considered, not the relative magnitudes of the functions. In this

way, we argue that function-transformation can be helpful with a priori articulation of

preferences, when the weights are used to represent the relative importance of the

objectives (as with rating methods). This finding, also overlooked in the literature,

provides a key guideline that should always be considered when setting weights and/or

transforming objective functions. As we will show in the next chapter, this is true for a

posteriori articulation as well. Alternatively, we argue that if the weights are viewed as

representing trade-offs between objective functions (as with paired comparison methods)

at the solution point, then retaining the original units of the objectives, without

transformation, is advantageous. In summary, the relationship between weights and

objective-function values must always be considered. The fundamental distinction

between this relationship when rating methods are used and when paired comparison

methods are used has never before been addressed.

Although some literature suggests that the weights be set such that 1

1k

iiw

=

=∑

and ≥w 0 , which results in a convex combination of objective functions, when the

weighted sum is used for a priori articulation of preferences, there is no need to place any

restriction on the values of the weights other than >w 0 , which ensures Pareto

59

optimality. In fact, as indicated in this chapter, using unrestricted weights can be

beneficial, making the determination of appropriate weight values easier. However, as

we will show in the next chapter, when systematically altering the weights for a posteriori

articulation of preferences, using a convex combination of objective function can be

helpful so as not to repeat any weighting vectors in terms of relative values. Thus,

weights should be treated differently when one uses a method for a priori articulation

than they are when one uses a method for a posteriori articulation.

Conclusions based on this work and guidelines for setting weights are presented

as follows:

1) The value of a single weight is significant relative to the values of other weights and

relative to the magnitude of its corresponding objective function.

2) All objective functions should be transformed such that they have similar ranges,

when using the weights to represent the relative importance of the objectives with a

priori articulation of preferences (as with rating methods).

3) Objective functions should not necessarily be transformed when the weights are set

using trade-offs (as with paired comparison methods).

4) Unrestricted positive weights should be used with paired comparison methods when

articulating well-understood preferences a priori. Otherwise, a convex combination

of objective functions should be used.

5) The weighted sum provides only a basic approximation of one’s preference function.

It is fundamentally incapable of incorporating complex preference information. Even

if one determines acceptable values for the weights a priori, the final solution to the

60

weighted sum problem may not accurately reflect initial preferences supposedly

incorporated in the weights.

6) Which Pareto optimal solution results from a specific set of weights depends on the

following factors: constraints that form part of the Pareto optimal set, the

relationships between the gradients of the different objective functions, the relative

magnitudes of the objective functions, and the shape of the Pareto optimal

hypersurface.

61

CHAPTER V

FUNCTION TRANSFORMATIONS

5.1 Introduction

Multi-objective optimization (MOO) inherently involves making decisions about

and comparing different objectives, which often are in the form of mathematical

functions. However, these functions may have different units and/or significantly

different orders of magnitude, making comparisons difficult if not irrelevant.

Furthermore, when a scalarization approach is used, it is possible that a single objective

function with relatively large values will dominate the aggregated objective function, thus

nullifying the effort to consider multiple objectives. Consequently, it is necessary to

transform the objective functions such that they have similar orders of magnitude. Such a

procedure is advantageous both for a priori and for a posteriori articulation of

preferences.

In terms of a priori articulation of preferences, when rating methods are used to

articulate preferences, transforming functions so that they have comparable values allows

one to set method-parameters more accurately. This advantage is highlighted in Chapter

4. In terms of a posteriori articulation of preferences, this chapter shows that using

transformed objective functions can improve a method’s ability to depict the complete

Pareto optimal set in the design space and/or the criterion space, with consistent variation

in method-parameters.

In this chapter, we study different function-transformation (and normalization)

methods that can help in generating an approximation of the Pareto optimal set. To this

62

end, the weighted sum method is used as representative of the many scalarization

approaches. New understanding is provided concerning the importance of function-

transformations, even when the original objective functions may not seem to warrant

such transformation. The pros and cons of different approaches are revealed. It is shown

that some methods can actually be detrimental to the process of generating a diverse

spread of Pareto optimal points, and criteria are proposed for determining when certain

approaches will fail to generate an accurate representation of the Pareto set. In

conjunction with Chapter 4, this chapter provides practical insight concerning the

difference between using the weighted sum for a priori articulation and a posteriori

articulation.

Function-transformation schemes not only serve as tools that can be used to

improve different approaches, but also constitute various formulations incorporated in a

single approach such as the global criterion approach, discussed in Chapter 6. In fact, in

conjunction with Chapter 4, this chapter demonstrates the nexus between weighting

parameters and nuances of formulation variations that alter the range and magnitude of

objective-function values. This chapter is based on the work by Marler and Arora (in

press), and corresponds to objective 2 in section 1.4.


Despite the apparent necessity for function transformation or normalization, the

topic has not been addressed adequately. In fact, there are many examples in the

literature where function-transformations have erroneously been un-addressed. For

example, in some instances, methods are presented without transformed objective

functions, and normalization is simply recommended for objectives of different units

63

(Zeleny, 1982; Osyczka, 1984). In addition, some comprehensive treatments of MOO

present and use methods with no mention of function transformation (Chankong and

Haimes, 1983; Yu, 1985; Zeleny, 1982; Stadler and Dauer, 1992). This trend is

especially apparent with theoretical studies and with evaluations of specific methods (Yu,

1973; Yu and Leitmann, 1974; Freimer and Yu, 1976; Wierzbicki, 1980; Steuer and

Choo, 1983; Bendsoe et al, 1984; Ballestero and Romero, 1991; Kaliszewski, 1995;

Stadler, 1995; Das and Dennis, 1997; Tind and Wiecek, 1999; Messac et al, 2000a). In

addition, work concerning practical applications has also been conducted without

function transformations (Koski, 1985; Kassaimah et al, 1995; Saramago and Steffen,

1998, 2000; Zhao and Bai, 1999; Chen et al, 2003).

Although some transformation methods are mentioned in the literature, there is no

systematic study or evaluation of their advantages and disadvantages. There is no

indication as to why one approach or another should be used. In addition, when a

function-transformation is used, often the intent is simply to produce functions with

similar units; the complete range of potential function-values resulting from a

transformation is not considered.


Based on the motivation discussed above, the following specific objectives are

pursued:

1) Present and discuss various function transformation methods in terms of potential

numerical difficulties and in terms of imposed limits on function-values.

2) Evaluate these methods with respect to their ability to generate an accurate

representation of the Pareto optimal set using the weighted sum approach.

64

3) Provide guidance concerning the use of the weighted sum for a posteriori articulation

of preferences.

5.2 Transformation Methods

In this section, we discuss the various function transformations from a

mathematical perspective. The methods are classified according to the range of the

transformed function’s values. Although the weighted sum method is eventually used in

this study to depict the Pareto optimal set, the present analysis is applicable to any MOO

approach.

A common approach to function-transformation is incorporated in the following

three schemes, the most basic of which is given as (Chen et al, 1999; Koski and

Silvennoinen, 1987; Zhang and Yang, 2002):

( )( )trans i

i

i

FF

F=

xx

�

(5.1)

where ( ){ }minimumi i

F F= ∈

x

x x X� . We refer to this as the lower-bound approach, and

it provides a non-dimensional objective function. The lower limit of ( )trans

iF x is

restricted to negative one (it is positive one if 0i

F >�

), while the upper value is

unbounded. With this approach, division by zero (or a very small number) can lead to

numerical difficulties.

Equation (5.1) may be modified as follows (Osyczka, 1978; Hwang and Md.

Masud, 1979; Salukvadze, 1979; Eschenauer et al, 1990; Saravanos and Chamis, 1992;

Adali et al, 1995; Gupta and Sivakumar, 2002):

( )( )trans i i

i

i

F FF

F

−

=

x

x

�

�

(5.2)

65

Equation (5.2) is the alternate lower-bound approach, and it yields non-dimensional

objective function values with a lower limit of zero. As with (5.1), computational

difficulties can arise if the denominator is close to zero.

A second variation on (5.1) is given in (5.3), often in the context of the global

criterion method (Osyczka, 1984, 1985; Rao, 1996; Makaraci, 2000):

( )( )trans i i

i

i

F FF

F

−

=

x

x

�

�

(5.3)

However, even when absolute value signs are incorporated (Osyczka, 1984; Surdaki and

Montusiewicz, 1996), the motivation for this approach is unclear. Consequently, (5.3) is

not evaluated in this study.

As an alternative to the methods discussed above, one may use the maximum

value of the function in the denominator rather than i

F�

. The consequent upper-bound

approach is shown as follows (Proos et al, 2001):

( )max

trans i

i

i

FF

F=

x

(5.4)

where max

iF represents the maximum value for objective-i. Equation (5.4) provides a

non-dimensional function value such that 1trans

iF ≤ with no restriction on the lower

value. As with (5.1), (5.2), and (5.3), division by zero is possible though less common.

We call the following transformation approach the upper-lower-bound approach

(Koski, 1984; Koski and Silvennoinen, 1987; Rao and Freiheit, 1991; Yang et al, 1994):

( )max

trans i i

i

i i

F FF

F F

−

=

−

x�

�

(5.5)

In this case, ( )trans

iF x generally has a value between zero and one, depending on the

66

accuracy and the method with which ( )max

iF x and ( )

iF x

� are determined. Unlike

previous approaches, the denominator is guaranteed to be positive. In addition, this is the

only approach that constrains the upper and lower limits of ( )trans

iF x . Consequently,

(5.5) provides a relatively robust approach.

Rao (1987) and Rao et al (1988) use the following transformation scheme, which

is often called scaling:

( )trans

i i iF m F= x (5.6)

such that ( ) ( ) ( )1 1 2 2

constants s k k s

m F m F m F= = = =x x x�

im are scalar coefficients, and

sx is a feasible starting point. This approach ensures that

the objective functions have similar orders of magnitude but only at the point at which

the coefficients are determined. That is, this approach constrains the function values at

the starting point rather than at the function’s upper and/or lower limits. However, the

significance of the starting point may be arbitrary; its relevance to the feasible space may

be unknown. In addition, the appropriate value for the constant is not always clear and

may be arbitrary as well, although using a value of one can constitute a form of

normalization. Technically, (5.6) is similar to (5.1) and (5.4). Conceptually, however,

the coefficients in (5.1) and (5.4) have significance in terms of their range of potential

function-values. The coefficients are not as difficult to select as those in (5.6). For these

reasons, we do not consider (5.6) further.

5.2.1 Determining Function-Maxima

In reference to (5.4) and (5.5), max

iF may be determined as the absolute maximum

(if it exists) of ( )iF x , or as an approximation of the maximum. Technically, such an

67

approach limits the maximum of the transformed function to one. However, we are

concerned primarily with the objective function values at points within the Pareto optimal

set, not just within the feasible design space. Yet, there is no guarantee that the

individual maxima of the objective functions are even within the vicinity of the Pareto

optimal set. Therefore, when determining Pareto optimal points, the absolute maximum

of a function may be irrelevant.

Alternatively, an approach more conducive to MOO is to define max

iF such that

( )max

1

max *i i jj k

F F≤ ≤

= x , where *j

x is the point that minimizes the jth

objective function.

*j

x is a vertex of the Pareto optimal set in the design space, and ( )*jF x is a vertex of

the Pareto optimal set in the criterion space. We call such a maximum the Pareto-

maximum. It approximates the Nadir vector, which has components defined by the upper

bounds of the Pareto optimal set in the criterion space, but is difficult to determine

exactly (Korhonen et al, 1997). Using a Pareto-maximum does not necessarily eliminate

the possibility of a maximum transformed-function value other than one. This is because

non-Pareto optimal points may be encountered during the running of the optimization

algorithm. In addition, the vertices of the Pareto optimal set only provide approximate

bounds on function values within the set (Weistroffer, 1985). Nonetheless, using the

Pareto maxima rather than the absolute maxima can improve performance, as will be

demonstrated. This approach for determining max

iF , mentioned briefly by Miettinen

(1999), has been used with membership functions for fuzzy MOO (Dhingra et al, 1992;

Rao et al, 1992), and is included as a component of some methods (Chen et al, 1999;

Rao, 1987; Tabucanon, 1988). However, it has not been adopted consistently for

68

function transformation in the context of MOO. Consequently, its potential benefits in

terms of normalization have not been realized.

5.3 Analysis of the Methods

5.3.1 Problem Statement

Whereas the previous section provides a basic understanding and analysis of the

transformation methods, the following example problem, is used to evaluate the various

methods computationally:

Minimize:x

( ) ( )

( ) ( )

( ) ( )

2 2

1 1 2

22 2

2 1 2

4 2

3 1 2 1 2

25 0.5 2 2 0.1

2.5 4 1.8

2.0 1.5 2.8 0.3 10

F x x

F x x

F x x x x

= − + − +

= − + −

= − + − + +

(5.7)

subject to:

( ) ( )2

1 1 2

2 1

3 2

4 2

2.1 0.08 2.2 0

0

0

3.0 0

g x x

g x

g x

g x

= − − − ≤

= − ≤

= − ≤

= − ≤

The complete Pareto optimal set for this problem is shown in the design space in Figure

5.1, and in the criterion space in Figure 5.2.

The functions in this illustrative, constrained problem is designed to yield clearly

distinguishable sets of points in both the two-dimensional design space and the three-

dimensional criterion space, with some points resulting in active constraints. For each

function, the difference between the maximum and the Pareto-maximum is significant, as

shown in Table 5.1. In Table 5.1, each row shows the values of the objective functions

when one of the functions is minimized independently. For example, the first row

contains values of 1F ,

2F , and

3F calculated at the point

1*x , which minimizes

1F .

Each column contains the values of a function at different points.

69

0 1 2 3 4

-1

0

1

2

3

4 g1 g2

g3

g4

x1

x2

Pareto

Optimal Set

F1-minimum

F3-minimum

F2-minimum

Figure 5.1: Complete Pareto Optimal Set in the Design Space

20

40

60

20 30

40

F3

F2

F1

Pareto

Optimal Set

0 20

40 60

20

10 20 30 40

F3

F2

F1

Figure 5.2: Complete Pareto Optimal Set in the Criterion Space

70

For instance, the column of 1F -values contains values of

1F calculated at the point that

minimizes 1F , the point that minimizes

2F , and the point that minimizes

3F . The lightly

shaded boxes indicate the maximum value of a function, when that function is evaluated

at each of the three points. These values are the components of the Pareto-maximum.

The darker boxes indicate the minimum of each function, i.e., the utopia point. The last

row indicates the absolute maximum of each function.

Function Values1F -values

2F -values

3F -values

at 1*x 0.1000 43.0336 20.0725

at 2*x 67.6807 0.0224 12.6562

at 3*x 32.0687 16.9994 11.2757

Maximum 84.2615 144.2401 37.7600

Table 5.1: Function-Comparison Matrix for

Transformation Study

Clearly, the different functions compete with one another; what reduces one

function may increase the other two. No function has a minimum value of zero that

could result in an undefined value for a transformed function. Using this problem, we

demonstrate how solution points may cluster in certain regions, and we present new

criteria for anticipating this condition with certain transformation methods.

5.3.1 Unrestricted Weights

We use a convex combination of functions in this study, because using weights

with unrestricted values can result in an arbitrary selection process when determining

71

weights for depicting the Pareto optimal set. In fact, to highlight this potential difficulty,

we first demonstrate the consequences of not using a convex combination. Two of the

three weights (one for each objective function) are fixed at 1.0, while the other is varied

from zero to 1000.0 in increments of 20.0. Then, while one weight is fixed at 1.0, the

other two are increased from zero to 1000.0 in increments of 20.0. This results in 306

solution points. Using a weight of 1.0 essentially models the case when a function is

relatively insignificant but is still considered in the problem (i.e., does not have a weight

of zero). The resulting Pareto optimal points are shown in Figure 5.3. When these

results are compared with Figure 5.2, it is clear that the Pareto optimal set is not well

represented.

F3

F2

F1

0

20

40

60

20

10 20

30 40

Utopia

Point

F1-minimum

F3-minimum

F2-minimum

0 20

40 60

20

10 20 30 40

F3

F2

F1

Figure 5.3: Pareto Optima without Transformation and with Unrestricted Weights –

Criterion Space

Another example of this approach to selecting weights is shown in Figure 5.4. In

this case, it is assumed that a weighting vector of (50,50,50) indicates equal significance

72

between the objectives. The weights are varied around this point in the weight space.

Two of the three weights are fixed at 50.0, while the other is varied from zero to 100.0 in

increments of 2.0. Then, while just one weight is fixed at 50.0, the other two are

increased from zero to 100.0 in increments of 2.0. This results in 306 solution points.

Again, the Pareto optimal set is not well represented, as seen in Figure 5.4.

F3

F2

F1

0

20

40

60

20

10 20

30 40

Utopia

Point F1-minimum

F3-minimum

F2-minimum

0 20

40 60

20

10 20 30 40

F3

F2

F1

Figure 5.4: Pareto Optima without Transformation and with Unrestricted Weights

(Second Version) – Criterion Space

Although the weight-selection processes outlined above are somewhat arbitrary, it

is reasonable to expect such approaches if a convex combination is not used. Thus, we

demonstrate the potential for poor results. Since minimizing the weighted sum always

results in a Pareto optimal point, the solution points for the two preceding problems

collectively represent the Pareto optimal sets. However, the results clearly do not provide

complete representations of the Pareto optimal set. Of course, it is theoretically possible

to select weights with unrestricted values such that the complete Pareto optimal set is

73

represented. However, to do so would require a difficult trial-and-error process that one

can improve upon by using a convex combination of functions, as will now be

demonstrated.

5.3.3 Convex Combination of Functions

In this section, the results of different transformation methods are illustrated and

discussed. A series of weighted sum problems is solved using a convex combination of

objective functions, and each problem provides one point. The weights are incremented

as follows. The first of the three weights is incrementally increased from 0.0 to .33 to

.66. With the first weight fixed at a particular value, the other two weights are varied

simultaneously in increments of 0.025 (one is increased while the other is decreased)

such that a convex combination of functions is used. This process is repeated for each

weight. Then, while one of the three weights is increased from zero to one in increments

of 0.05, the other two weights, assigned the same two values, are reduced simultaneously

such that a convex combination of functions is used. This results in 309 solution points,

which is approximately the same number of points used in the two previous cases.

Technically, using a weight of zero can yield weakly Pareto optimal points. However,

such weighting values are used with the example problems in order to illustrate the

independent minima of the objective functions.

As a point of reference, the Pareto optimal solution points that are obtained when

no function transformation is used are shown in Figures 5.5 and 5.6. Although there is a

significant improvement compared to Figures 5.3 and 5.4, the solution points tend to

cluster in one area rather than spread out evenly over the entire Pareto optimal set. The

points in the design space appear to shift away from the minimum of 3

F , towards the

74

minima of 1F and

2F . This clustering of Pareto points is in part, a consequence of the

values for 1F and

2F shown in Table 5.1.

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

Figure 5.5: Pareto Optima without Transformation – Design Space

F3

F2

F1

0

20

40

60

20

10 20

30 40

Utopia

Point

F1-minimum

F3-minimum

F2-minimum

0 20

40 60

20

10 20 30 40

F3

F2

F1

Figure 5.6: Pareto Optima without Transformation – Criterion Space

75

Although 1F has a low minimum value and

2F has the highest ratio of Pareto-maximum

to minimum, at 2*x and at

3*x the value of

1F is relatively high, so

1F dominates the

weighted sum at these points. In addition, 1F has a relatively high average value, which

provides a general indication of how much a function dominates the weighted sum when

no transformation method is used. At 2*x ,

2F has the highest value of the three

functions, so it dominates the weighted sum at this point. 3

F has the lowest average

value, which suggests that even slight differences in the ranges of values for objective

functions (within the Pareto optimal set) can have a detrimental effect when the functions

are not transformed.

Theoretically, unrestricted weights can be used to compensate to some extent for

differences in function-values. However, as suggested earlier, using such weights in a

systematic fashion can be difficult. Thus, having resolved to use weights that yield a

convex combination of functions, the next issue is how to restrict the ranges and values of

the functions themselves.

The upper-bound approach for transformation, given in (5.4), acts to equalize the

upper bounds of the functions by dividing by the maximum of each function. This can

improve the distribution of the Pareto optimal points, as shown in Figure 5.7. However,

we are only interested in Pareto optimal points, and there is no guarantee that the point

that maximizes a function is close (in the design space or in the criterion space) to the

Pareto optimal set. For example, the absolute maximum of 2

F is 144.2401, whereas the

Pareto-maximum is 43.0336. Thus, it makes more sense to use the Pareto-maximum.

The results from such an approach, shown in Figure 5.8, reflect slight improvements.

76

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

Figure 5.7: Pareto Optima with Upper-Bound Approach and Absolute Maximum –

Design Space

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

Figure 5.8: Pareto Optima with Upper-Bound Approach and Pareto-Maximum – Design

Space

77

A similar increase in the dispersion of solution points occurs in the criterion space. Thus,

we show that the Pareto-maximum can provide better results than using the absolute

maximum, and this concurs with intuitive and theoretical analysis. In addition, there is

no guarantee that an absolute maximum even exists for every objective function.

Although the denominator is the function-minimum rather than the maximum, the

lower-bound approach given in (5.1) is conceptually similar to the upper-bound

approach. Both approaches essentially scale each objective function with a constant.

However, the results can be significantly different as demonstrated in Figures 5.9 and

5.10.

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

Figure 5.9: Pareto Optima with Lower-Bound Approach – Design Space

78

F3

F2

F1

0

20

40

60

20

10 20

30 40

Utopia

Point

F1-minimum

F3-minimum

F2-minimum

0 20

40 60

20

10 20 30 40

F3

F2

F1

Figure 5.10: Pareto Optima with Lower-Bound Approach – Criterion Space

The transformation actually worsens the results, and in this case, using the alternate

lower-bound approach in (5.2) yields similar consequences. 3

F seems to play an

insignificant role in governing which Pareto optimal solution points are provided. That

is, the results shown in Figures 5.9 and 5.10 are the same as they would be if 3

F were not

considered in the problem.

The results in Figures 5.5 through 5.8 can be improved upon and the degradation

shown in Figures 5.9 and 5.10 can be avoided by using the upper-lower-bound approach

given in (5.5), as shown in Figures 5.11 and 5.12. In conjunction with a Pareto-

maximum, (5.5) provides a reliable approach that consistently improves the distribution

of Pareto optimal solution points.

79

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

Figure 5.11: Pareto Optima with Upper-Lower-Bound Approach and Pareto-Maximum –

Design Space

F3

F2

F1

0

20

40

60

20

10 20

30 40

Utopia

Point F1-minimum

F3-minimum

F2-minimum

0 20

40 60

20

10 20 30 40

F3

F2

F1

Figure 5.12: Pareto Optima with Upper-Lower-Bound Approach and Pareto-Maximum –

Criterion Space

5.3.4 New Transformation Criteria

Despite the success of the upper-lower-bound approach, the question arises as to

why the degradation occurs with the lower-bound approach and when it can be

80

anticipated. In this section, we develop simple criteria that can be used to anticipate the

performance of the lower-bound and upper-bound transformation methods.

The above-mentioned difficulties may seem to surface because the minima of 1F

and 2

F are relatively small when compared to their respective Pareto-maxima. Thus,

when the functions are transformed using the lower-bound approach, the transformed

versions of 1F and

2F dominate the weighted sum, and

3F essentially becomes

irrelevant. This does, in fact occur, but it is not the only cause for difficulty. If this were

the only cause for difficulty, then the following ratio would indicate whether one

transformed function would dominate other transformed functions:

Pareto-maximum value

minimum valueρ = (5.8)

The values of (5.8) for each function would indicate whether using the lower-bound

approach would be effective or not. If ρ were relatively high for a particular objective

function(s), then the lower-bound approach would result in the above-mentioned

difficulties. However, if the ratio in (5.8) were always applicable, then one would expect

similar difficulties if the upper-bound approach were used with functions that have a

relatively large Pareto-maximum. 1 ρ would be used in place of ρ , and division by the

large number would result in relatively small values, supposedly indicating potential

difficulties. Yet, we see in Figures 5.7 and 5.8 that this is not necessarily the case. Is

there a unified process for explaining the results of both the lower-bound approach and

the upper-bound approach?

We demonstrate with additional examples that what dictates the success of the

lower-bound approach is the following ratio:

81

( ) ( )Pareto-maximum value minimum value

minimum valueξ

−

= (5.9)

Generally, if this ratio is similar for different objective functions, then using the lower-

bound approach will not be detrimental. We find that both the range of function-values

and the ratio between the Pareto-maximum and minimum values have an effect on

transformation results.

A similar metric is provided for the upper-bound approach (with the Pareto-

maximum), as follows:

( ) ( )Pareto-maximum value minimum value

Pareto-maximum valueς

−

= (5.10)

If the ratio in (5.10) is similar for different functions, then using the upper-bound

approach will not be detrimental.

The ratios given in (5.9) and (5.10) are calculated for this problem and shown in

Table 5.2.

ξ ς

1F 675.807 0.999

2F 1920.143 0.999

3F 0.780 0.439

Table 5.2: Ratio Values

Note that the values for ξ , which relate to the lower-bound approach, are significantly

different, thus explaining the poor results depicted in Figures 5.9 and 5.10. Although the

82

values for ς are similar, the value corresponding to 3

F is slightly lower than the other

two, explaining the acceptable, but less than ideal results shown in Figure 5.8.

The criteria involving the ratios in (5.9) and (5.10) are valid regardless of whether

a convex combination of functions is used (whether or not the values of the weights are

restricted), except for one caveat. When unrestricted weights are used, one of the reasons

for clustering is that there is no systematic approach for ensuring an even, consistent

sampling of the weight space in terms of the weights’ relative values. If such sampling is

carefully enforced, then a convex combination of functions is inadvertently used, with

which each function is multiplied by the same scalar that represents the largest common

denominator among the weights. Multiplying the weighted sum by such a scalar does not

affect the solution point, and the results are thus the same as with a convex combination

of functions.

5.4 Example Problems

5.4.1 Truss Problem 1

With this first example, which concerns the optimal design of the three-bar truss

shown in Figure 13, the advantages of the upper-lower-bound approach and the Pareto-

maximum are again demonstrated. In addition, whereas previous results illustrate the

clustering of solution points, this example demonstrates more severe potential

consequences when functions are not transformed. In Figure 13, 0.1L m= , the modulus

of elasticity 9 2200 10E N m= × , and the load

320 10F N= × . F is applied

simultaneously in both the negative x- and y-directions. This structural design problem is

a variation of the problem presented by Koski (1985), and Koski and Silvennoinen

(1987).

83

L

L L 3

1 2

3

x

y

F

F

Figure 5.13: Three-bar Truss

The areas of the members (in units of 2

m ) are the design variables. The total volume of

the truss and the stress in member 1 are the objectives, both of which are minimized.

Member 1 is considered because it is most likely in compression, and thus subject to

buckling. Limits are placed on the design variables. The MOO problem is formulated as

follows:

1 2 3, ,

Minimize:A A A

2

1 1

2 1 2 3

stress in member 1

total volume 2 2

F

F A L A L A L

σ= =

= = + +

(5.11)

subject to: 0.0004 0.002; 1,2,3i

A i≤ ≤ =

Although one is typically concerned with the absolute value of stress, 2

σ is minimized to

avoid computational difficulties.

Table 5.3, which is similar to Table 5.1, shows the values of the objective

functions when each function is minimized independently.

84

Function Values (Stress)2-Values Volume-Values

at s*A 2.9504E-05 6.57550E-04

at v*A 3.13822E+14 1.76569E-04

Maximum 8.93531E+14 8.82843E-04

Table 5.3: Function-Comparison Matrix for Truss

Problem 1

The vector s*A is the point in the design space that minimizes stress, and

v*A is the

point that minimizes volume. In this case, the two different functions have substantially

different orders of magnitude and range. However, the absolute maxima are not

significantly different from the Pareto-maxima, as was the case previously.

A series of weighted sum problems is solved using a convex combination of

objective functions. The weight for stress is increased from zero to one in increments of

0.02, while the weight for volume is decreased in the same increment.

When no function transformation is used, the only two resulting solution points

are the individual minima for each objective function, because stress dominates the

weighted sum. Consequently, the minimum stress is found for all sets of weights until w1

is zero, in which case the minimum volume is found. Note that the minimum value for

the stress in member 1 is approximately zero, which occurs when the areas of the three

members are such that the displacement of the node where the loads are applied is

perpendicular to element 1. The minimum volume is achieved at the point where all

areas are at their lower limit, i.e., 1 2 3

0.0004A A A= = = . The same results are obtained

when the lower-bound and alternate lower-bound approaches are used.

85

These results can be anticipated using the values for the ratios given in (5.9) and

(5.10), as shown in Table 5.4. ξ is 1.06366E+19 for stress and 2.72404 for volume,

which clearly indicates the dominance of stress. These results can have severe

consequences, as they imply that there are only two Pareto optimal designs. However,

the use of proper transformation schemes can alleviate this misleading difficulty.

ξ ς

Stress

1.06366E+19 1.0

Volume

2.72404 0.73147

Table 5.4: Ratio Values for Truss

Problem 1

In reference to the upper-bound approach (with a Pareto-maximum), ς is 1.0 for

stress and 0.73147 for volume. These values are similar, and consequently, results are

improved, as shown in Figures 5.14 and 5.15. Note that although the maximum value for

stress is relatively large, using it as a denominator does not yield detrimental results. The

Pareto optimal solution is a function of 3

A primarily, as shown in Figure 5.14. For most

of the trials, 1A and

2A have a shared value of 0.0004 (their lower limit), thus reducing

the volume. 3A varies between 0.0004 and 0.002 and redirects the consequent

displacement, thus changing the value of stress in member 1.

86

A2

A1

A3

Stress-

minimum

Volume-

minimum

0.00053

0.0015

0

0.001

0.002

0.00053

0.0015

Figure 5.14: Truss Problem 1, Pareto Optima with Upper-Bound Approach and Pareto-

Maximum – Design Space

0.0E+00

1.0E-04

2.0E-04

3.0E-04

4.0E-04

5.0E-04

6.0E-04

7.0E-04

0.0E+00 5.0E+06 1.0E+07 1.5E+07 2.0E+07

Stress

Volume

Figure 5.15: Truss Problem 1, Pareto Optima with Upper-Bound Approach and Pareto-

Maximum – Criterion Space

87

The gap between the minimum point for stress and the rest of the Pareto optimal

set is a consequence of an extremely high slope for the Pareto optimal set in the criterion

space. If additional weighted sum problems are solved with the increment in the weights

reduced to the order of 61 10

−

× , then solution points begin to fill this gap.

Further improvements in the distribution of Pareto optimal points are achieved by

using the upper-lower-bound approach with the Pareto-maximum.

5.4.2 Truss Problem 2

With this second example, the significance of (5.9) and (5.10) are clearly

illustrated. In addition, the importance of restricting both upper and lower bounds of a

transformed function is illustrated. Although this problem is similar to Problem 1, here

the stress in member 3 is minimized, rather than the stress in member 1. The stress in

member 3 is objective function 1, and the volume is objective function 2. With the given

loading condition, there is no chance that member 3 is subject to compression.

Consequently, the stress values for this member are always positive, and are not squared

as they are in Problem 1.

Table 5.5 shows the values of the objective functions when each function is

minimized independently.

Function Values Stress-Values Volume-Values

at s*A 8.65417E+06 8.82843E-04

at v*A 4.32708E+07 1.76569E-04

Maximum 4.32708E+07 8.82843E-04

Table 5.5: Function-Comparison Matrix for Truss

Problem 2

88

As with Problem 1, the two different functions have substantially different orders of

magnitude. However, in this case, the range of values for stress is significantly smaller

than for Problem 1. In addition, the absolute maxima are identical to the Pareto-maxima.

When no transformation is used, results are similar to those in Problem 1; there

are only two solution points, and they represent the individual minima of the two

functions. However, in contrast to Problem 1, results with the lower-bound approach in

(5.1), the alternate lower-bound approach in (5.2), and the upper-bound approach in (5.3)

all are nearly identical and are illustrated in Figures 5.16 and 5.17.

A2

A1

A3

Stress-minimum

Volume-

minimum

0.00053

0.0015

0

0.001

0.002

0.00053

0.0015

Figure 5.16: Truss Problem 2, Pareto Optima with Upper-Bound Approach – Design

Space

89

0.0E+00

1.0E-04

2.0E-04

3.0E-04

4.0E-04

5.0E-04

6.0E-04

7.0E-04

8.0E-04

9.0E-04

1.0E-03

0.E+00 1.E+07 2.E+07 3.E+07 4.E+07 5.E+07

Stress

Volume

Figure 5.17: Truss Problem 2, Pareto Optima with Upper-Bound Approach – Criterion

Space

One can anticipate these results using the ratios from (5.9) and (5.10), as shown in Table

5.6. The values in Table 5.6 suggest that either the lower-bound or upper-bound

approach may be used to improve original results, and this prediction is substantiated by

the results.

ξ ς

Stress

4.0 0.8

Volume

4.0 0.8

Table 5.6: Ratio Values

for Truss Problem 2

Using the upper-lower-bound approach in (5.5) yields results similar to those

provided in Figures 5.16 and 5.17. In fact, with this particular problem, every

90

transformation method produces results in which the minimum value for the transformed

version of stress is the same as the minimum value for the transformed version of

volume. Concurrently, the maximum value for the transformed version of stress is the

same as the maximum value for the transformed version of volume. In addition, as is

always the case when there are only two objectives, the minimum of one function

corresponds to the Pareto-maximum of the other function, and visa versa. Consequently,

it is not just the upper-lower-bound approach but also every approach that constrains the

upper and lower limits of the functions. Although this can occur naturally with certain

problems, using the upper-lower-bound approach with a Pareto-maximum ensures that it

occurs regardless of the nature of the problem. Thus, although using the proposed

normalization approach is not necessary for improving results, it is sufficient. In other

words, the upper-lower-bound approach is not always required for acceptable results,

but it provides a safeguard against certain potential difficulties.

Figure 5.16 is similar to Figure 5.14 (for Problem 1) in that much of the Pareto

optimal curve is a function of 3A . However, in this case, once

3A reaches its upper limit

of 0.002, then 2

A is forced to increase. Eventually 1A increases until all areas reach their

upper limits and stress is minimized.


In this chapter, we have provided an analysis of methods for transforming

objective functions in a MOO problem, and we have demonstrated the significant effect

that various transformation methods can have. We have introduced and exploited the

idea of using transformation methods to limit the potential values of functions in different

ways. In general, we have shown that function transformation is beneficial. In particular,

91

we have shown that the use of the upper-lower-bound approach in (5.5), along with the

Pareto-maximum, is robust and ensures that the transformed functions have similar

orders-of-magnitude and ranges. This can help prevent any single objective from

dominating an aggregated function. We have shown that some transformation methods

can actually worsen results, and we have provided new criteria for determining when

using such methods is detrimental. Although the performance of MOO methods depends

on more than just relative function-values, we have shown that proper function-

transformation is one way to improve performance.

In Chapter 4, it was shown that the solution resulting from a particular set of

weights, can depend on four factors: the relative function values, which can be altered

using transformations; the shape of the Pareto optimal set in the criterion space; the active

constraints; and the relationship between function gradients. The clustering of points is

tied to these issues as well. However, discussing each of these facets is beyond the scope

of this study. Rather, we have focused on the first issue: the significance of function

values and transformation methods.

Note that using transformations alone to eliminate the potential for the numerical

dominance of a function is not always sufficient for ensuring acceptable results. Rather,

one must eliminate the potential for numerical dominance in any one of the terms, i i

w f ,

in (4.1). As indicated in Chapter 4, the value of a weight is significant relative to the

values of the other weights and relative to the value of its corresponding objective

function. Along these lines, we have demonstrated that one achieves superior results for

a posteriori articulation of preferences when a convex combination of normalized

objective functions is used. This is because using normalized objective functions ensures

92

that the transformed functions have the same range of values as the weights and allows

one to use the weights to control the relative size of each term in the weighted sum.

Using a convex combination ensures that the weight space is sampled consistently.

In considering the terms, i i

w F , it is apparent that there is subtle distinction

between using weights to transform functions and using weights to indicate preferences

(or to depict the Pareto optimal set). This idea is touched on indirectly by Hunt et al

(2004), and we elaborate on it here. In (5.6), weights are used as a transformation

method to change the nature of i

F , and as suggested, they are relatively ineffective in this

capacity. Thus, throughout this study, the weights have been treated somewhat

independently from the objective functions. To clarify this distinction, we could consider

a weighted sum formulated as follows:

( )1

k

i i i

i

U w Fγ

=

= ∑ x (5.12)

where i

w provide the kind of weight we have discussed in reference to the weighted sum

and i

γ represent separate weights used for transformation. Obviously, these two

numbers can be multiplied together to provide a single coefficient. However, these two

types of weights have separate roles, and thus, they should be considered separately.

This is a key conceptual issue that is often overlooked with casual use of the weighted

sum method.

The findings in this study are significant with respect to other MOO methods, not

just the weighted sum method. As we have shown, transformation methods are

significant in their ability to affect the weighted terms in (4.1), and these terms can

appear in many different scalarization methods. In addition, transformation methods can

93

be viewed not just as a means to alter objective functions but also as means to modify a

multi-objective formulation, as will be seen in Chapter 6.

Conclusions based on the present work are presented as follows:

1) Although using unconstrained weights (with no limits on their values) can be

convenient for a priori articulation of preferences as discussed in Chapter 4, it results

in an arbitrary process for a posteriori articulation of preferences (depicting the

complete Pareto optimal set). Using a convex combination of functions is

advantageous in the latter case.

2) Using the minimum or maximum of a function as the denominator in a transformation

scheme can result in numerical difficulties and/or poor performance in terms of

depicting the complete Pareto optimal set, as predicted by the ratios in (5.9) and

(5.10).

3) When using the upper-bound approach or the upper-lower-bound approach, one

should use the Pareto-maximum rather than the absolute maximum.

4) Using the upper-lower-bound approach can improve the performance of the weighted

sum method, and using this approach with a Pareto-maximum provides superior

results. This is the most robust approach to function transformation, and is

recommended in conjunction with a convex combination of weights.

94

CHAPTER VI

GLOBAL CRITERION APPROACH

6.1 Introduction

In this chapter, an in-depth analysis of the global criterion approach and its many

variants, including the min-max approach, is provided. Whereas the weighted sum

method, discussed in Chapters 4 and 5, involves only one relatively simple formulation

and only one type of method-parameter, the global criterion approach encompasses many

subtly different formulations and potentially involves three types of method-parameters.

Independent uses of variations on the approach appear throughout the current literature,

but there is no comprehensive investigation concerning the different facets of the

approach. Here, we take a holistic look at the approach in order to contrast the different

formulations and in order to determine what effects each type of parameter has on the

solution.

New guidelines are provided for indicating when one formulation or another

should be used. In addition, we present a unique study of the significance of the method-

parameters, which considers the complete range of potential values for the parameters

rather than just the traditional values. In conjunction with Chapter 7, this study resolves

the confusion surrounding how one should set multiple types of method-parameters

concurrently. We provide new criteria for predicting the nature of the solution (Pareto

optimality and regularity) to the min-max approach. Finally, we propose a modified

global criterion approach that enables one to approximate min-max results with fewer

constraints, fewer design variables, and a reduced potential for computational difficulties.

95

In addition, it allows one to vary the value of the exponent on a continuous scale, thus

allowing one to use the exponent to articulate preferences more effectively. As with

Chapters 4 and 5, this chapter responds to objective 2 in section 1.4, which involves the

investigation of method-parameters and alternate formulations for a given method.

The global criterion approach is a common, general scalarization approach that

encompasses a few different methods; so understanding this approach provides insight

into additional methods. Conclusions are drawn that sort out the many different facets of

the global criterion approach.

6.1.1 Overview of the Formulations

In general, a global criterion is an amalgamation of the objective functions,

forming a single function ( ) 1: R

gF → F x Z . Technically, the term global criterion can

refer to any scalarized function, but it is often reserved for the formulations in this

chapter.

The first formulation is given as follows (Osyczka, 1984; Eschenauer et al, 1990;

Miettinen, 1999):

Find: ∈x X (6.1)

to minimize: ( ) ( )

1

1

;

k pp

g i i i i

i

F Q Q F z=

= = − ∑x x

where p is a constant, typically between one and infinity. z is called a reference point

(Wierzbicki, 1980, 1982; Lewandowski and Wierzbicki, 1989) or an aspiration point

(Wierzbicki, 1986; Miettinen, 1999), and it represents either a goal in the criterion space

or an approximation of the utopia point F� . Much of the literature uses =z F� and does

not consider the use of an aspiration point, because F� provides the ideal goal. In such

96

cases, the approach is referred to as compromise programming (Yu, 1973; Zeleny, 1973),

as the decision-maker usually has to compromise between the final solution and the

desired outcome. When an aspiration point is used in place of F� , ( )gF x is often called

an achievement function. Achievement function methods may be distinguished from

compromise programming, because a reference point is used as a variable method-

parameter rather than as a goal. Formulations are presented here using z, simply to

represent a general criterion.

Often, the absolute-value sign in (6.1) is omitted as follows, especially when

=z F� , since ( )≤F F x

� (Stadler, 1988; Stadler and Dauer, 1992):

( )

1

1

k pp

g i

i

F Q=

= ∑x (6.2)

We refer to this formulation as the root global criterion. In addition, the global criterion

may be expressed without a root, to provide the following basic global criterion:

( )

1

kp

g i

i

F Q=

=∑x (6.3)

The criterion in (6.1) represents the Lp-norm, and the minimum solution (in the design

space and the criterion space) can vary depending on the value of p (Yu and Leitmann,

1974; Koski and Silvennoinen, 1987). Although p can vary without restriction, in the

literature it usually has a value of one, two, or infinity. Goal programming, which is

discussed in Marler and Arora (2003), is based on the use of an achievement function,

typically in the form of (6.1), with 1p = . When 1p = , (6.2) and (6.3) result in a simple

sum of the objective functions. Formulations with 2p = (characteristic of what are

called distance functions) represent the idea of Euclidean distance in the criterion space,

97

and they can be used to determine the point that is as close as possible to the aspiration

point (or utopia point). However, as suggested in section 2.6, there is no reason to

assume the idea of closeness corresponds to the Euclidean norm, especially if the values

of the different objective functions have different units (Vincent, 1983). Consequently,

one can view the summation arguments in global criteria, in two ways. Mathematically,

they simply represent modified objective functions. Conceptually, however, they

represent components of a distance function that minimizes the distance between the

solution point and z.

We call the formulations in (6.1) through (6.3) standard global criterion

formulations. Alternatively, when p = ∞ , (6.1) represents the L∞-norm, which is a

special case of the global criterion and has the following format:

( )

1

maxg i

i kF Q

≤ ≤

=x (6.4)

(6.4) is called the min-max formulation. Again, the absolute value signs may be excluded

(Yu, 1973; Zeleny, 1982; Miettinen, 1999). Given that (6.4) is often discontinuous, the

following formulation, in which the additional unknown parameter λ is introduced, has

become popular (Bendsoe et al, 1984; Eschenauer et al, 1990):

Find: , λ ∈x X (6.5)

to minimize: λ

subject to: ( ) 0 for 1,2, ,i

Q i kλ− ≤ =x …

We refer to this as the alternate min-max formulation. The additional constraints shown

in (6.5) are called function constraints. At the solution point, maxi

i

Qλ = .

98

Technically, any global criterion is a utility function; it is used to approximate the

decision-maker’s preference function. In this respect, the p-exponent and the aspiration

point provide parameters that can be used to design or customize the utility function. In

addition, each component of the global criteria can involve a weight that loosely

represents the relative significance of each objective (Zeleny, 1982; Chankong and

Haimes, 1983; Yu, 1985; Wierzbicki, 1986; Miettinen, 1999). This is especially

prevalent with the min-max formulation, which provides a necessary condition for Pareto

optimality when the weighting vector is altered systematically (Bowman, 1976; Bendsoe

et al, 1984; Romero et al, 1998; and others). However, in order to simplify method-

analysis, this chapter considers the first two parameters only, and thorough consideration

of a weighting vector is reserved for Chapter 7. Nonetheless, some findings in this study

have significance when applied to weighted formulations, and such observations are

noted.

As discussed in Chapter 4, when different objective functions have significantly

different orders of magnitude, it becomes necessary to normalize or otherwise modify the

objective functions such that each function contributes significantly to the value of the

global criterion. Thus, the transformation methods in Chapter 4 may be applied to global

criteria. For instance, assuming =z F� , the following form may be used:

( )i i

i

i

F FQ

F

−

=

x�

�

(6.6)

(6.6) has been used with (6.2) (Salukvadze, 1979; and Koski and Silvennoinen, 1987,

Adali et al, 1995; Gupta and Sivakumar, 2002) and with (6.3) (Hwang and Md. Masud,

1979; Rao, 1996; Makaraci, 2000; Gupta and Sivakumar, 2002). The following form

99

also may be implemented and is based on the transformation scheme in (4.5) (Yang et al,

1994):

( )max

i i

i

i i

F FQ

F F

−

=

−

x�

�

(6.7)

The modifications shown in (6.6) and (6.7) may be applied to the min-max formulation as

well (Osyczka, 1978, 1981, 1984; Koski, 1984; Eschenauer et al, 1990).

In the context of global criteria, theoretically and conceptually, function

transformation is not necessary if weights are not used. Computationally, however,

prophylactic use of normalization is advantageous; it prevents any single objective

function from dominating the global criterion. In this study, for the sake of general

analysis, it is not assumed that objective functions are normalized. However, scenarios

are highlighted in which normalization has a particularly significant effect on

performance. We address issues concerning computational problems with normalization

unique to global criteria, which have not been addressed previously.

As suggested in section 5.5, there is a subtle distinction between the use of

different transformed objective functions and different multi-objective formulations. In

this study, in accordance with the general trend in the literature, we view different forms

of i

Q as different global criteria rather than as transformed objective functions.

Regardless of the exact formulation, global criteria usually incorporate the idea of

distance or difference between the solution point and the aspiration point (or utopia

point).

100


6.1.2.1 Standard Global Criterion Formulations

The global criterion was introduced as a solution concept, referred to as a

compromise solution, which is discussed in section 2.5 (Salukvadze; 1971a, 1971b). It

provides a means to determine a point that is as close as possible to the utopia point.

However, no discussion is given concerning the characteristics of the method.

Yu (1973) and Zeleny (1973, 1982) provide the first formal development of

compromise programming, in which the exponent p is treated as a method-parameter that

is varied between one and infinity, yielding a series of different Pareto solutions. Yu

(1973) presents compromise programming as a means to solve problems involving a

group of people, with each person having a separate, explicit utility function.

Mathematical and economic properties of compromise solutions are discussed. These

types of solutions are posed as being more easily understood and more easily obtained

than solutions typically associated with group decision-theory or game theory, which

provide means of modeling groups of decision-makers. With =z F� , (6.2) is said to

represent group regret, where ( )i iF F−x

� is the regret for individual-i. Using 1p =

provides the sum of all individual regret, and p = ∞ yields the maximum regret. Yu and

Leitmann (1974) also consider multiple decision-makers and suggest that as the value for

p increases, the significance of group utility (the sum of the utility functions) is balanced

with individual regret.

Koski and Silvennoinen (1987) briefly discuss the use of (6.1) as a basis for

comparison with a newly developed variation on the weighted sum method. They present

101

a visual interpretation of the global criterion in the criterion space using values of one,

two, and infinity for p. The authors then use (6.1) to solve a three-bar truss problem.

Proos et al (2001) use (6.2) to minimize compliance and maximize the first mode

of the natural frequency, with topology optimization of two-dimensional plane stress

problems. However, the results provide a test of a new approach to topology

optimization rather than a study of MOO.

Makaraci (2000) applies (6.3) to a general mass-spring-dashpot system.

However, little discussion is provided concerning MOO or the performance of the global

criterion approach relative to other approaches. The focus is on the problem being

solved.

Yu (1985) and Wierzbicki (1986) provide comparisons between achievement

functions and utopia-point functions. However, the discussion by Yu (1985) is brief, and

Wierzbicki (1986) focuses on mathematical aspects in the framework of a broad review

of MOO methods. The latter work is thorough in terms of the fundamental formulations

that are addressed, but practical conclusions are not extracted.

Messac et al (2000a, 2000b), and Messac and Yahaya (2001) provide insightful

discussions of general global criteria, investigating relationships between the criteria and

the Pareto optimal set. It is concluded that the most effective format should allow the

user to adjust the curvature of the criteria (in the criterion space) in order to provide a

necessary condition for Pareto optimality even with nonconvex Pareto sets. Basic

example problems focus on (6.3) and on the pitfalls of the weighted sum method.

Generally, the literature concerning the analysis of the global criterion approach is

limited and tends to be of a theoretical nature. Other literature tends to focus on the

102

application of the method rather than on the practical characteristics of the method itself.

There is little analysis of the pros and cons of methods, and there is no direct comparison

between the different global criterion formulations in terms of computational robustness

or in terms of their ability to yield Pareto optimal solutions. In addition, because of their

clear physical meaning, p-values of one, two, and infinity are used most often. However,

p has mathematical and preferential significance, valid for a continuous range of values.

Yet, this significance is not thoroughly investigated nor are the characteristics of using p-

values other than one, two, and infinity.

6.1.2.2 Min-max Formulations

Osyczka (1978, 1981, 1984, 1989) provides some of the earliest work with the

min-max approach and uses (6.4) to develop min-max principal of optimality, which is

discussed in Marler and Arora (2003). Osyczka (1978) approaches (6.4) as a standard,

single-objective problem, where ( )maxi i

i

F z−x provides the objective function values at

the point x . Although he considers an integer problem with a finite set of solutions,

Osyczka (1978) uses (6.4) in the context of the min-max principal of optimality to solve

problems involving the design of a machine gear toolbox. Osyczka (1981) takes a similar

approach with simple structural members. Tseng and Lu (1990) incorporate Osyczka’s

approach with a ten-member cantilever truss, a twenty-five-member transmission tower,

and a two-hundred-member plane truss, all of which are detailed by Haug and Arora

(1979).

Romero et al (1998) give a thorough comparison of variations on the formulation

in (6.5), in an effort to extract commonalties between goal programming, compromise

programming, and reference point methods. The authors contend that these three

103

approaches are equivalent when the utopia point is used. In fact, it is suggested that

differences between compromise programming and goal programming are more

philosophical than analytical. However, despite the title of the work, the authors focus

primarily on difficulties with the augmented Tchebycheff approach (a variation of the

min-max approach, which is discussed by Marler and Arora (2003)) and present

alternative lexicographic methods. Findings are demonstrated with a simple

mathematical problem.

In summary, there is little discussion in the literature concerning the basic min-

max approach as it relates to the global criterion from which it is derived. In addition,

there is no investigation into the repercussions of sing additional constraints with eth

alternate min-max formulation.


The following specific issues are addressed with this study:

1) The different global criterion formulations, including the alternate min-max

formulation in (6.5), are studied in terms of their ability to yield Pareto optimal

solutions. Although the root global criterion and the basic global criterion

theoretically provide the same solution, why one formulation may be more

advantageous than the other one, is investigated.

2) The use of a utopia point is contrasted with the use of an aspiration point.

3) The significance of the p-exponent is explored, and an infinite range of values is

considered.

104

4) Special characteristics of the alternate min-max formulation, concerning irregularity

and Kuhn-Tucker conditions, are revealed. Unique benefits of using normalization

with this formulation are uncovered.

5) The alternate min-max formulation requires the use of additional constraints, which

may require additional gradient evaluation and associated computational demands.

Consequently, the potential for approximating min-max results with a modified

standard global criterion and a relatively large value for p, is evaluated. A modified

normalization scheme is suggested and tested for improved performance.

6.2 Analysis of the Standard Global Criterion

Formulations

In this section, the root global criterion and the basic global criterion are studied

in terms of what is required in order to guarantee a Pareto optimal solution and in terms

of computational obstacles. Two critical theorems are used to uncover practical

properties of the different formulations. Values of p such that 1 p≤ < ∞ , are considered

here, whereas the cases with 1p < and p = ∞ (min-max formulation) are discussed later,

independently.

If 1 p≤ < ∞ and =z F� , assuming F� is unattainable, then minimizing any of the

criteria discussed thus far provides a sufficient condition for Pareto optimality (Chankong

and Haimes, 1983). The proof for this condition can easily be extended to show that the

global criteria also provide a sufficient condition for Pareto optimality if ≤z F� . In

addition, under certain conditions, achievement functions can provide a necessary

condition for Pareto optimality with consistent variation in the aspiration point.

105

However, as discussed in Marler and Arora (2003), their use in this capacity is rare and

can be cumbersome.

Although (6.1) is common, having evolved from the idea of an Lp-norm, it has

some pitfalls that preclude its consideration in this study. First, the absolute value signs

may result in difficulty with differentiation, and may provide a stumbling point for most

gradient-based optimization engines. Secondly, if the utopia point is determined

inaccurately or is overly expensive to compute, consequent use of an aspiration point can

elicit difficulties.

For instance, if ∈z Z is not Pareto optimal (if it is not on the boundary of Z), then

the solution to the global criterion in (6.1) cannot be Pareto optimal. This is because if z

is attainable, then it is necessarily the minimum of (6.1) in the criterion space. If on the

other hand, ∈z Z is Pareto optimal, then the solution is Pareto optimal. However, the

capability of predetermining whether or not a point is Pareto optimal defeats the purpose

of minimizing the global criterion. In addition, g

F∇F

is undefined in the criterion space

where =F z . Consequently, Theorem 2.2, which requires that the global criterion be

differentiable with

gF∇ > ∀ ∈

F0 F Z , suggests that (6.1) may not be sufficient for Pareto

optimality.

If ∉z Z , then Theorem 6.1 characterizes the solutions to (6.1) (Wierzbicki,

1986).

Theorem 6.1 Let k

p E∈F be a Pareto optimal point in the criterion space.

Let ( ),g

F F z be an achievement function formulated as shown in (6.1),

106

with 1 p< < ∞ . Let either Z be convex or p

≤z F . If F minimizes

( ),g

F F z over ∈F Z and >F z , then F is properly Pareto optimal. If F

is properly Pareto optimal, then there exists a <z F such that F

minimizes ( ),g

F F z over ∈F Z .

The implications of Theorem 6.1 are discussed as follows. To guarantee that the

conditions ∈F Z and >F z are satisfied, one must determine z such that <z F� , or one

must incorporate additional constraints such that ( ) >F x z . Then, the theorem suggests

that minimizing an achievement function like (6.1) provides a sufficient condition for

proper Pareto optimality (which entails Pareto optimality) if a Pareto optimal point p

F

exists such that p

<z F or if Z is convex. The former is not a difficult condition to satisfy

if z is set to approximate F� . In addition, convex criterion spaces are common, especially

with unconstrained problems. Nonetheless, using (6.1) opens the door to potentially non-

Pareto optimal solutions, depending on z. Consequently, the root global criterion in (6.2)

and the basic global criterion in (6.3) are left as the preferred criteria; they both can

circumvent complications when used correctly.

Still, there are some pitfalls with these criteria as well, which are revealed using

Theorem 2.2 and the gradient of the criteria in criterion space. In addition, although the

two formulations theoretically yield the same solution point, they have significant

differences, which are discussed throughout this study. The gradient for the root global

criterion is given as follows:

107

11

1

11

pp

k pp

g i

i p

k

Q

F Q

Q

−

−

=−

∇ =

∑F� (6.8)

Recall that we have already determined that this criterion yields a Pareto optimal solution

if ≤z F� . If >z F

� , it is possible to ensure that

gF∇ > ∀ ∈

F0 F Z by restricting p to odd

values. As stipulated in Theorem 2.2, this in turn ensures Pareto optimal solutions.

Generally, the exact value selected for p is somewhat arbitrary, so there is no detriment to

choosing an odd value. However, if >z F� , it is possible that a component of Q becomes

negative, which would result in an irrational value for (6.8) if an odd p-value were used.

Such difficulties with g

F∇F

can result in potential computational difficulties, considering

that [ ]T

g gF F∇ = ∇x F

J , where [ ]J is the Jacobian matrix in which each row represents

the gradient of each objective function. If components of g

F∇F

are irrational or

intractable, then so are components of g

F∇x

, which is a key component of gradient-based

optimization engines. In summary, although one can take steps such that the root global

criterion provides a sufficient condition for Pareto optimality regardless of the value for

z, computational difficulties can arise.

The gradient for the basic global criterion does not present these potential

difficulties. It is given as follows:

1

1

1

p

g

p

k

Q

F p

Q

−

−

∇ =

F� (6.9)

108

In this case again, if >z F� , ( )g

F∇ > ∀ ∈F

F 0 F Z as long as p is an odd number. This

ensures Pareto optimality regardless of the nature of z, and there is no risk of numerical

difficulty.

Regardless of which formulation is used, incorporating the utopia point is

advantageous. However, for cases when the utopia point is unavailable, the basic global

criterion in (6.3) provides the most suitable formulation for ensuring Pareto optimality

with minimal potential for computational difficulties.

6.3 Significance of the p-exponent

In addition to the different overall formulations and the difference between using

a utopia point and using a reference point, which are discussed in the previous section,

the actual value of p also can affect the solution obtained when using the global criterion

approach. It may be treated purely as a mathematical parameter or as a device to

approximate the preference function more accurately. In this section, the value of p is

discussed in these capacities. Furthermore, its relationship to different function

transformation schemes is studied.

For the sake of thorough, general analysis, a continuous range of p-values is

considered. Because p may be used as a heuristic parameter, it should not be limited to

the usual discrete values of one, two, and infinity. References to relatively large values

of p are significant in terms of using a global criterion to approximate min-max results.

Much of this section concerns the nature of a global criterion as a single

scalarized objective function, in the context of the criterion space. In this context,

analysis is relevant to both constrained and unconstrained problems, as constraints are

109

implicit in the feasible criterion space and in the Pareto optimal set, as mentioned in

section 4.2.2 with reference to Figure 4.1.

6.3.1 Mathematical Significance

In this section, we study the effect that increasing the value of p has on the

different global criteria. Although the current literature touches on the basics of this

issue, to date, a rigorous study in terms of the different formulations has not been

provided. In addition, we demonstrate the potential for using a standard global criterion

to approximate min-max results, by incorporating a relatively high value for p. In doing

so, we set the groundwork for a modified global criterion method. Later in the chapter,

this method is developed further and tested.

6.3.1.1 Curvature

Changing the value of p changes the curvature of the global criterion contours in

the criterion space (Athan and Papalambros, 1996; Messac et al, 2000a, 2000b).

Although the literature does not explicitly define curvature in reference to this topic, we

assume that Gaussian curvature is used. It is a common metric for curvature in n-

dimensional space and is determined as the product of the principal curvatures, which are

approximately equal to the eigenvalues of the Hessian matrix (matrix of second

derivatives with respect to F). The components of the Hessian are determined by

differentiating the components of the gradients shown in (6.8) and (6.9), with respect to

F. The Hessian for the basic global criterion is a diagonal matrix, and thus, each

component represents an eigenvalue directly. Each component is necessarily non-

negative for all 1 p< < ∞ assuming that ≥Q 0 and that ≠Q 0 . Consequently, the

Hessian is always positive semi-definite. Given the same assumptions, the diagonal

110

elements for the Hessian for the root global criterion are always non-negative, and the

off-diagonal terms are always non-positive.

The curvature of the global criterion function depends on the point in the criterion

space at which it is evaluated, and it is not necessarily constant. Increasing p increases

the curvature of the global criterion contours in the neighborhood of the point where all

of the components of Q are equal. Ultimately, the position of this point can be varied by

using different weighting vectors with the weighted global criterion approach, and thus,

providing different Pareto optimal solution points. Increasing p decreases the curvature

of the contours at all other points.

The curvature of the global criterion is relevant for two reasons. First, it relates to

the method’s ability to yield all of the Pareto optimal solution points when weights are

incorporated. In fact, Messac et al (2000a, 2000b) show that in general, a scalarized

objective function is more effective in yielding all of the Pareto optimal points (providing

a necessary condition), if its curvature can be increased at the above-mentioned point by

adjusting method-parameters such as p. This is demonstrated by comparing the basic

global criterion with the objective sum method (or weighted sum method) for which

curvature is always zero. Messac and Yahaya (2001) arrive at a similar conclusion

concerning the relevance of p, although they do not directly refer to curvature. Athan and

Papalambros (1996) prove that minimizing a weighted version of the basic global

criterion provides a necessary condition for Pareto optimality if p, which they imply

relates to curvature, can be increased. Note that this conclusion is also supported by the

idea that the weighted min-max approach, for which p is infinity, can yield all of the

Pareto optimal points, regardless of the nature of the Pareto optimal set. That is, the

111

weighted min-max approach provides a necessary condition for Pareto optimality

(Miettinen, 1999). Alternatively, the standard global criterion formulations cannot; they

typically provide a sufficient condition only (assuming that p is fixed at a constant value).

The relationship between these two approaches is illustrated in the next section.

Curvature is significant also in terms of a global criterion’s ability to provide a

unique Pareto optimal solution. When minimizing a global criterion, curvature

determines the nature of the intersection between the global criterion contour with the

minimum value, and the Pareto optimal hypersurface. If the contour intersects the

hypersurface at more than one point, then the solution is non-unique and depends on the

starting point for the optimization engine. Ultimately, this can be troublesome in terms of

articulating preferences when weights are incorporated. This potential difficulty is

explained as follows. The primary objective of MOO methods ultimately is to select the

best (according to the preferences of the decision-maker) solution from the Pareto

optimal set, and when one uses a priori articulation of preferences, the inability to ensure

a unique solution can inhibit one from achieving this objective. This is because ideally,

there should be a one-to-one mapping between the different indications of the relative

importance of objectives (different sets of weights), and the final solution points.

However, if a single set of weights, which represents a single set of preferences, can

result in different solutions depending on the starting point, then there is ambiguity in the

final solution, especially if the starting point is an arbitrary feature of the optimization

process. Consequently, it becomes unclear which solution is most advantageous from a

design perspective and which solution best reflects user preferences.

112

6.3.1.2 Approximating Min-max Results

As p increases, the general shape of the global criterion contours in the criterion

space begins to resemble the shape of contours for the min-max formulations ((6.4) and

(6.5) result in global criterion contours with the same shape). However, we show that

theoretically, only the root global criterion in (6.2) should be used to approximate min-

max results as p approaches infinity.

Figures 6.1 through 6.3 demonstrate the effect that increasing the value of p has

on the two global criteria, in the criterion space. The contours for the root global

criterion are shown with the solid lines, while the contours for the basic global criterion

are shown with the dashed lines. The contours (for both formulations) represent criterion

values of 0.1, 0.5, 1.0, and 2.0, with the 2.0-contour being furthest away from the origin.

Contours for the min-max formulation given in (6.4), are shown in Figure 6.4.

0.5 1.0 1.5 2.0 2.5

0.5

1.0

1.5

2.0

2.5

Q1

Q2

Contour for the Root

Global Criterion

Contour for the Basic

Global Criterion

Figure 6.1: Global Criterion Contours in the Criterion Space, with 2p =

113

0.5 1.0 1.5 2.0 2.5

0.5

1.0

1.5

2.0

2.5

Q1

Q2 Contour for the Root

Global Criterion


Global Criterion


0.5 1.0 1.5 2.0 2.5

0.5

1.0

1.5

2.0

2.5

Q1

Q2 Contour for the Root

Global Criterion


Global Criterion


114

0.5 1.0 1.5 2.0 2.5

0.5

1.0

1.5

2.0

2.5

Q1

Q2

Single contour for the

Basic Global Criterion

as p approaches

infinity

Figure 6.4: Min-max Contours in the Criterion Space

These figures illustrate how changing the value of p affects the curvature of the

contours. In addition, they show that as the value of p increases, the shape of the global

criterion contours approaches that of the min-max contours. In particular, curvature

approaches infinity where 1 2

Q Q= . Thus, a standard global criterion can be used to

approximate min-max results if p is relatively large. This observation is especially

significant considering that the weighted min-max approach is able to capture all of the

Pareto optimal points, whereas the weighted global criterion approach cannot.

However, there are two caveats to this potential use. First, with the min-max

formulations, significant portions of the contours that are parallel to axes represent points

where components of g

F∇F

are zero. This can result in solutions that are only weakly

Pareto optimal. Consequently, as the value of p approaches infinity, theoretically, the

formulation gradually looses the ability to provide a sufficient condition for Pareto

optimality.

115

Secondly, the contours for the basic global criterion converge to a single contour

indicated in Figure 6.4. This can occur regardless of what contour-values are considered,

and it is explained for the general case (any number of objective functions) by

manipulating (6.3) as follows:

( ) ( )

1,

k pp

j ii i jF c F

= ≠

= −∑x x (6.10)

c represents a contour value. (6.10) is irrational when ( )1,

k p

ii i jc F

= ≠

<∑ x , and as p

increases, this condition holds if any function-value is above one. Alternatively, if

( ) 1i

F i j≤ ∀ ≠x , then ( ) lim 1p

jp

F c→∞

= =x .

In addition to explaining the shape of the contours for (6.3), this analysis points to

significant consequences when it comes to approximating the min-max results. If, as the

value of p approaches infinity, the contours for (6.3) converge on a single contour, then

theoretically the possibility that a solution even exists diminishes with high values for p.

Practically, the risk of numerical difficulties increases. This is because, although the

contours for (6.3) simulate those of (6.2) and (6.4) to some extent, the form of the global

criterion is different. Theoretically, the basic global criterion in (6.3) does not duplicate

the min-max formulations as p approaches infinity; however, the root global criterion in

(6.2) does.

This is illustrated further with Figures 6.5 through 6.7 in which 2k = . These

figures show the shape of the contours for the global criteria in three-dimensional

criterion space. Figures 6.5 and 6.6 show the root global criterion and the basic global

criterion respectively, with 8p = . Figure 6.7 shows the min-max formulation given in

116

(6.4). The contour-shape for root global criterion is similar to that for the min-max

formulation, whereas the contour-shape for the basic global criterion is not.

-2

0

2

-2

0

2

4000

8000

Q1

Q2

Fg

Figure 6.5: Global Criterion Surface for the Basic Global Criterion with 8p =

-2

0

2

-2

0

2

1

3

Q1

Q2 Fg

Figure 6.6: Global Criterion Surface for the Root Global Criterion with 8p =

117

Q1

Q2 Fg

-2

0

2

-2

0

2

1

Figure 6.7: Global Criterion Surface for the Min-max Formulation

In the above-mentioned figures, 8p = . Using an odd value for p alters the nature

of the global criterion surfaces, and there may be numerical difficulties with the root

global criterion, especially with negative values for components of Q, as suggested in

section 6.2. Nonetheless, these figures demonstrate general similarities and differences

between the surfaces for a fixed p-value. Note that with relatively low p-values, despite

the unique form of basic global criterion shown in Figure 6.5, often the only

distinguishing solution-feature between the root global criterion and the basic global

criterion is the difference in contour values; the values for the basic global criterion

contours are significantly higher.

6.3.1.3 Kuhn-Tucker Conditions

The idea that different values of p yield different Pareto optimal solutions may not

be intuitively clear. Consequently, we discuss the theoretical basis for this characteristic

118

using Kuhn-Tucker optimality conditions stated in Marler and Arora (2003). Later, we

demonstrate this characteristic with an example problem. Because this analysis concerns

optimality conditions in the design space rather than the shape of the criterion contours, it

is necessary to distinguish between constrained and unconstrained problems. First, we

consider unconstrained problems. Although such problems are less common with single-

objective optimization, they correspond to constrained problems when no constraints are

active, which is a common condition with MOO. Then, we consider constrained

problems with active constraints.

In order for *x to be a minimum of an unconstrained global criterion, it is

necessary that ( )*gF∇ =

xx 0 . The gradients with respect to the design variables are

given for the root global criterion and the basic global criterion respectively, as follows:

( ) [ ]

11

1

11

pp

k p Tp

g i

i p

k

Q

F Q

Q

−

−

=−

∇ = =

∑xx J 0� (6.11)

( ) [ ]

1

1

1

p

T

g

p

k

Q

F p

Q

−

−

∇ = =

xx J 0� (6.12)

(6.11) and (6.12) can be rewritten as follows:

1

1

1 1

pk kp

p p

i j j

i j

Q Q F

−

−

= =

∇ =

∑ ∑ x

0 (6.13)

1

1

kp

j j

j

p Q F−

=

∇ =

∑ x

0 (6.14)

The solutions to the systems of equations in (6.13) and (6.14) depend on the value of p.

Thus, the value of p can effect which solution is provided.

119

(6.13) and (6.14) can be simplified further in which case they both demonstrate

similarities to a weighted sum. If ≤z F� or if p is an odd number, then the first term in

(6.13) is zero only if all of the i

Q are zero, which in turn implies that the second term is

zero. Therefore, we assume that the first term in (6.13) is not zero and can be discarded.

In addition, we assume that the p in (6.14) is never equal to zero, so it also can be

omitted. Then, (6.13) and (6.14) both are simplified as follows:

1 1 1

1 1 2 2

p p p

k kQ F Q F Q F− − −

∇ + ∇ + + ∇ =x x x

0� (6.15)

Note that when 1p = , (6.15) is comparable to the necessary condition for an objective

sum, and the solution depends only on function gradients. However, when one uses a

global criterion (when 1p ≠ ), then the solution also depends on the i

Q , which vary

throughout the criterion space. Thus, the 1p

iQ

−

terms are like non-constant weights,

indicating the significance of a particular gradient within the context of (6.15). Although

these terms cannot be set directly by the user, they can be controlled to some extent with

the aspiration point and with p. Thus, with the use of an aspiration point and p, a global

criterion models variance (over the criterion space) in the relative importance of

different objectives. If one interprets the i

Q as non-constant weights, then p magnifies

the difference between the values of these weights. In the extreme case, as p approaches

infinity, all but one i

Q become relatively insignificant. Recall that the weighted sum

requires that the decision-maker model the relative importance of each objective function

as being constant (section 4.2.1.1). However, in many cases, this type of model may be

inappropriate; the importance of achieving a goal or of minimizing a particular function

may vary throughout the criterion space. If 1

0p

iQ

−

< for any i, then the solution point

120

cannot be Pareto optimal. This is congruous with properties of the weighted sum

approach.

With constrained problems (and active constraints), (6.11) and (6.12) are

modified as follows:

1

1

1 1 1

a

pmk kp

p p

i j j i i

i j i

Q Q F gµ

−

−

= = =

∇ = − ∇

∑ ∑ ∑x x

(6.16)

1

1 1

amk

p

j j i i

j i

p Q F gµ−

= =

∇ = − ∇

∑ ∑x x

(6.17)

where a

m is the number of active constraints. In this case, the i

Q can not necessarily be

interpreted as variable weights. Nonetheless, (6.16) and (6.17) are comparable to (4.11)

for the weighted sum, when 10

p

jQ j−

> ∀ . As with (6.13) and (6.14), (6.16) and (6.17)

demonstrate that the Pareto optimal solution that is obtained with a global criterion

depends on the value of p.

6.3.2 Preferential Significance and the Effects of

Normalization

Whereas the previous sections address the mathematical significance of p, this

section concerns preferences. Although the p-exponent is mentioned in reference to

preferences only rarely in the literature (Gupta and Sivakumar, 2002), we argue that it

can in fact be used to articulate a form or preference, as suggested in the previous section.

Thus, it should not be restricted to the usual values of one, two, and infinity, which are

common because of their mathematical significance. In terms of preferential

significance, p is proportional to the amount of relative (to other objective functions)

emphasis placed on minimizing the function with the largest difference between ( )iF x

121

and iz (Yu, 1973; Koski and Silvennoinen, 1987). Essentially, p magnifies the difference

between an objective function value and the corresponding component of the aspiration

point or utopia point. p is used to help design the utility function (Yu and Leitmann,

1974; Koski and Silvenoinen, 1987; Athan and Papalambros, 1996), which in turn

approximates the preference function Thus, p can be used to articulate a form of

preference; p is set based on how important it is to the decision-maker to restrict

functions with particularly high values.

This interpretation is intuitively clear from the formulations in section 6.1.1. As p

increases, the relative size of the highest component of the vector Q increases

exponentially and dominates the criterion, which is being minimized. This interpretation

can also be derived from the discussion in section 6.3.3. In equations (6.15) through

(6.17), if one uses a relatively large value for p, then if the difference between ( )iF x and

iz increases, the corresponding gradient becomes more heavily weighted and dominates

other gradients.

As suggested in Chapter 4 in reference to the weighted sum method, any

indication of emphasis or preference should involve a relative scale. Thus, objective

functions should be transformed. Only then are the numerical differences between the

functions, comparable. Consequently, normalization is recommended when using the

global criterion approach. This is especially true when functions have significantly

different orders of magnitude. However, normalization can result in computational

difficulties, as explained here. If the objective functions are normalized, then the

gradients with respect to the design variables are given for (6.2) and (6.3) in (6.11) and

(6.12) respectively, by replacing i

Q with norm

iQ , which is defined in (6.7). As the value

122

of p increases, the gradient in (6.12) approaches zero relatively quickly. This may result

in premature convergence in the sense that computational results will not reflect

theoretical expectations, because of numerical inaccuracies. Thus, if normalized

objective functions are used, the root global criterion should be used to avoid premature

convergence with relatively large p-values. Technically, this applies to any problem in

which function-values are primarily between zero and one. In any case, if one uses

relatively large p-values, standard normalization is not necessarily advantageous, even

when one uses the root global criterion, as we will demonstrate later. Instead, the

following modification to (6.7) is suggested:

( )max

1i i

i

i i

F FQ

F F

−

= +

−

x�

�

(6.18)

The advantages of this modification also are demonstrated later with respect to a

modified global criterion approach.

6.3.3 Values of p Less than One

Although values of p that are less than one are rarely used, they are discussed here

to provide a thorough evaluation of theoretical aspects, which is not provided in the

literature. Whereas using 1p > increases the difference between arguments in the global

criteria, 1p < reduces the difference. In the extreme, when 0p = , each component of

the criteria is equal to one. Using 1p < can be advantageous in situations when all of the

objective functions should be treated equally. In addition, using 0 1p< < does not effect

the sign of the gradients in (6.8) or (6.9), nor the consequent outcome of Theorem 2.2.

However, with 0 1p< < , there is a potential for irrational global criterion values. This

123

may occur when components of the global criterion are negative, as with an inaccurate

utopia point or a poorly selected aspiration point.

In addition, with 0 1p< < , the orientation of the criterion contours in the criterion

space is altered as shown in Figure 6.8; the contours are convex. This is significant,

because unconstrained portions of the Pareto optimal hypersurface in the criterion space

are often convex. Constrained problems can result in convex Pareto optimal

hypersurfaces as well. Consequently, there is a greater chance that the criterion contours

and the Pareto optimal set are co-linear at more than one point, yielding non-unique

solutions (discussed in section 6.3.1.1).

0.5 1.0 1.5 2.0 2.5

0.5

1.0

1.5

2.0

2.5

Q2

Q1


Global Criterion

Contour for the Root

Global Criterion

Figure 6.8: Global Criterion Contours in the Criterion Space, with 0.8p =

Theoretically, p can be negative in which case the global criterion should be

maximized in order to minimize the individual objectives. Rao (1996) provides a general

form of this approach called the inverted utility function method. In such cases, as p

124

decreases, more emphasis is placed on the smaller components of the criterion. However,

using 0p < can cause difficulties. As with 0 1p< < , it results in convex criterion

contours in the criterion space. Furthermore, there is significant potential for division by

zero, especially if an attainable aspiration point is used. Finally, according to Theorem

2.2, the root global criterion may not provide a sufficient condition for Pareto optimality,

if 0p < . Consequently, one should avoid using negative values for p.

6.4 Analysis of the Min-max Formulation

In this section, we discuss solution-properties of the min-max formulations and

issues concerning the Kuhn-Tucker optimality conditions for the alternate min-max

formulation in (6.5).

The min-max approach provides a sufficient condition for weak Pareto optimality

if a utopia point is incorporated (Wierzbicki, 1986; Koski and Silvennoinen, 1987;

Miettinen, 1999). In addition, dropping the absolute value signs in (6.4) insures that any

solution is weakly Pareto optimal even if an attainable aspiration point is used rather than

a utopia point (Wierzbicki, 1986; Miettinen, 1999).

If the solution is unique, then it is Pareto optimal (Yu, 1973; Stadler, 1988), and if

Z is strictly convex, then using the min-max approach provides a unique solution

(Freimer and Yu, 1976). Strictly convex is defined as follows:

Definition 6.1 Strictly Convex: The set kR∈Z is strictly convex if given

any two different points in Z, 1 2 and F F , and any two positive scalars,

1 2 and w w with

1 21w w+ = ,

1 1 2 2intw w+ ∈F F Z .

125

In most cases however, one cannot verify uniqueness, so it is unclear whether the solution

is Pareto optimal or not. Despite the fact that this approach is sufficient only for weak

Pareto optimality, it provides a necessary condition for Pareto optimality when weights

are incorporated.

6.4.1 Kuhn-Tucker Conditions and Regularity

The alternate min-max formulation in (6.5) is typically used instead of min-max

formulation in (6.4) to avoid discontinuities. It involves additional constraints that result

in special properties regarding irregularity and regarding the Kuhn-Tucker necessary

conditions for optimality, which often provide the stopping criteria for gradient-based

algorithms. The Kuhn–Tucker conditions for the alternate min-max formulation are

given as follows:

1

1 0

k

i

i

L µλ

=

∂ = − =∂ ∑ (6.19)

( )1 1

k m

i i j k j

i j

L Q gµ µ+

= =

∇ = ∇ + ∇ =∑ ∑x x xx 0 (6.20)

0; 1,2, ,

iQ i kλ− ≤ = … (6.21)

( ) 0; 1,2, ,

i iQ i kµ λ− = = … (6.22)

( ) 0; 1,2, ,

jg j m≤ =x … (6.23)

( )

10; 1,2, ,

k jg j mµ

+= =x … (6.24)

0; 1,2, ,

ii k mµ ≥ = +… (6.25)

where [ ] ( )1 1

k m

i i j k j

i j

L Q gλ µ λ µ+

= =

= + − +∑ ∑ x . Recall that ( )i i i

Q F z= −x , and therefore,

( )i i

Q F∇ = ∇x x

x .

126

Note that if a function-constraint is active, then the Lagrange multiplier for that

constraint must be positive and non-zero. This idea deviates slightly from the typical

Kuhn-Tucker conditions in which 0i

µ ≥ and is derived as follows. Theoretically, if a

constraint is active and has a Lagrange multiplier of zero, then that constraint must be

orthogonal to the objective function. This means that the dot product of the objective

function gradient with the constraint gradient must be zero. However, this dot product is

written for (6.5) as follows:

0

0 1

1

1

iF

∇ = − −

x

i�

(6.26)

Therefore, it is not possible to have an active function-constraint with a Lagrange

multiplier of zero, since the gradients are not orthogonal as shown in (6.26).

If a solution point is irregular, the Kuhn-Tucker conditions become inapplicable.

Irregularity implies that a vector k m

R+

∈c exists such that ≠c 0 and such that the

following condition is met:

( ) ( )

1 11 0

a ak m

ji

i j k

i j

gFc c

+

= =

∇ ∇ + =

− ∑ ∑

xxxx

0 (6.27)

where ak is the number of active function-constraints and

am is the number of active

model-constraints. For problems with active model-constraints, the likelihood for an

irregular solution is no greater than it is with standard single-objective optimization

problems. However, this is not the case when all of the model constraints are inactive, a

condition that is more common with multi-objective problems than with single-objective

problems.

127

For such problems, (6.27) may be written as follows:

T = J' c 0 (6.28)

where

T

T

T

=

JJ'

e

(6.29)

e is a x 1ak vector of one’s. J is the x

ak n Jacobian matrix for the objective functions

that correspond to active function-constraints. Consequently, T

J' has dimension

1 x a

n k+ . c has dimension x 1ak . The right-hand side of (6.28) has dimension

1 x 1n + .

(6.28) implies that the regularity of the solution point is based on two potential

scenarios. These scenarios are determined by considering the three possible relationships

between the quantities 1n + and ak , which represent the dimensions of

TJ' in (6.28).

First, if 1a

n k+ = and T

arank k = J' (

TJ' has full rank), then =c 0 . However, if

T

arank k < J' , then there are fewer equations than there are variables, and the system

has infinitely many solutions. Secondly, if 1a

n k+ < and T

J' has full rank, then

T

arank k < J' and again there are infinitely many solutions. This is also the case if

1T

rank n < + J' , which implies T

arank k < J' . Finally, if 1

an k+ > and

T

arank k = J' , then =c 0 . If

T

arank k < J' , then there are infinitely many solutions.

Based on these three cases, the following two potential scenarios for irregularity may

occur:

128

1) If the rank of T

J' is equal to ak , then (6.28) has a unique, trivial solution; =c 0 .

However, for a solution point to be irregular, ≠c 0 . Therefore, in such a case, the

solution is regular.

2) If the rank of T

J' is less than ak , then there are infinitely many solutions to (6.28),

and the solution to the min-max problem is always irregular. Note that this case

necessarily includes situations where 1a

n k+ < . Therefore, if 1a

n k+ < , the solution

is necessarily irregular and the Kuhn-Tucker conditions do not apply.

6.4.1.1 Two Active Function-Constraints

A special case occurs when there are just two active function-constraints ( 2ak = ).

Although the min-max approach provides a sufficient condition only for weak Pareto

optimality, we prove that in this case the solution is guaranteed to be Pareto optimal.

If 2ak = , then it is not possible that 1

an k+ < , and (6.28) is written as follows:

1 1 2 2c F c F∇ + ∇ =

x x0 (6.30)

1 20c c+ = (6.31)

2 1 2 2c F c F⇒ − ∇ + ∇ =

x x0 (6.32)

1 2F F⇒∇ =∇ (6.33)

Assuming the utopia point is unattainable, which means [ ] [ ]≠J 0 , then (6.33) implies

that it is possible to reduce both functions simultaneously. However, the solution to the

min-max problem is necessarily weakly Pareto optimal, which means it is not possible to

improve all of the objectives simultaneously. Consequently, (6.33) cannot apply. By

contradiction, the solution point must be regular. If the solution point is regular, then the

129

Kuhn-Tucker conditions are satisfied and the following conditions apply, as suggested by

(6.19) and (6.20):

1 1 2 2F Fµ µ∇ + ∇ =

x x0 (6.34)

1 21µ µ+ = (6.35)

1 2F F⇒∇ = −∇

x x (6.36)

This indicates that the solution point is necessarily Pareto optimal. Thus, when the

alternate min-max formulation yields a solution with no active model-constraints and just

two active function-constraints, the solution point is always regular and Pareto optimal.

6.4.2 Normalization

Normalizing objective functions can be especially important with the min-max

approach. Here, we explain why. If just one function-constraint in (6.5) is active, then

the solution point must be a local minimum of that function. Such a scenario leaves other

objectives unconsidered. However, this occurs only if the objective functions are not

transformed, assuming the utopia point is unattainable. This is because all objective

functions that correspond to active function-constraints have the same value at the

solution point: i

F λ= . Therefore, if all of the functions are normalized or transformed

such that their respective minima all have the same value, then if one function achieves

its minimum value, all other functions must achieve the same minimum value. This

would imply that the solution point was the utopia point, which we assumed was

unattainable. Thus, the individual minima of the functions are avoided. Consequently,

functions should always be normalized when the min-max approach is used. Note that in

such cases, at least two function-constraints must be active. Otherwise, one of the

130

functions would be allowed to achieve its individual minimum, which is not possible as

discussed above.

6.5 Development of A Modified Global Criterion

Approach

In addition to providing new understanding as to how various global criterion

formulations and method-parameters work, the preceding analysis has flushed out

conclusions that culminate in a modified global criterion approach. Specifically, we have

found:

1) Using any global criterion with ≤z F� theoretically ensures the solution is Pareto

optimal.

2) The root global criterion (not the basic global criterion) can be used to approximate

min-max results by using a relatively high value for p.

3) Using the root global criterion helps avoid premature convergence when standard

normalization is used.

4) The modified transformation scheme in (6.18) may provide better results than

standard normalization.

In this section, an example problem is used to solidify these findings and demonstrate the

advantages of a modified global criterion approach that entails using 1) the root global

criterion, 2) the modified transformation scheme in (6.18), and 3) a relatively high value

for p that can be varied on a continuous scale.


The example problem is formulated as follows:

131

Minimize:x

( ) ( )

( ) ( )

( ) ( )

( )

2 2

1 1 2

2 2

2 1 2

4 2

3 1 2 1 2

2

4 1 2

20 0.75 2 2

2.5 1.5

1.8 2.5 0.3

5 1.6 2

F x x

F x x

F x x x x

F x x

= − + −

= − + −

= − + − +

= − +

(6.37)

subject to: ( ) ( )

1 2

2

2 1 2

0

2.1 0.08 2.2 0

g x

g x x

= − ≤

= − − − ≤

This illustrative, constrained problem is designed to have functions with different ranges

of values within the Pareto optimal set and different gradient profiles. The functions

intentionally have orders of magnitude that are not substantially different, suggesting that

normalization may not be necessary. However, a mathematical bi-product of

normalization (in addition to ensuring that no single function dominates the global

criterion) is that all function-values are less than one. This example demonstrates the

computational consequences of this condition and the benefits of using the modified

normalization scheme in (6.18).

As with Table 5.1, Table 6.1 shows the values of the objective functions when

each function is minimized independently. The highest values for each function are

lightly shaded, and darker shading indicates the lowest values for each function.

For this problem, the general global criterion is written as follows:

( )

1

21

1

k pp

g i

i

F Q=

= ∑x (6.38)

where i

Q can have the following forms:

132

( )

( )

( )

( )

max

max

case 1

case 2

case 3

1 case 4

i

i i

i i

i

i i

i i

i i

F x

F x F

F x FQ

F F

F F

F F

− −

= −

− +−

x

�

�

�

�

�

(6.39)

1F

2F

3F

4F

at 1*x 0.0000 3.3125 3.6905 5.6125

at 2*x 39.7071 0.1285 2.0380 4.3956

at 3*x 11.7487 2.2440 0.9937 5.2673

at 4*x 18.4500 3.0600 6.2516 0.0000

Table 6.1: Function-Comparison Matrix for Global

Criterion Study

The version of i

Q for case 1 essentially assumes that =F 0�

and provides a benchmark

for computational performance. Case 2 provides the most basic representation of

distance between the solution point and the utopia point (in criterion space). Case 3

incorporates normalization, and case 4 represents the function-normalization scheme

given in (6.18). (6.38) is minimized using each of the four cases with increasing values

for the exponents p1 and p2. In each case, first, p2 is fixed with a value of one, which

results in a basic global criterion. Then, p1 and p2 are increased in unison, providing a

root global criterion. Therefore, eight different global-criterion problems are solved

(excluding the min-max problems). IDESIGN (Arora, 1989), which utilizes the recursive

quadratic programming method, is used as the optimization engine.

133

6.5.2 Results

6.5.2.1 Alternate Min-max Formulation

As a basis for comparison with the standard global criterion formulations, the

min-max solutions are determined for the four cases using the alternate min-max

formulation and are given in Table 6.2. The min-max solutions for cases three and four

are the same. The solution point in the design space (in terms of x) and in the criterion

space (in terms of F), the values of the Q-components, the number of optimization

iterations, and the final value for lambda, are listed. Each solution point is a regular

point. As one can see, the min-max solution depends on the form of i

Q . This is also true

for other global criterion formulations, as will be seen later. Q-values that are

approximately equal to the value for lambda indicate active function-constraints. In each

case, the function constraints for 1

Q , 3

Q , and 4

Q are active.

Case 1x

2x

1Q

2Q

3Q

4Q # itrs lambda

1 1.13367 0.936471 2.960197 2.184423 2.960250 2.960260 13 2.960246

2 1.09338 0.707944 2.699383 2.477465 2.699253 2.699207 12 2.699199

3 & 4 1.57775 0.967555 0.345218 0.315818 0.345208 0.345227 12 0.345207

1x

2x

1F

2F

3F

4F

1 1.13367 0.936471 2.960197 2.184423 2.960250 2.960260

2 1.09338 0.707944 2.699383 2.605933 3.692993 2.699207

3 & 4 1.57775 0.967555 13.707612 1.134043 2.808796 1.937585

Table 6.2: Min-max Solutions for Global criterion Study

134

We show that the min-max approach does not necessarily result in a solution

point where all of the function constraints are active. However, some authors imply that

all function constraints are necessarily active (Eschenauer et al, 1990; Romero et al,

1998). Ballestero and Romero (1991) prove that all of the function constraints are, in fact

active assuming the problem is unconstrained and the efficient frontier is convex,

continuous, and differentiable.

6.5.2.2 Standard Global Criterion Formulations

The results for the standard global criteria are summarized in Figures 6.9 through

6.12.

0

100

200

300

400

500

0 15 30 45 60 75 90p-value

# i

tera

tio

ns

0.0

0.1

0.2

0.3

0.4

dis

tance

fro

m m

in-m

ax s

olu

tio

n

(err

or)

case one - # itrs.

case two - # itrs

case one - error

case two - error

Figure 6.9: Global Criterion Solutions for the Basic Global Criterion with Cases One and

Two

135

0

2

4

6

8

10

12

14

16

0 50 100 150 200

p-value

# i

tera

tio

ns

0.00

0.10

0.20

0.30

0.40

dis

tan

ce f

rom

min

-max

so

luti

on

(err

or)

case one - #itrs.

case two - #itrs.

case one - error

case two - error

Figure 6.10: Global Criterion Solutions for the Root Global Criterion with Cases One and

Two

0

2

4

6

8

10

12

14

16

0 20 40 60 80 100

p-value

# i

tera

tio

ns

0.0

0.2

0.4

0.6

0.8

1.0

1.2

dis

tan

ce f

rom

min

-max

so

luti

on

(err

or)# itrs.

error

Figure 6.11: Global Criterion Solutions for the Basic Global Criterion with Case Three

136

0

2

4

6

8

10

12

14

16

0 250 500 750 1000p-value

# i

tera

tio

ns

0.00

0.10

0.20

0.30

0.40

dis

tan

ce f

rom

min

-max

so

luti

on

(err

or)

# itrs.

error

Figure 6.12: Global Criterion Solutions for the Root Global Criterion with Case Three

Each figure shows the number of optimization iterations necessary to minimize the global

criterion, on the left y-axis. The right y-axis indicates the Euclidean distance in the

design space between the minimum of the global criterion and the min-max solution.

This provides an indication of the accuracy (or error) with which the global criterion

formulations approximate the solution to the min-max formulation.

In Figures 6.9 through 6.11, an increase in p beyond the maximum value shown,

results in a zero gradient at the end of the first iteration, which yields premature

convergence as discussed in section 6.3.2. On the other hand, using a normalized

objective function with the root global criterion (Figure 6.12) allows for an increase of p

to 5000, without resulting in a zero gradient. For the sake of clarity, results are shown

only for 1000p < .

The following significant points are extracted from the example-problem results:

137

1) In each case, when a valid solution is provided, both the root global criterion and the

basic global criterion result in similar solutions for a given value of p, and increasing

p increases the accuracy with which the global criteria approximate the min-max

results.

2) With cases one and two, as seen in Figures 6.9 and 6.10, an initial gradient (during

the first iteration with the optimization engine) that is approximately equal to zero

corresponds to an upper bound on the value of p. This means that one may gradually

increase p and the consequent accuracy with which the min-max solution is

represented, only until the initial gradient of the global criterion is approximately

zero. Consequently, a clear indication of the maximum practical value for p is

provided. However, this is not the case when normalized objective functions are

used.

3) As shown in Figures 6.11 and 6.12, when standard normalization is used (case three),

the allowance of a large value for p can result in eventual degradation in performance;

accuracy eventually decreases as p increases. Thus, one may unknowingly increase p

to the point of increasing error with no clear indication that the most practical value

for p has been passed.

4) In each case, the root global criterion allows for a larger value of p than the basic

global criterion does, and ultimately, this results in a slightly more accurate

approximation of the min-max results. This difference in accuracy between the two

criteria is relatively large when standard normalization is used.

138

5) When using the root global criterion with normalized functions (Figure 6.12), the

minimum error is not necessarily any lower than that which is achieved with the other

cases (Figure 6.10), although a relatively large value for p becomes practicable.

6) Generally, using a larger value for p requires additional iterations from the

optimization engine. This is especially true with the basic global criterion, as shown

in Figure 6.11. This is not the case when normalized objective functions are used. In

fact, the number of necessary optimization iterations eventually decreases once the

optimum value for p is exceeded. However, this decrease is a consequence of

numerical underflow.

In general, as suggested by analysis earlier, we have demonstrated that the root global

criterion is preferred when using relatively large values for p and approximating min-max

results. Normalization is also advantageous but can result in computational difficulties.

6.5.2.3 Modified Function-Normalization Scheme

In response to the pitfalls of normalization, the modified function-normalization

scheme, shown in (6.18), is tested. It is shown that this approach to normalization can

avoid computational difficulties, as the analysis in section 6.3.2 suggested. The results

are shown in Figures 6.13 and 6.14 and are much more stable in terms of the error and in

terms of the number of necessary iterations. The accuracy and the necessary number of

iterations increase consistently (relative to the previous results) as p increases. (6.18)

provides the preferred normalization approach when large values of p are used. In

addition, the root global criterion requires fewer iterations in this case, which was true for

cases one and two as well.

139

0

10

20

30

40

50

60

70

0 100 200 300 400 500

p-value

# i

tera

tio

ns

0.0

0.1

0.2

0.3

0.4

dis

tan

ce f

rom

min

-max

so

luti

on

(err

or)

# itrs.

error

Figure 6.13: Global Criterion Solutions for the Basic Global Criterion using a Modified

Normalization Scheme

0

2

4

6

8

10

12

14

16

0 250 500 750 1000

p-value

# i

tera

tio

ns

0.00

0.10

0.20

0.30

0.40d

ista

nce

fro

m m

in-m

ax s

olu

tio

n (

erro

r)

# itrs.

error

Figure 6.14: Global Criterion Solutions for the Root Global Criterion using a Modified

Normalization Scheme

140

6.5.2.4 Pareto Optimal Design Space

Here, the solution points discussed in the previous sections are illustrated in the

context of the Pareto optimal design space. Figure 6.15 shows the results that are

obtained using the basic global criterion, while Figure 6.16 shows the results using the

root global criterion. The shaded area represents the Pareto optimal set. The x’s and

squares represent the results with case one and case two respectively. The triangles and

circles represent the results with case three and case four respectively. The small black

dots represent the min-max results. Neither of the model-constraints is active at the

solution points. These results were verified using graphical analysis.

0.5 1.0 1.5 2.0 2.5

0.0

0.5

1.0

1.5

2.0

2.5

x

x2

1F

F3

g1 F4

F2

g2

Figure 6.15: Solutions for the Basic Global Criterion - Design Space

141

0.5 1.0 1.5 2.0 2.5

0.0

0.5

1.0

1.5

2.0

2.5

x

x2

1F

F3

g1 F4

F2

g2

Figure 6.16: Solutions for the Root Global Criterion - Design Space

Although different formulations result in separate clusters of points, it is clear that

different values for p result in distinctly different solutions. In some cases, as the error

(difference between the solutions with the global criterion and with the min-max

formulations) indicated in Figures 6.8 through 6.13 becomes small, it is difficult to

discern different solution points visually. In addition, the solution points are spaced

sporadically. This is because delta-p is not constant. In fact, as one can see in section

6.5.2.2, the maximum allowable value for p varies between 90 and 5000, depending on

computational difficulties. The intent in changing p was to determine the consequences

of using a relatively high value of p, so it was not necessary to thoroughly explore the

complete range of p-values. Nonetheless, Figures 5.14 and 5.15 show that varying p

142

between one and (an approximation of) infinity, results in a series of points that

culminates at the min-max solution point.

An exception to this trend is seen with case three. When normalization is used,

error may increase significantly, as shown in Figures 6.10 and 6.11. In fact, with the

basic global criterion, some of the solutions are not Pareto optimal, although

theoretically, the formulation provides a sufficient condition for Pareto optimality. The

non-Pareto optimal points are a result of substantial computational difficulties. Using

(6.18) rectifies this situation.


With this chapter, we have presented an unparalleled investigation of the global

criterion approach. The approach involves two method-parameters, p and z, which can

be used to design a utility function, and we have provided a unique and practical study

concerning the relationship between and the significance of these parameters in terms of

preferences and computational issues. Furthermore, we have contrasted different global

criterion formulations, including the min-max formulation, in terms of the nature of the

solution they provide and in terms of computational difficulties. We have provided new

insight concerning the solution to the min-max approach, thus outlining conditions for

irregularity. In addition, we brought to light the necessity for normalization with the min-

max approach. In general, we have resolved potential confusion concerning the many

different variations on the global criterion approach, and this clarification is continued in

Chapter 7.

Furthermore, we have developed a modified global criterion, including a modified

normalization scheme. This new criterion effectively approximates the results of the

143

alternate min-max formulation. It provides two advantages. First, it provides a means of

avoiding the additional design variable and constraints in (6.5) that can place unnecessary

demands on the optimization engine. Secondly, it allows one to vary p continuously as a

method-parameter incorporating preferences, with little risk of computational difficulty

when relatively high values are used.

Note that just as there are no “best” values for weights in a weighted sum, there is

no “best” value for p. This is especially true in terms of preferences. p essentially

represents a gauge that is altered in order to modify the amount of emphasis placed on

minimizing the argument (in a global criterion) with the highest value; it provides a new

means of articulating preferences. We have shown that if the modified global criterion is

used, it need not be restricted to the typical values of one, two, or infinity. The precise

value for p is also undetermined when the global criterion is used simply to approximate

the results of the min-max approach. Indeed, precision is not a prerequisite for any

approximation. However, with the modified global criterion, we avoid significant errors

that stem from computational difficulties. Therefore, the user is free to increase the value

of p as necessary, with little risk of obtaining irrelevant results. In Chapter 10, we

provide a practical example where using the modified global criterion with higher values

for p is advantageous.

Many important conclusions are highlighted (italicized) throughout this chapter,

and the most significant of these findings and guidelines are summarized as follows:

1) Compared to the weighted sum, the global criterion involves two types of method

parameters in the form of p and z. p is proportional to the amount of relative (to other

objective functions) emphasis placed on minimizing the function with the largest

144

difference between ( )i

F x and iz . Because p can represent a type of preference, it

should not be limited to the values of one, two, and infinity; it is significant on a

continuous scale.

2) If possible, the aspiration point should always be set such that ≤z F� , in order to

ensure Pareto optimal solutions. Using z as a variable method-parameter in an effort

to provide a necessary condition for Pareto optimality is impractical.

3) When the utopia point is unknown or when one risks determining the utopia point

inaccurately, then using the basic global criterion with an odd value for p ensures

Pareto optimality and avoids potential computational difficulties regardless of the

value for z.

4) Theoretically and conceptually, normalization is not necessary if weights are not

used. Practically and computationally, however, all objective functions should be

normalized, especially if they have significantly different orders of magnitude.

Generally, the normalization scheme shown in (6.7) is robust, but if a relatively high

value of p is used in an effort to approximate min-max results, then the modified

normalization scheme in (6.18) should be used. Functions should always be

normalized when one uses a min-max formulation.

5) Theoretically, the min-max formulation is derived from the root global criterion, not

the basic global criterion. When using the alternate min-max formulation, if no

model constraints are active and the number of active function-constraints is less than

1n + , where n is the number of design variables, then the solution point is necessarily

irregular, and the Kuhn-Tucker conditions do not apply. If no model constraints are

145

active and only two function-constraints are active, then the solution point is

necessarily regular and Pareto optimal.

6) Increasing the value of p with a standard global criterion provides a solution that

approximates min-max results. However, if p is too large, two forms of

computational difficulty may surface: 1) the initial gradient may be zero resulting in

premature convergence, or 2) an eventual decrease in accuracy may occur. A higher

p-value can require more optimization iterations, especially when the basic global

criterion is used. In addition, this criterion more readily results in underflow or

overflow if p is too large. The root global criterion allows for a higher p-value and a

slightly more accurate approximation of min-max results. In summary, using the root

global criterion in conjunction with the modified function-normalization scheme

shown in (6.18) and a relatively high value for p, yields a modified global criterion

that approximates min-max results and enables one to use a continuous range of p-

value to articulate preferences with minimal risk of computational difficulties. In

addition, it removes the necessity for additional constraints and an additional design

variable that accompanies the alternate min-max formulation.

146

CHAPTER VII

WEIGHTED GLOBAL CRITERION APPROACH

7.1 Introduction

Whereas the previous chapter concerns the significance of the exponent p, the

significance of the aspiration point, and the nature of the fundamental global criterion

formulations, this chapter provides a study of the weights. This chapter supplements

Chapter 6, and together, these two chapters offer a revealing analysis of the global

criterion approach and its method-parameters, and they ameliorate the difficulty

surrounding the selection of an appropriate global criterion. Two ways of incorporating

weights are investigated, and it is demonstrated that although these methods are treated as

being equivalent in the literature, they are in fact significantly different. To this end, we

show that only one weighted global criterion formulation approximates the weighted

min-max formulation as the value of p increases. In fact, we prove that as the value of p

increases, the weights can become irrelevant for a particular formulation. We extend the

modified global criterion, which was presented in Chapter 6, and present a specific way

of including weights. Then, we study the effectiveness of a modified weighted global

criterion and the weighted min-max approach (a variation on the weighted global

criterion) in depicting the complete Pareto optimal set. New information is uncovered

concerning the ability of the min-max approach to depict the Pareto optimal set. Along

with Chapters 4 through 6, this chapter responds to objective 2 in section 1.4, which

concerns detailed studies of methods in terms of solution characteristics, significance of

147

method-parameters, evaluation of different potential formulations, and effectiveness in

providing an evenly spaced set of Pareto optimal points.

Like the global criterion approach, the weighted global criterion is relatively

general and encompasses other methods. Consequently, studies of this approach apply to

other methods as well. In addition, a thorough understanding of method-parameters such

as weights, allows for a more effective use of the methods. However, the current

literature is insufficient in terms of analysis of method-parameters. Conclusions are

drawn that provide a better understanding of how to use effectively the weighted global

criterion approach and the weighted min-max approach, and that dispel the assumption

that the min-max approach is advantageous for a posteriori articulation of preference.

7.1.1 Overview of the Formulations

There are two primary approaches for incorporating weights in a global criterion.

These approaches are shown in (7.1) (Yu and Leitmann, 1974; Chankong and Haimes,

1983) and in (7.2) (Zeleny, 1982):

( )( )

1

max

1

;

k pp i i

g i i i

i i i

F FF wQ Q

F F=

− = =

− ∑

x

x

�

�

(7.1)

( ) [ ]( )

1

max

1

;

k pp i i

g i i i

i i i

F FF wQ Q

F F=

− = =

− ∑

x

x

�

�

(7.2)

We refer to (7.1) as the weighted exponential approach and to (7.2) as the weighted

function approach. Typically, w is determined such that 1

1k

iiw

=

=∑ and >w 0 . In the

context of weighted methods, i

Q is usually defined as ( )i i

F F−x� . However, (7.1) and

(7.2) may involve different forms of function transformation, and with (7.2), changing the

scale of the objective function has the same effect as changing the weights (Yu, 1973; Yu

148

and Leitmann, 1974). Based on the results of Chapters 4 through 6, this chapter

incorporates the form of i

Q shown in (5.5) along with a Pareto maximum.

Yu (1985) presents an additional weighted global criterion formulation that is

similar to (7.2) and uses absolute value signs as shown in (6.1). However, as

demonstrated in Chapter 6, using (6.2) (use of the root with no absolute value signs),

rather than (6.1), most accurately approximates the min-max approach, provides better

results when normalized objective functions are used, and results in fewer numerical

difficulties when relatively large values are used for p (assuming ≤z F� ).

Minimizing (7.1) is sufficient for Pareto optimality as long as >w 0 (Chankong

and Haimes, 1983; Miettinen, 1999). Minimizing (7.2) also is sufficient for Pareto

optimality (Zeleny, 1982). Furthermore, Athan and Papalambros (1996) prove that

minimizing (7.2) provides a necessary condition for Pareto optimality. However, some

clarification of their proof is needed. Technically, the proof suggests that for each Pareto

optimal point p

x , there exists a vector w and a scalar p, such that p

x is a solution to (7.2)

. However, a relatively large value of p may be required in order to capture certain

Pareto optimal points, especially with non-convex Pareto optimal sets. As p approaches

infinity, minimizing (7.2) is no longer sufficient for Pareto optimality; it is sufficient only

for weak Pareto optimality. Therefore, for a fixed value of p, (7.2) cannot be both

necessary and sufficient for Pareto optimality.

As mentioned in Chapter 6, the value of p determines to what extent a method is

able to capture all of the Pareto optimal points (with variation in w), even when the

feasible criterion space may be non-convex. With (7.2), using higher values for p

increases the effectiveness of the method in providing the complete Pareto optimal set

149

(Athan and Papalambros, 1996). Some authors suggest this is the case with (7.1) as well

(Messac et al, 2000a, 2000b; Messac and Yahaya, 2001). However, although using a

higher value for p enables one to better capture all Pareto optimal points (with variation

in w), it may also yield non-Pareto optimal points with either formulation.

The formulations in (7.1) and (7.2) are called standard weighted global criterion

formulations. Alternatively, the weighted min-max formulation, which is a special case

of the weighted global criterion approach with p = ∞ , has the following format:

( ) { }maxg i i

iF wQ=x (7.3)

(7.3) is an extension of the min-max approach discussed in Chapter 6, so it may be

formulated as shown in (6.5):

Find: , λ ∈x X (7.4)

to minimize: λ

subject to: ( ) 0 for 1, 2, ,i i

wQ i kλ− ≤ =x …

We refer to (7.4) as the alternate weighted min-max formulation.

As suggested earlier, increasing the value of p increases a method’s ability to

yield all of the Pareto optimal solution points, and (7.3) is the limit of (7.2) as p→∞ .

Therefore, using (7.3) or (7.4) can provide the complete Pareto optimal set; it provides a

necessary condition for Pareto optimality (Miettinen, 1999). Minimizing either

formulation is sufficient only for weak Pareto optimality (Koski and Silvennoinen, 1987;

Miettinen, 1999). However, if the solution is unique, then it is Pareto optimal (not just

weakly Pareto optimal).

150


What distinguishes (7.1) and (7.2) is the application of the exponent to the

weights. However, the significant consequences of this difference are not mentioned in

the literature. Rather, these two formulations are treated equivalently. Adali et al (1995)

use (7.1) with 2p = to design the laminate for a closed-end cylindrical shell. The

optimum fiber angle is determined to maximize axial buckling load, external buckling

pressure, and maximum internal pressure. However, only two objective functions are

considered at a time. Chen (1998) develops an interactive algorithm based on (7.1)

without the root. This algorithm is used with a power-planning problem in which

operating cost, voltage deviation, and system security margin (toleration of a high load

demand without voltage collapse) are optimized. Initially, all of the weights are equal.

Then, with each interactive iteration, the values for the weighting vector w and for p are

set by the user. Yang et al (1994) use (7.2) with 2p = for automotive crash analysis of a

front horn. Five design variables are used to describe geometric features of the horn.

The objectives are to maximize absorbed energy and to minimize weight, force deviation

(in terms of the force-deformation curve), and first peak force. The constraints consist of

limits on the design variables. Gupta and Sivakumar (2002) use a form of (7.2) with

3p = , to solve a job-shop scheduling problem for semiconductors.

The limited amount of literature concerning the weighted global criterion

indicates some deficiencies. First, there is minimal detailed analysis of the method in

terms of the significance of the weights; generally, it is considered only as a tool for a

specific application. Secondly, the difference in performance between the two

approaches to incorporating weights is not addressed.

151

Although there are substantial applications of weighted min-max approach in the

literature, there are deficiencies surrounding this method as well. Kumar and Tauchert

(1992) apply (7.3) to the design of laminated plates. Laminate ply angle and thickness

are determined in order to optimize axial buckling-load, shear buckling-load, bending

strength, and stiffness, but only two objectives are considered at a time. Surdacki and

Montusiewicz (1996) use a formulation like (7.3) to select what they call a representative

solution from a finite set of Pareto optimal solutions that are determined using other

methods. Shimoda et al (1996) use (7.3) for the shape optimization of basic linear elastic

structures and use the compliance resulting from different loading conditions as the

different objectives.

Bendsoe et al (1984) use (7.4) to the design of a cantilever beam and to the design

of a two-bar truss, but the actual values of the weights are not discussed. Matsumoto et

al (1993) use (7.4) for sizing optimization of a motorcycle frame. The diameter and

thickness of each frame-member are design variables. The acceleration response levels at

the handlebars, front footrest, and rear footrest, which are subjected to multiple excitation

forces, are minimized. In addition, static torsional compliance and total mass are

minimized. The weight for each acceleration response value is constant and is set based

on the vibration frequency experienced by the rider. Chen et al (1999) apply (7.4) to bi-

objective robust design problems.

Despite the above-mentioned applications, the weighted min-max approach is not

critically viewed as a version of the weighted global criterion. The two variations

(standard global and min-max) are not compared. Although the min-max approach may

be used based on the advantage that it is able to capture all of the Pareto optimal points

152

even when the Pareto optimal set is non-convex (Kumar and Tauchet, 1992; Matsumoto

et al, 1993), as noted in Chapter 6, it may provide non-Pareto optimal solutions as well.

In addition, although there are some examples in which weighted criteria are used to

depict the Pareto optimal set for two objective functions (Kumar and Tauchert, 1992;

Adali et al, 1995; Shimoda et al, 1996; Chen et al, 1999), the weighted min-max

approach and the standard weighted global criterion method are not thoroughly studied in

terms of their ability to depict the Pareto optimal set accurately. In fact, in some

instances, although a “weighted criterion” is discussed, the weights are all fixed with a

value of one (Saravanos and Chamis, 1992; Gupta and Sivakumar, 2002), making their

use moot.


The following specific issues are addressed in this chapter:

1) The two approaches for incorporating weights in a global criterion are studied from a

mathematical perspective in terms of what effect the weighting vector has on the

solution point.

2) The effectiveness of the weighted global criterion approach and the weighted min-

max approach in depicting the complete Pareto optimal set, is studied in terms of the

ability to yield an even spread of solution points.

3) The modified global criterion presented in Chapter 6 is extended by incorporating

weights, and the consequent formulation is tested.

153

7.2 Analysis of the Weighted Global Criterion

Formulations

In this section, the two approaches for incorporating weights in a global criterion

are studied from an analytical perspective. The formulations shown in (7.1) and (7.2) are

evaluated in terms of the effect that different weighting vectors have on the global

criterion contours in the criterion space. This analysis is then extended to the case where

p = ∞ with the min-max approach. Note that a detailed analysis of the effects of

different values for p was provided in Chapter 6. Here, we focus on the weighting vector

w.

7.2.1 Standard Global Criterion Formulations

With (7.1), i

w is applied to the term p

iQ rather than to

iQ . Thus, generally, it

indicates the relative importance of p

iQ . However, the physical significance of p

iQ is

unclear. Consequently, the actual potential effect of i

w may be unclear. Based on the

study in Chapter 4, the weights are significant relative to each other and relative to the

magnitude of the objective functions. Therefore, when weights are incorporated in the

global criterion, one should use the weighted function approach in (7.2) and apply the

exponent p to the weights as well as to the functions. In this way, the weights are

associated directly with the objective functions i

Q , and the values of the weights are

significant relative to the values of the functions.

Figures 7.1 through 7.6 illustrate this idea by using global criteria with two

objective functions and with 7p = .

154

2 4 6 8 10

2

4

6

8

10

Q1

Q2

1.5g

F =

2.5g

F =

Figure 7.1: Global Criterion Contours with =w 1 and 7p =

0

2

4

6

8

10

2

4

6

8

10

5

10

Q1

Q2 Fg

Figure 7.2: Global Criterion Surface with =w 1 and 7p =

155

2 4 6 8 10

2

4

6

8

10

Q1

Q2

1.5g

F =

2.5g

F =

Figure 7.3: Global Criterion Contours for the Weighted Exponential Approach with

( )0.3,0.7w = and 7p =

0

2

4

6

8

10

2

4

6

8

10

5

10

Q1

Q2 Fg

Figure 7.4: Global Criterion Surface for the Weighted Exponential Approach with

( )0.3,0.7w = and 7p =

156

2 4 6 8 10

2

4

6

8

10

Q1

Q2

1.5g

F =

2.5g

F =

Figure 7.5: Global Criterion Contours for the Weighted Function Approach with

( )0.3,0.7w = and 7p =

0

2

4

6

8

10

2

4

6

8

10

5

10

Q1

Q2 Fg

Figure 7.6: Global Criterion Surface for the Weighted Function Approach with

( )0.3,0.7w = and 7p =

157

Figures 7.1 and 7.2 depict a global criterion with =w 1 (essentially no weights). Figures

7.3 and 7.4 show the contours for the weighted exponential criterion in (7.1) with

10.3w = and

20.7w = . Figures 7.5 and 7.6 show the contours for the weighted function

criterion in (7.2) with the same weights. Contour lines are plotted for

0.5,1.0,1.5, and 2.5g

F = .

The results with =w 1 (negating the effects of using weights) and with the

weighted exponential approach are similar, whereas the results with the weighted

function approach are clearly distinguishable. Note that a change in the shape of global

criterion contours results in a change in the solution point; the solution depends on the

nature of the global criterion contours in the criterion space. Consequently, using =w 1

and using the weighted exponential approach with [ ]0.3,0.7=w may result in similar

solutions points, despite the differences in the weighting vector. Such performance

defeats the purpose of incorporating weights. Alternatively, using the weighted function

approach allows the weights a more significant role in governing the solution.

7.2.2 Min-max Formulations

In this section, we prove that as the value of p increases, a given set of weights

has less effect on the solution to (7.1). In the limit, when p = ∞ , (7.1) can be written as

follows:

( ) [ ]

1

1

lim

k pp

g i ip

i

F w Q→∞

=

=

∑x (7.5)

Assume that 1

Q is the largest component of the vector function Q. That is, 1

maxi

i

Q Q= .

It may be separated as follows:

158

( )

1

1 1

2

lim

k pp p

g i ip

i

F wQ wQ→∞

=

= +

∑x (7.6)

Then, as long as k is a finite number, the second term in (7.6) is relatively small. Thus,

(7.6) may be written as follows:

( ) { }1

1 1lim

p p

gp

F wQ→∞

=x (7.7)

which in turn may be simplified as follows:

( ) ( )1

max 1 max maxg i i i i

i i iF w Q Q Q∞

= × = × =x (7.8)

(7.8) does not involve the weights. On the other hand, when p = ∞ , (7.2) reduces to the

common form of the weighted min-max formulation shown in (7.3), an alternate form of

which is given in (7.4). Thus, when relatively high values for p are used, the weighted

function approach in (7.2) approximates the weighted min-max approach, but the

weighted exponential approach in (7.1) does not.

7.3 Development of A Modified Weighted Global

Criterion Approach

In this section, we present a modified weighted global criterion method that stems

from the work presented in Chapter 6. We briefly re-state the example problem from

Chapter 4, which is used to study the effectiveness of the modified weighted global

criterion method (with different values for p) and the weighted min-max approach in

terms of their ability to depict the Pareto optimal set in the criterion space and in the

design space.

Based on the discussion in section 7.2.1, the weighted function approach, as

shown in (7.2), is used as a foundation. As determined in Chapter 6, as the value of p

159

increases, the modified normalization scheme shown in (6.18) should be incorporated.

Thus, the following modified weighted global criterion is proposed:

( ) [ ]( )

1

max

1

1 ; k pp i i

g i i i

i i i

F FF wQ Q

F F=

− = + =

− ∑

x

x

�

�

(7.9)

Based on the work in Chapter 5, ( )max

1

max *i i jj k

F F≤ ≤

= x , where *j

x is the point that

minimizes the jth

objective function.


The problem from Chapter 5 is re-stated as follows:

Minimize:x

( ) ( )

( ) ( )

( ) ( )

2 2

1 1 2

22 2

2 1 2

4 2

3 1 2 1 2

25 0.5 2 2 0.1

2.5 4 1.8

2.0 1.5 2.8 0.3 10

F x x

F x x

F x x x x

= − + − +

= − + −

= − + − + +

(7.10)

subject to:

( ) ( )2

1 1 2

2 1

3 2

2 2

2.1 0.08 2.2 0

0

0

3.0 0

g x x

g x

g x

g x

= − − − ≤

= − ≤

= − ≤

= − ≤

As suggested in Chapter 4, this problem is designed to yield clearly distinguishable sets

of points in both the two-dimensional design space and the three-dimensional criterion

space, with some points resulting in active constraints. The functions have significantly

different minima, maxima, and Pareto maxima, as shown in Table 5.1. The Pareto

optimal set for this problem is shown in Figure 5.1 for the design space and in Figure 5.2

for the criterion space.

For the computational results, a series of problems is solved. While one of the

three weights (one for each objective function) is fixed at 0.0, .33, or .66, the other two

are varied in increments of 0.025 (one is increased while the other is decreased) such that

160

a convex combination of functions is used. Then, while one of the three weights is

increased in increments of 0.05, the other two weights, both assigned the same two

values, are reduced simultaneously. This results in 309 solution points. The problems

are solved using Excel Solver, which utilizes the generalized reduced gradient method.

7.3.2 Results

As a point of reference, the problem is first solved with 1p = (using a weighted

sum), the results of which are shown in Figures 5.11 and 5.12. In these figures, the

Pareto optimal solution points are relatively well dispersed over the Pareto optimal set.

The Pareto optimal solution points resulting from the weighted global criterion in (7.2),

with 7, 50, and 800p = are shown in Figures 7.7, 7.8, and 7.9 respectively. Although

only the design space is shown, comparable degrees of dispersion were reflected in the

criterion space as well.

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x1

x2

F1-minimum

F3-minimum

F2-minimum

Figure 7.7: Pareto Optimal Set using a Weighted Global Criterion with 7p = - Design

Space

161

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x1

x2

F1-minimum

F3-minimum

F2-minimum


Space

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x1

x2

F1-minimum

F3-minimum

F2-minimum


Space

162

Compared to the weighted sum results of Chapter 5 (see Figures 5.11 and 5.12),

as p increases, the effectiveness of the weighted global criterion approach in providing an

even spread of Pareto optimal points can decrease. We note two general trends and

provide explanations for this degradation. First, we discuss the tendency of the Pareto

optimal solution points to gravitate towards the periphery of the actual Pareto optimal set.

Theoretically, when the objective functions have similar magnitudes and no single term

in a weighted global criterion dominates the other terms, then minimizing the criterion

generally yields a Pareto optimal solution point near the center of the Pareto optimal set

(in the design space and in the criterion space). However, as the value of p increases,

differences in the terms are magnified; the terms do not contribute equally to the final

value of the criterion. Instead, the values of the terms associated with relatively small

weights shrink exponentially and become insignificant, while other terms become

relatively significant. In general, as the relative value of a particular term in a criterion is

reduced, the solution point moves away from the minimum of the function corresponding

to that term and towards the minimum of the function that dominates the criterion. The

periphery of the Pareto optimal set represents points where the value of at least one

objective function (and the corresponding term on the global criterion) is zero. Thus, for

a fixed set of weights that results in a solution point that is close to one of the boundaries

of the Pareto optimal set, increasing the value of p causes the relative value of at least one

objective function to approach zero and the resulting solution point to approach the

boundary.

163

The second trend that surfaces as p increases, is that solution points tend to

gravitate towards the utopia point in the criterion space. This is because when a global

criterion has a high p-value or when the min-max approach is used ( p = ∞ ), more

emphasis is placed on functions that have higher values, i.e. points in the criterion space

that are farther away from the utopia point. This idea is illustrated in Figure 7.10, in

which the points from Figure 7.9 are shown in the criterion space along with the utopia

point. As one can see, the Pareto optimal solution points are drawn towards the utopia

point, as distinguished from the even spread of points shown in Figure 5.12, resulting

from the weighted sum approach.

F3

F2

F1

Utopia

point

Figure 7.10: Pareto Optimal Set using a Weighted Global Criterion with 800p = -

Criterion Space

164

The two above-mentioned trends are demonstrated further in Figures 7.11 and

7.12, which illustrate the Pareto optimal solution points that are determined using the

min-max approach. Such results represent the use of a weighted global criterion with

p = ∞ . These figures show a slight continuation of the aforementioned trends, with a

slight increase of the clustering of points towards the edge of the Pareto optimal set and

towards the utopia point. These figures also demonstrate how the results with the

modified weighted global criterion (Figures 7.9 and 7.10) approximate the min-max

results (Figures 7.11 and 7.12).

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x1

x2

F1-minimum

F3-minimum

F2-minimum

Figure 7.11: Pareto Optimal Set using the Weighted Min-max Approach – Design Space


In this chapter, the weighted global criterion approach has been studied. We

show that despite the presence in the literature of two different methods for incorporating

weights, one is more effective than the other is. The weighted function approach in (7.2)

165

is superior to the weighted exponential approach shown in (7.1). This finding further

supports the contention in Chapter 4 that the magnitude of a weight should be significant

relative to the function with which it is associated. With (7.1), the weighting vector has a

decreasing effect on the solution as p increases. In fact, we have proven that with (7.1),

in the limit as p→∞ , the weights become irrelevant. We have shown and explained

why the weighted min-max method is not necessarily suitable for depicting the Pareto

optimal set.

F3

F2

F1

Utopia

point

Figure 7.12: Pareto Optimal Set using the Weighted Min-max Approach – Criterion

Space

By incorporating weights, we have extended the development of the modified

global criterion provided in Chapter 6. We show that, as with the modified global

criterion, the modified weighted global criterion can be used to approximate weighted

166

min-max results without an additional design variable or constraints. In addition, p may

be varied continuously in an effort to articulate a form of preference. However, we have

also demonstrated that using (or approximating) the weighted min-max approach to

depict the Pareto optimal set may not always be suitable for a posteriori articulation of

preferences.

The weighted global criterion and the weighted min-max approach are touted as

providing a necessary condition for Pareto optimality; both methods are able to capture

all of the Pareto optimal points, even with a non-convex Pareto optimal set. However,

when it comes to depicting the Pareto optimal set with an even spread of points, for a

posteriori articulation of preferences, we have shown that these approaches are not

necessarily more advantageous than methods that only provide a sufficient condition for

Pareto optimality, such as the weighted sum method. The tendency of Pareto optimal

solution points to veer towards the utopia point and towards the periphery of the Pareto

optimal set can be detrimental, and this characteristic has not been discussed in the

current literature.

Coupled with Chapter 6, this chapter provides substantial clarification concerning

the use of the global criterion approach (and weighted global criterion approach), an

approach with many different variations and multiple method-parameters. Until now,

deciding which formulation (of the three in Chapter 6 and the two in this chapter) was

most appropriate and deciding how to set the different parameters (w, z, and p) was an

intimidating task at best. Work provided in this chapter and in Chapter 6, helps clarify

the details of the global criterion approach.

167

CHAPTER VIII

ε -CONSTRAINT METHOD

8.1 Introduction

In this chapter, the ε -constraint method is discussed. Whereas Chapters 4

through 7 concern common scalarization methods that involve the combination of

multiple objectives into one function, the ε -constraint method takes a slightly different

approach. It is not a scalarization approach. Instead, a single primary objective function

is minimized, while all other objectives are used as constraints. This is another common

method, as many practical applications lend themselves to such a formulation. That is,

the objective function limits, which are the method-parameters in this case, are often set

based on readily available and quantifiable information rather than on potentially vague

preferences.

However, the function limits can be used simply as mathematical parameters, and

their significance in this capacity has never been investigated. Thus, we study the

effectiveness of the ε -constraint method in depicting the complete Pareto optimal set, as

the function limits are altered systematically. We demonstrate how and explain why

certain constraint limits can result in infeasible solutions or in duplicate (the same

solution for different sets of ε -values) solutions, especially when more than two

objectives are considered. In addition, we investigate the effects of normalization and of

altering the primary objective. The end-result is a substantially increased understanding

of how the ε -constraint method works as well as guidelines for its use. Along with

Chapters 4 through 7, this chapter completes objective 2 in section 1.4, which concerns

168

detailed studies of methods in terms of solution characteristics, significance of method-

parameters, and effectiveness in providing an evenly spaced set of Pareto optimal points.

Thorough understanding of the function limits allows for a more effective use of

this method when appropriate values are not indicated by problem information. In

addition, the ε -constraint method is most often used with just two objectives, and its

effectiveness or lack there of with additional objectives has not been demonstrated.

Conclusions are drawn that provide a better understanding of how to use the ε -constraint

method effectively.

8.1.1 Overview of the Formulation

The ε -constraint method is one method in a class called the bounded objective

function approach, which involves minimizing the single, most important objective

function and creating constraints with all other objective functions as follows (Hwang

and Md. Masud, 1979):

Find: x XŒ (8.1)

to minimize: ( )s

F x

subject to: ( ) ; 1, 2, , ; i i il F i k i sε≤ ≤ = ≠x …

The decision-maker selects ( )s

F x , which is called the primary objective function. The

decision-maker also determines il and

iε , which represent respectively, the lower and

upper bounds for the objective function i

F .

Haimes et al (1971) introduce the ε -constraint method (also called the trade-off

method), in which il is excluded. Then, the additional constraints are modeled as

follows:

169

( ) ; 1, 2, , ;

i iF i k i sε≤ = ≠x … (8.2)

We refer to these additional constraints as function-constraints. Because the bounded

objective formulation in (8.1) is relatively uncommon in the literature and because the

practicality of using a lower limit is unclear, we focus on (8.2).

If it exists, a solution to the ε -constraint formulation is always weakly Pareto

optimal (Miettinen, 1999). In addition, any weakly Pareto optimal point can be obtained

if the feasible region is convex and all objective functions are explicitly quasiconvex

(Ruiz-Canales and Rufian-Lizana, 1995; Luc and Schaible, 1997), where the term

explicitly quasiconvex is defined as follows:

Definition 8.1 Explicitly Quasiconvex: Let X be a convex set. Then, a

function ( )F x is quasiconvex if for all 1 2, ∈x x X and 0 1β< < ,

( )( ) ( ) ( )1 2 1 21 ,F Max F Fβ β+ − ≤ x x x x . ( )F x is strictly quasiconvex

if for all 1 2, ∈x x X and 0 1β< < , such that ( ) ( )

1 2F F≠x x ,

( )( ) ( ) ( )1 2 1 21 ,F Max F Fβ β+ − < x x x x . A function is explicitly

quasiconvex if it is quasiconvex and strictly quasiconvex.

The distinction between different types of convexity is clarified by considering a function

of one variable x. Such a function is considered convex if the graph of the function

between any two points lies below the line joining the points. Alternatively, the function

is quasiconvext if the graph of the function between any two points lies below the

maximum of the function-values at the two points. Essentially, this means that to be

170

quasiconvex, the function cannot decrease once it begins to increase, as x increases.

Strictly quasiconvex generally means the function must continue to increase once it

begins to increase, as x increases. For most practical purposes, strictly convex and

explicitly convex are equivalent, although mathematical examples distinguishing these

two definitions are available (Karamardian, 1965). Technically, a strictly quasiconvex

function is not always a quasiconvex function.

If the solution to the ε -constraint formulation is unique, then it is Pareto optimal

(Haimes et al, 1971; Chankong and Haimes, 1983; Sakawa, 1993; Miettinen, 1999). Of

course, uniqueness can be difficult to verify unless the problem is convex and ( )s

F x is

strictly convex, in which case the solution is necessarily unique (Chankong and Haimes,

1983). Solutions with all ε -constraints active (and non-zero Lagrange multipliers) are

necessarily Pareto optimal, but a solution point with an inactive ε -constraint (or

Lagrange multiplier equal to zero) may not necessarily be Pareto optimal (Cohon, 1978;

Carmichael, 1980). In summary, solving the ε -constraint formulation provides neither a

necessary condition nor a sufficient condition for Pareto optimality.

Figure 8.1 illustrates how the ε -constraint method works in terms of two general

objectives in the criterion space, and elements of the figure are discussed throughout this

study. 2

F is the primary objective function, and 1F forms the ε -constraint with

3.75ε = . The shaded area represents the feasible criterion space in the absence of the ε -

constraint. Note that modifying the ε -constraint limit modifies the final Pareto optimal

set, whereas modifications in the method-parameters with previous methods altered the

shape and/or orientation of the utility function contours. Minimizing 2

F with the

indicated ε -constraint results in the solution point C, where 2

2.5F = .

171

1 2 3 4 5 6 7

1

2

3

4

5

6

7

F1

F2

A B

Pareto Optimal

Set

Z1

3.75F ≤

F2-contour

C

Figure 8.1: ε -constraint Method

A convenient property of the ε -constraint method is that the Lagrange multipliers

for the function-constraints represent trade-offs between the different objectives

(Carmichael, 1980; Steuer, 1989; Miettinen, 1999). That is, at the solution point, a

Lagrange multiplier indicates how much its corresponding objective function (associated

with its corresponding function-constraint) must increase in order to compensate for a

decrease in the primary objective function, while remaining on the Pareto optimal

hypersurface. This can provide useful information, especially if the user has to choose

between a few different solutions.

Assuming the solution point is a regular point, and assuming the Kuhn-Tucker

necessary conditions are satisfied for the problem in (8.1), a Lagrange multiplier for an

active function-constraint gives the following relationship based on the common

sensitivity theorem (Arora, 2004):

172

( )*;

s

i

i

Fi sλ

ε

∂= − ≠

∂

x

(8.3)

where i

λ is the Lagrange multiplier corresponding to the constraint ( ) 0i i

F ε− ≤x . If

( )( )* 0i i iFλ ε− =x as stipulated by the Kuhn-Tucker conditions, then with an active

constraint, iε can be replaced with

iF . Consequently, (8.3) can be used to indicate the

trade-off between two different objectives.

8.1.2 Setting the Constraint Limits

A systematic variation of k

R∈ε can yield a set of Pareto optimal solutions

(Hwang and Md. Musad, 1979). However, setting ε to provide all Pareto optimal points

without resulting in an infeasible problem and/or duplicate solutions is nearly impossible

with general problems.

In reference to a linear bi-criterion problem, Cohon and Marks (1973) suggest

setting ε at “…0 or at some predetermined value” and then increasing it incrementally

until the solution becomes infeasible. However, determining the initial lower limit on ε

improperly can result in duplicate solutions and wasted computational time.

Stadler (1988) outlines a relatively complex algorithm for determining ε -values,

which provides only Pareto optimal solutions and is able to generate the entire Pareto

optimal set. It avoids the possibility of an infeasible problem and the possibility of

determining duplicate solutions. However, in order to determine appropriate ε -values,

this algorithm requires the solution of multiple intermediate optimization problems while

determining a single Pareto optimal point. This burden increases significantly as the

number of objective functions increases. In fact, it is suggested that the approach is

appropriate only for problems with no more than five objective functions.

173

8.1.2.1 Primary-Objective Approach

In this study, we focus on two basic approaches for setting the ε -constraint

limits. Although ε is often incremented such that ( )maxi i i

F Fε≤ ≤x

x� , the first

approach that we discuss provides a more restrictive guideline as follows (Carmichael,

1980):

( )*i i i s

F Fε≤ ≤ x� (8.4)

where *s

x is the point that minimizes the primary objective function. We refer to this as

the primary-objective approach. Although (8.4) is presented for general problems (more

than two objective functions), it is best suited for bi-criterion problems. As we will show,

with general problems, (8.4) may unnecessarily restrict the Pareto optimal solution points

that can be obtained. In addition, it may result in duplicate solutions and infeasible

problems.

8.1.2.2 Pareto-Maximum Approach

Cohon (1978) replaces ( )*i s

F x in (8.4) with the Pareto-maximum of i

F , as

defined in Chapter 5. We refer to this as the Pareto-maximum approach, but we show

that this approach also can result in duplicate solutions and infeasible problems. As one

can see from Figure 8.1, in the case of just two objective functions, as long as the

primary-objective approach or the Pareto-maximum approach is used to determine the

value for ε , a feasible solution always exists. However, although Carmichael (1980)

suggests that (8.4) “generally avoids” non-Pareto optimal solutions, if the ε -constraint

happens to cross line segment AB, which indicates weakly Pareto optimal points (see

discussion of Figure 2.6), the solution may be non-unique and only weakly Pareto

optimal. Such a scenario is possible with the Pareto-maximum approach as well.

174


When modeling an optimization problem, the difference between an objective and

a constraint can often be ambiguous at best. Consequently, there is little literature

pertaining to the formulation in (8.1), in the context of MOO. Most of the published

applications for ε -constraint method are relatively simple and involve depicting the

Pareto optimal set for a posteriori articulation of preferences with only two objective

functions. One exception to this trend in terms of problem complexity is provided by

Cohon and Marks (1973). They use the ε -constraint method to approximate the Pareto

optimal curve for a linear bi-criterion water resource investment problem that has 240

constraints and 600 design variables. ε is varied to yield several Pareto optimal solutions

from which the user may select a final solution point. The variables represent decisions

and designs concerning facilities, reservoirs, diversions, irrigation sites, etc. The primary

objective function involves the minimization of the absolute deviation of regional water

use from average water use among four regions. An ε -constraint is applied to the

second objective, which represents the maximization of income. Model-constraints are

applied to the water use for each region and to the amount of water diversion for

irrigation.

Carmichael (1980) applies the ε -constraint method to a five-bar, two-objective

truss problem from Majid (1974). Again, the method is used for a posteriori articulation

of preferences. Weight is minimized with an ε -constraint on nodal displacement. Four

design variables represent various dimensions and the areas of the truss members. Two

equality constraints are used to represent the structural constitutive relations, and limits

are placed on each of the design variables.

175

Rajurkar et al (1987) also use the ε -constraint to produce multiple solutions for a

two-objective problem. An electrical-discharge machining (EDM) problem is solved in

which total machining cost is minimized with an ε -constraint on total machining time.

Material is removed in three stages, and there are three design variables for each stage,

resulting in nine design variables. The design variables are volume of removed material,

tool cost, and “on-time” for each of the stages. Model-constraints are placed on surface

finish specifications, total volume of material to be removed, and total machining time.

In an effort to study the effect of residual stresses, Shabeer and Wang (2000)

represent the Pareto optimal set for a two-objective brakeforming (bending) process that

is applied to stringers and spars of wing structures. It is shown that the solutions to this

particular problem are unique and consequently guaranteed to be Pareto optimal. Shape

error is minimized with an ε -constraint on curvature variation. The design variables are

the bending moment and the location of the bend. As with the work of Rajurkar et al

(1987), it is not indicated how exactly ε is varied.

Hsiao and Chien (2001) use the ε -constraint method in an interactive algorithm

for determining the optimum placement for a series of capacitors in a distribution system.

Essentially, the user alters ε with each iteration, until an acceptable solution is

determined. Capacitor construction expense, real power loss, and deviation of bus

voltage are minimized, while the security margin of feeders and transformers is

maximized. Model-constraints are placed on total load (power). The design variables

represent capacitor sizes and control settings.

As the literature suggests, to date, the ε -constraint method has been used

primarily with bi-objective problems. Its performance with more than two objective

176

functions has not been studied. In addition, unless the constraint limits are set based on

problem information in an effort to model physically significant data, how to set the

limits is unclear. Currently, there is no thorough evaluation of the ε -constraint method

and no clear indication as to how one can use the method-parameters most effectively.


The following specific issues are addressed in this chapter:

1) Using the primary-objective and Pareto-maximum approaches for determining the

bounds on ε , study the effectiveness of the ε -constraint method in depicting the

complete Pareto optimal set for a problem with more than two objectives.

2) Demonstrate that certain constraint limits lead to infeasible problems and/or duplicate

solutions when more than two objectives are involved.

3) Investigate the significance of normalized objective functions.

4) Investigate the repercussions of changing the primary objective function.

5) Compare the ε -constraint method with weighted methods.

8.2 Analysis of the Method


In this section, we briefly re-state the example problem from Chapter 5, which is

used to study the effectiveness of the ε -constraint method in terms of its ability to depict

the Pareto optimal set in the criterion space and in the design space. The problem is

stated as follows:

Minimize:x

( ) ( )

( ) ( )

( ) ( )

2 2

1 1 2

22 2

2 1 2

4 2

3 1 2 1 2

25 0.5 2 2 0.1

2.5 4 1.8

2.0 1.5 2.8 0.3 10

F x x

F x x

F x x x x

= − + − +

= − + −

= − + − + +

(8.5)

177

subject to:

( ) ( )2

1 1 2

2 1

3 2

2 2

2.1 0.08 2.2 0

0

0

3.0 0

g x x

g x

g x

g x

= − − − ≤

= − ≤

= − ≤

= − ≤

As stated previously, this problem is designed to yield clearly distinguishable sets of

points in both the two-dimensional design space and the three-dimensional criterion

space, with some points resulting in active constraints. The functions have significantly

different minima, maxima, and Pareto maxima, as shown in Table 5.1. The Pareto

optimal set is shown in the design space and criterion space, in Figures 5.1 and 5.2

respectively.

To obtain computational results, first, a primary objective function is selected.

The ranges of values for the ε -constraint limits pertaining to the two remaining functions

are determined based on the primary-objective approach or the Pareto-maximum

approach. Recall that the primary-objective approach entails setting the upper limit of i

ε

based on the value of ( )*i s

F x , where *s

x is the point that minimizes the primary

objective function. The Pareto-maximum approach entails setting the upper limit of i

ε

based on the Pareto-maximum of i

F . The values for the limits are given in Table 8.1. A

series of problems is solved, where each ε -value is incremented consistently within its

range, based on the pre-selected number of problems that are to be solved. Each problem

is solved using SNOPT software (Gill et al, 2002), which utilizes sequential quadratic

programming.

178

Primary-Objective

Approach

Pareto-Maximum

Approach

lower limit upper limit lower limit upper limit

1F 0.1000 32.0687 0.1000 67.6807

2F 0.0224 16.9994 0.0224 43.0336

Table 8.1: ε -constraint Limits with 3

F as the Primary Objective

8.2.2 Representing the Complete Pareto Optimal Set

Figures 8.2 and 8.3 show results in the design space and in the criterion space,

respectively. With these initial results, the Pareto-maximum approach is used to set ε -

constraint limits for 1F and

2F , while

3F is minimized as the primary objective function.

Thus, 1ε (the constraint limit for

1F ) is varied between 0.1 and 67.6807, while

2ε (the

constraint limit for 2

F ) is varied between 0.0224 and 43.0336. Each constraint limit is

divided into 18 increments. First, 1ε is set at a particular value, and

2ε is incremented

within its range, resulting in 18 problems. Then, 1ε is increased by one increment, and

the process is repeated. This results in 18 18 324× = problems and presumably 324

solution points, which is approximately the same number of points used in Chapters 5

and 7 with the weighted sum method and the weighted global criterion method.

However, some problems have no feasible solution, and some problems yield duplicate

solution points. Consequently, there are fewer than 324 unique solution points shown in

the figures. Because the number of solution points is sparse, the actual Pareto optimal set

is outlined in the design space.

Compared to Figures 5.1 and 5.2, which indicate the Pareto set when determined

analytically, Figures 8.2 and 8.3 show an incomplete Pareto set. In fact, portions of the

179

set appear to be cut off. This is because of the values that are used for ε . Based on the

number of problems solved and the consequent increments for 1ε , the smallest value for

1ε is 3.854. The contour for this function-value is shown in Figure 8.2, and the plane

indicating points in the criterion space with this value for 1F are shown in Figure 8.3.

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

22.4114F =

132.0687F =

216.9994F =

13.854F =

Outline of

Analytical

Pareto-

Optimal Set

Figure 8.2: Pareto-Maximum Approach - 324 Problems – Design Space

Although this value is relatively small in terms of the complete range for 1F within the

Pareto set (0.1000 to 67.6807), restricting 1F to values above 3.854 excludes significant

portions of the set. The same effect is seen with 2

F , for which the smallest function

value is 2.4114. Similar restrictions are imposed on the upper limits for ε as well,

though the consequences are less severe.

180

0

40

60

20

10 20

30

F3

F2

F1

22.4114F =

13.854F =

20

40

60

40

10

20

30

F3

F2

F1

22.4114F =

13.854F =

Figure 8.3: Pareto-Maximum Approach - 324 Problems – Criterion Space

The above-mentioned phenomenon occurs because although the stipulated change

in ε is consistent (with each problem in the series), the concurrent change in the

function-values across the Pareto set is not. This inconsistent change in the function-

values is a consequence of the function gradients. With the functions in this example, the

gradients of the functions can be relatively small even for points that are not necessarily

close to a respective minimum, as shown in Figures 8.4 through 8.6. Thus, a relative

small change in a function-value (or active ε -constraint limit) can correspond to a

significant change in x, in the design space. Therefore, even if an ε -constraint limit has

a relatively small value (close to the minimum value of the corresponding function), it

can effect a relatively large portion of the Pareto optimal set in the design space. This

means that the success of the ε -constraint method in yielding the complete Pareto

optimal set for general problems is dependent on the gradients of the individual

objectives in addition to the number of problems that are solved. The trends in the

181

gradients discussed here will later be contrasted with gradient trends that occur as one

moves significantly farther away (in the design space) from the minima of the individual

objective functions.

1 2

3

0

1

2

3

50

100

150

x1

x2

F1

Approximate

Minimum of

F1

Figure 8.4: Surface for 1F

1 2

3

0

2

3

100

200

x1

x2

F2

Approximate

Minimum of

F2

Figure 8.5: Surface for 2

F

182

20

30

1 2

3

0

x1

x2

F3

Approximate

Minimum of

F3

Figure 8.6: Surface for 3

F

The difficulty in representing the complete Pareto optimal set can be remedied to

some extent by increasing the number of problems that are solved, and this is

demonstrated in Figures 8.7 through 8.9. As one would expect, solving more problems

provides more points with which to represent the Pareto optimal set. In addition, more

increments are used for ε , and this results in lower ε -constraint limits. As shown in

Figure 8.7, where 625 problems are solved, the solution points cross the previous

boundaries (in Figures 8.2 and 8.3) where 1

3.854F = and 2

2.4114F = . When 1225

problems are solved as shown in Figure 8.8, the lowest value for 1ε is 2.0308, and the

lowest value for 2

ε is 1.2513. Although this results in a more complete Pareto optimal

set (see Figures 8.8 and 8.9), portions of the set are still absent as compared to Figure 5.2.

Although increasing the number of ε -constraint problems that are solved can improve

183

results, ensuring that all portions of the Pareto optimal set are represented can be

difficult if not impossible.

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

22.4114F =

132.0687F =

216.9994F =

13.854F =


0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

22.4114F =

132.0687F =

216.9994F =

13.854F =

21.2513F =


184

40

60

20

10 20

30

F3

F2

F1

F3

F2

F1

10 20 30 40 0

20 40

60

20

Figure 8.9: Pareto-Maximum Approach - 1225 Problems – Criterion Space

8.2.3 Infeasible Problems and Duplicate Solutions

When the ε -constraint results are compared to the results of the weighted

methods in Chapters 5 and 7, they indicate relatively few points despite the fact that at

least 324 problems are solved. That is, the set of points representing the Pareto optimal

surface is less dense than the sets depicted in the previous chapters. The low number of

distinguishable solution points is a consequence of two phenomena: some problems have

no feasible solution, and some problems result in duplicate solutions. When 324

problems are solved (Figures 8.2 and 8.3), there are five infeasible problems; with 625

problems (Figure 8.7), there are ten infeasible problems; and with 1225 problems

(Figures 8.8 and 8.9), there are 29 infeasible problems. In general, infeasible problems

result when ε -constraint limits are relatively small, and such results cannot be avoided.

185

In addition, as suggested, many of the solutions are duplicates. Although it is

difficult to anticipate values of ε that result in duplicate solutions, duplicate solutions are

guaranteed to surface if the following condition holds for two consecutive problems:

( )*i i s

F i sε ≥ ∀ ≠x (8.6)

If the condition in (8.6) is satisfied, then the solution to the ε -constraint problem is

always the point that minimizes s

F . When the Pareto-maximum approach is used to set

the limits for ε , (8.6) is easily satisfied with many problems and duplicate solutions are

common. On the other hand, using the primary-objective approach can reduce the

number of duplicate solutions. However, other difficulties may arise as shown in Figures

8.10 and 8.11, which represent the results when 625 problems are solved.

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F1-minimum

F3-minimum

F2-minimum

132.0687F =

216.9994F =

Figure 8.10: Primary-Objective Approach – 625 Problems – Design Space

186

40

60

20

10 20

30

F3

F2

F1

0

10

20

F3

F2

F1

20 40

60

20

40

132.0687F =

Figure 8.11: Primary-Objective Approach – 625 Problems – Criterion Space

Using the primary-objective approach results in a higher number of unique

solution points and a denser cluster of points. However, a pitfall of this approach is that a

significant portion of the Pareto optimal set is excluded. This is because in this case, with

(8.4) the value for 1ε is always less than 32.0686 as dictated by ( )1

*s

F x , and the value

for 2

ε is always less than 16.9994 as dictated by ( )2*

sF x . The former restriction results

in an especially severe misrepresentation of the Pareto optimal set as shown in Figure

8.11. Generally, one cannot anticipate this condition.

When the primary-objective approach is used, 91 problems have no feasible

solution. This represents a significant increase from the results with the Pareto-maximum

method. However, keep in mind that the range of values for the ε -constraint limits is

decreased significantly, resulting in more problems with relatively low values for ε .

This results in a relatively high percentage of problems with no feasible solution.

187

In summary, portions of the Pareto optimal set can be inadvertently excluded in

two ways. As suggested with respect to the Pareto-maximum approach, sections of the

Pareto optimal set are skipped when the increments for ε are too large (not enough

problems are solved). This difficulty is remedied to some extent by solving more

problems. With respect to the primary-objective approach, sections of the Pareto optimal

set can also be skipped if the upper limits for ε are too small. There is no remedy for this

difficulty. Consequently, when one is concerned with depicting the complete Pareto

optimal set, the primary-objective approach should not be used.

8.2.4 Normalization

One advantage to the ε -constraint method is that normalization is not necessary

and essentially has no effect on the results. This is demonstrated with Figure 8.12, in

which 1F and

2F are normalized such that their values are between zero and one, and the

ε -constraint limits are varied between zero and one. For the most part, the results in

Figure 8.12 are the same as those in Figure 8.7. Similar results were seen in the criterion

space as well.

The results in Figure 8.12 are explained by considering that method parameters

(weights) are most effective when their values are significant relative to the

corresponding function values, as discussed in Chapters 5 and 7. In addition, using a

convex combination of normalized objective functions provides the user with clear upper

and lower limits for the method parameters. With the ε -constraint method, however,

these characteristics are naturally present without transforming the objective functions.

The ε -constraint limits clearly are significant with respect to the function values, and the

user has clear method-parameter limits stemming from either the primary-objective

188

approach or the Pareto-maximum approach. Thus, objective functions need not be

normalized or transformed in any way.

F1-minimum

F2-minimum

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F3-minimum

Figure 8.12: Pareto-Maximum Approach and Normalization - 625 Problems – Design

Space

8.2.5 Alternate Primary Objective

In addition to selecting the increment with which the ε -constraint limits are

varied, one must also select the primary objective function. Often this decision is based

on problem information. However, when one depicts the Pareto set with ε -constraint

limits acting only as mathematical parameters, determining which primary objective

function is most appropriate, can be difficult. Using different functions can have

significantly different affects, as shown in Figures 8.13 and 8.14, where 1F is used as the

primary objective rather than 3

F .

189

F1-minimum

F2-minimum

0.0

0.5

1.0

1.5

2.0

2.5

0.0 0.5 1.0 1.5 2.0 2.5

x2

x1

F3-minimum

Figure 8.13: Pareto-Maximum Approach with 1F as the Primary Objective - 1225

Problems – Design Space

40

60

20

10 20

30

F3

F2

F1

F3

F2

F1

10 20 30 40 0

20 40

60

20

Figure 8.14: Pareto-Maximum Approach with 1F as the Primary Objective - 1225

Problems – Criterion Space

190

In this case, with the Pareto-maximum approach, 3

11.2757 20.0725ε≤ ≤ . As indicated

in Table 8.1, the limits for 2

ε are set such that 2

0.0224 43.0336ε≤ ≤ . When compared

to previous results, Figures 8.13 and 8.14 indicate a trend in terms of the clustering of

solution points, which is discussed in the next section.

8.2.6 Clustering

With the results shown so far, including Figures 8.13 and 8.14, the solution points

tend to cluster towards the minimum of the primary objective function, in the design

space, whereas there is relatively little clustering in the criterion space. This distinction is

the clearest in the cases when 3

F is the primary objective. The difference between the

Pareto sets in the design space and the Pareto sets in the criterion space is a consequence

of the ε -constraint limits representing objective-function values and being altered in

equal increments. In the design space, as an ε -constraint limit increases, the consequent

solution point tends to move away from the minimum of the corresponding objective

function and towards the minimum of the primary objective function. As noted in section

8.3.2, the functions in this problem are relatively flat in the neighborhood of the

perspective minima. However, as one moves farther away from the minimum of an ε -

constraint function and towards the minimum of the primary objective function, the

gradient of the ε -constraint function may eventually increase. This in turn results in a

relatively small change in x, for a given change in the function-value stipulated by the

incremental change in the ε -constraint limit. Alternatively, in the criterion space,

consistent changes in ε -constraint limits result in consistent changes in solution points,

because ε -constraint limits correspond directly to function values. In addition, for small

changes in ε , there is a linear relationship between the change in ε and the change in the

191

primary objective, as implied by (8.3). Thus, although the ε -constraint method tends to

leave out portions of the Pareto optimal set, the portions that are in fact indicated can

involve more evenly distributed solution points in the criterion space, when compared to

the results of weighted methods. The direct correspondence between the method-

parameter ε and the objective function values does not exist with the weighted methods

discussed in previous chapters.


In this chapter, we have conducted a much needed, previously unavailable, in-

depth study of the ε -constraint method and its ability to depict the Pareto optimal set.

We have determined that there is little utility in normalizing the constrained objective

functions. We have demonstrated the repercussions of using an alternate primary

objective, discovering the potential tendency of solution points to cluster near the

minimum of the primary objective. In addition, we have explained why this clustering

can occur, in terms of objective-function gradients. We have provided numerical analysis

of two approaches for setting the ε -constraint limits, identifying the pros and cons of

each approach, and we have compared the results of this analysis to those obtained with

weighted methods. We have evaluated the tendency of the ε -constraint method to yield

infeasible problems and duplicate solutions, and we have provided new understanding as

to why these deficiencies surface.

Although some authors suggest that ε -constraint is preferred to the weighted

methods (Cohon and Marks, 1973; Cohon and Marks, 1975; Goicoechea et al, 1976), in

particular the weighted sum method, we have shown that this should not necessarily be

the case, especially when there are more than two objective functions. In such cases, the

192

ε -constraint method can result in problems with no feasible solution and in duplicate

solutions, both of which constitute wasted computational time.

When there are just two objective functions (just one ε -constraint), the Pareto-

maximum and primary-objective approaches for setting the limits of ε are identical.

Using such an approach with a bi-criterion problem significantly reduces the possibility

of an infeasible problem or duplicate solutions. This explains why most of the

applications for the ε -constraint method only involve bi-criterion problems, although

this explanation is not provided in the current literature.

The two approaches presented in this work for determining limits for ε provide a

trade-off in terms of performance. Although the primary-objective approach can reduce

the number of duplicate solutions, it can result in a severely incomplete depiction of the

Pareto optimal set. Alternatively, the Pareto-maximum approach can provide an

improved description of the Pareto optimal set if small enough increments are used for ε ,

but one is guaranteed to get some duplicate solution points with most problems.

We find that the weighted methods of Chapter 5 and 7 generally require less

computational time and provide a more complete representation of the Pareto optimal set,

especially when there are more than two objective functions. Theoretically, when used

for a posteriori articulation of preferences, the primary difference between the weighted

methods and the ε -constraint method is that altering the method parameters with the

weighted methods modifies the utility function contours in the criterion space, whereas

altering the method parameters with the ε -constraint method modifies the actual feasible

criterion space and the consequent Pareto optimal set, as demonstrated in Figure 8.1.

Thus, especially when there are more than two objective functions, this method should be

193

reserved for use with a priori articulation of preferences. However, this method can only

be used in such a capacity when information in the problem statement provides the

appropriate, physically significant ε -values. Then, preferences are articulated in terms

of specified limits on the objective functions rather than the relative importance of

objectives. These limits can be modified and the problem resolved when a few different

Pareto optimal points are needed or required, assuming the solutions to the consequent

problems exist and are unique.

The primary conclusions and guidelines for use are listed as follows:

1) The method can result in infeasible problems and/or duplicate solution points,

especially when there are more than two objective functions, thus resulting in wasted

computational time.

2) When there are more than two objective functions, the ε -constraint method should

not be used for a posteriori articulation of preferences. Rather, it should be reserved

for problems that lend themselves to physically significant ε -constraint limits.

3) When using the ε -constraint to provide multiple solution points for general

problems, one should use the Pareto-maximum approach to set limits for ε .

4) The success of the method in depicting the complete Pareto optimal set depends on

the gradients of the individual objectives in addition to the number of problems that

are solved (increments for ε ). Nonetheless, there is no way to guarantee that the

complete Pareto optimal set is represented.

5) Normalizing the constrained objective functions has little or no effect on the results.

194

6) Solution points may tend to cluster around the minimum of the primary objective in

the design space, but clustering is minimal in the criterion space, which is most

relevant for decisions made with a posteriori articulation of preferences.

195

CHAPTER IX

SYSTEM IDENTIFICATION PROBLEM FOR A SIMPLIFIED CRASH MODEL

9.1 Introduction

Up to this point, example problems have only provided theoretical or

computational insight into how various multi-objective optimization (MOO) methods

work. However, the true merit of MOO is its ability to provide engineering insight and

incorporate information that single-objective optimization may not be able to. Thus, in

this chapter, we draw on the work of Marler and Arora (2004a), and use different MOO

techniques to solve a complex system identification problem for a simplified dynamic

crash simulation. In general, we propose a MOO-based approach to system

identification.

As suggested in section 1.3, automotive crash simulations for an entire vehicle

can be demanding computationally, and a simplified model can provide useful upstream

design information relatively quickly. One approach to developing such a model

involves representing the vehicle with a lumped-mass system in which connections

between the masses are represented by restoring forces that are approximated with a

series of basis functions. Kim et al (2001) and Kim and Arora (2003a) describe a system

identification problem for dynamic systems, in which one minimizes the error between

approximate restoring forces and actual force-data, in order to determine coefficients for

the basis functions and thus create a simplified crash model. There is one error function

for each restoring force, and this function is defined essentially as the difference between

the actual force-data and the value of the approximate force.

196

However, to date, with problems that involve more than one error function, the

functions have simply been added together, and the problem has been treated as a single-

objective problem. MOO has not been fully utilized in terms of obtaining results or in

terms of interpreting and gleaning from the results practical information. Thus, the

purpose of this chapter is to consider each error function as a separate objective function

and use various MOO methods in the context of the above-mentioned system

identification problem. We consider a priori articulation of preferences, whereas a

posteriori articulation is considered in Chapter 10. This study corresponds to objective 3

in section 1.4, concerning significant applications for MOO methods.

We elaborate on and further demonstrate the newly revealed characteristics of the

MOO methods that are discussed in previous chapters. In accordance with the discussion

in Chapter 6, the necessity for normalization with the min-max approach is demonstrated.

Computational difficulties that can arise with normalization are identified and resolved.

Although in section 3.8 we pointed out that methods with no articulation of preferences

generally provide an arbitrary solution relative to the Pareto optimal set, in this chapter,

we demonstrate that with special cases when the Pareto optimal set has a specific form,

methods with no articulation of preferences can in fact provide the most suitable solution

point. That is, articulating preferences may not always be beneficial. Nonetheless, by

using a priori articulation of preferences, the significance of the weights that was

discussed in Chapter 4 is demonstrated. Computational requirements for the different

methods are contrasted, and it is shown that changes in the weighting vector can have

significant effects on computational demands.

197

9.1.1 Motivation and Objectives of the Present Study

System identification problems have been studied extensively (Masri and

Caughey, 1979; Hollowell, 1986; Radwan and Hollowell, 1990; Masri et al, 1993; Ni et

al, 1999; Kim and Arora, 2003b). In addition, some work has been completed using

MOO with design for crash worthiness (Yang et al, 1994). However, MOO methods

have not been exploited with system identification problems, such as the one that is used

in this paper for the development of a simplified crash model. In terms of a dynamic

system identification problem, if there are multiple degrees of freedom and/or multiple

restoring forces, then multiple error functions must be minimized. Up to this point, only

a basic objective sum has been used to combine these objective functions and provide a

single solution point. Evaluating different potential solutions and user-preferences with

regard to the different error functions, can provide additional insight into the system

identification problem. Furthermore, the problem provides a substantially complex test

bed for comparing the MOO methods discussed in this dissertation.

Thus, the objectives for this study are stated as follows:

1) Use a system identification problem to study the MOO methods presented in previous

chapters, with no articulation of preferences and with a priori articulation of

preferences.

2) Investigate further the necessity for and the potential issues associated with

normalization.

3) Use MOO to study the relationships between the error functions in the system

identification problem. Determine the general form of the Pareto optimal set and

evaluate qualitatively the trade-offs between different error functions.

198

4) Examine the effects of articulating different sets of preferences (emphasizing

different objective functions).

9.2 Overview of Dynamic System Identification

Problems

Generally, with system identification problems, one minimizes the error between

analytical data from an approximate model, and given data from experiments or

numerical simulations. For the present problem, each restoring force is approximated

with a series of basis functions. Each basis function is multiplied by a coefficient called

an expansion parameter, and these coefficients are the design variables for the

optimization problem. The approximate restoring forces are used in the equations of

motion to determine approximate values for acceleration, velocity, and displacement.

Then, an error function is developed using the given data and approximate data for the

acceleration, velocity, or displacement for each degree of freedom. Alternatively, an

error function may be developed based on the data given directly for each restoring force.

In any case, each error function is minimized in a MOO problem. The problem is

constrained by stipulating that the displacements and velocities predicted by the

equations of motion and the approximate model must coincide with the values from the

experimental data. The problem is summarized as follows:

Find: b = vector of basis function coefficients (expansion parameters) for each

restoring-force approximation function

to minimize: ( )E b = a vector of error functions

199

subject to: ( )dlh b : displacement defined by the equation of motion must equal the

actual displacement, for each degree of freedom

( )vlh b : velocity defined by the equation of motion must equal the

actual velocity, for each degree of freedom

where 1,2, ,dl DOF= … , 1, 2, ,

vl DOF= … , and DOF is the number of degrees of

freedom.

Details concerning different types of system identification problems, each with

different types of error functions and basis functions, is beyond the scope of the present

research. Here, we consider a simple single degree-of-freedom dynamic system in order

to describe the basic system identification problem and to outline its formulation. With

the system depicted in Figure 9.1, M is the mass, and ( )FS t represents the time

dependent restoring force that needs to be determined.

M

x

v0

( )FS t

Figure 9.1: Single-Degree-of-Freedom Dynamic Model

The equation of motion is given as follows:

( ) ( ) ( ) ( ) 0 0 for 0 ; 0 0; 0 M x t FS t t T x x v+ = ≤ ≤ = =�� (9.1)

200

The approximation for ( )FS t is represented as follows:

( ) ( )0

,

mappr

j j

j

FS t b t

=

= Φ∑b (9.2)

where j

b are the expansion parameters treated as design variables for the system

identification problem, and ( )jtΦ are the basis functions for 0 t T≤ ≤ . T represents the

total elapsed time. 1m + is the number of terms used to represent the force. There is one

term for each time break point, where time break points delineate the periods in time

where basis functions have a non-zero value. Alternatively, a time step is used to

discretize the time interval T for numerical integration. A time step represents the point

in time at which the forces and kinematic quantities are evaluated, and data is given.

In general, the error for the dynamic system is expressed as follows:

( ) ( ) ( ), ,

given appre t u t u t= −b b (9.3)

where ( )givenu t is the given data and ( ),

appru tb is the approximate data. u may

represent acceleration, velocity, or displacement and may be manipulated as necessary

with numerical differentiation or integration. Alternatively, the problem can be

formulated such that u represents force, as discussed in the next section. The error is

given in scalar form using an L2 norm, which is determined as follows with numerical

integration:

( ) 2

0

( ) ,T

E e t dt= ∫b b (9.4)

The system identification problem entails determining b in order to minimize the

error function defined in (9.4), where ( ),

appru tb is determined either from (9.1) in the

case of given kinematic data or from (9.2) in the case of given force data. This in turn

201

results in a simplified expression for acceleration, velocity, displacement, or force. The

simplified expression can then be used to evaluate the system relatively quickly during

design and analysis processes.

9.3 Multi-objective Problem Statement

If there are multiple degrees of freedom and/or multiple restoring forces, then

multiple error functions must be minimized in the context of a MOO problem. The

multi-objective problem that we consider is developed in Kim (2001) and is depicted in

Figure 9.2.

x1

x3

x2

v0

FS3 FS6

FS5

FS4 FS2

FS1

M3

M2

M1

Figure 9.2: Lumped Mass Representation of the Front End of a Car

This model represents a simplification of the front end of a car. Three masses comprise a

three-degree-of-freedom system with six restoring forces. 1

M represents the frame and

car body; 2

M represents the wheel and suspension system; and 3

M represents the engine.

The weights of masses one, two, and three are 1665 lbs, 100 lbs, and 305 lbs,

respectively.

202

The restoring force 1

FS represents the sheet metal of the fenders and the occupant

compartment. 2

FS represents the intermediate frame, which connects the cross member

(2

M ) to the occupant compartment (1

M ). The structural components attaching the engine

to the occupant compartment (vehicle body) is modeled with 3

FS . The structural

members between the cross member and the rigid wall are modeled with 4

FS . The

structures connecting the engine to the cross member are modeled with 5

FS . Finally, the

radiator and surrounding structure is modeled with 6

FS .

The equations of motion for the system in Figure 9.2 are given as follows:

( ) ( ) ( ) ( ) ( ) ( )1 1 1 2 3 1 1 0 0 for 0 ; 0 0; 0M x t FS t FS t FS t t T x x v+ + + = ≤ ≤ = =�� (9.5)

( ) ( ) ( ) ( ) ( ) ( )2 2 2 4 5 2 2 0 0 for 0 ; 0 0; 0M x t FS t FS t FS t t T x x v− + + = ≤ ≤ = =�� (9.6)

( ) ( ) ( ) ( ) ( ) ( )3 3 3 5 6 3 3 0 0 for 0 ; 0 0; 0M x t FS t FS t FS t t T x x v− − + = ≤ ≤ = =�� (9.7)

9.3.1 Design Variables and Objective Functions

In this case, force data is provided for each restoring force, so there are six error

functions (one for each restoring force). Each error function is defined in terms of time,

as follows:

( ) ( ) ( )

( ) ( )20

0

, ,

; 1, 2, , 6

given appr

i i i

given

i ij j

j

e t FS t FS t

FS t b t i

=

= −

= − Φ =∑

b b

…

(9.8)

Then, a scalar form of the error, i

E , is determined using (9.4) and represents an objective

function. ijb represents the j

th coefficient (expansion parameter) for the i

th restoring force

and constitutes a design variable. i ranges between one and six, indicating six restoring

forces. j ranges between zero and 20, indicating 21 time break points and consequently a

203

series of 21 basis functions (and expansion parameters) for each restoring force. Thus,

there are 6*21 126= expansion parameters, which provide the design variables. The

total elapsed time T is 0.1s. The force and consequent error are evaluated at 101 time

steps. The design space is defined in terms of the 126 expansion parameters, and the

criterion space is defined in terms of the six error functions. The vector of expansion

parameters *i

b is the point that minimizes the error function i

E .

Basic Hat functions are used for the basis functions, ( )jtΦ , and are defined as

follows:

( ) ( )

( ) ( )

1

1 1 1

1 1 1

1

0; 0

; ( )

;

0;

j

j j j j j

j

j j j j j

j

t t

t t t t t t t

t

t t t t t t t

t t T

−

− − −

+ + +

+

≤ <

− − ≤ <Φ =

− − ≤ <

≤ ≤

(9.9)

where 1i i

t t t−= − ∆ ,

1i it t t+= + ∆ , and t∆ is an increment of time, independent of the

time between break points. Other basic basis functions are considered in Kim (2001), but

Hat functions result in a problem that can be solved relatively quickly, providing an

accurate solution. In addition, when Hat functions are used, ijb represents the value of

the restoring force i

FS at the ( )1th

j + time break point. Consequently, with this

problem, force-time plots that illustrate the value of the forces at each time step, provide

a general representation of the solution point in the design space (values of the expansion

parameters at each time break point).

9.3.2 Constraints

The final displacements and final velocities, which are determined by numerically

integrating the equations of motion, are constrained as follows:

204

( )

( )

( )

1

2

3

22.1292585

14.2233772

14.0722970

x T in

x T in

x T in

=

=

=

(9.10)

( )

( )

( )

1

2

3

111.088217

64.5102012

66.5444159

v T in s

v T in s

v T in s

= −

= −

= −

(9.11)

These constraints are normalized when implemented in the optimization routine.

IDESIGN (Arora, 1989), which uses sequential quadratic programming (SQP) (Arora,

2004), provides the optimization engine. All gradients are determined using forward

difference approximations.

9.3.3 Multi-objective Optimization Formulations

The multi-objective methods that are discussed in Chapters 4 through 7 are used

to solve the problem described above. The basic formulations for these methods are

restated in this section in terms of the system identification problem. To begin with, the

objective sum entails minimizing the following aggregate function:

( )6

1

i i

i

U w E

=

=∑ b (9.12)

where i

w is a scalar weight. The weights in (9.12) and in all of the formulations that

follow are first set such that =w 1 , in anticipation of using no articulation of preferences.

However, these weights are altered and discussed in later sections. The Hessian of the

objective sum function is positive definite, and the constraints are linear. Therefore, the

optimal solution is a global optimum.

The function for a basic global criterion is shown as follows:

205

( )( )

1

6

1

p po

i i i

i

U w E E

=

= − ∑ b (9.13)

where o

iE is a component of the utopia point (the minimum of a single objective) and p is

a positive scalar. The function in (9.13), with an exponent of 2, is also called a distance

function, because it minimizes the Euclidean distance between the solution point and the

utopia point, in the criterion space.

The standard min-max method entails minimizing the following function:

( )maxi i

i

U w E= b (9.14)

The alternate min-max formulation, which does not involve the potential discontinuities

that (9.14) does, is formulated with an additional design variable λ and additional

constraints, as follows:

Find: λ , b (9.15)

to minimize: λ

subject to: ( ) 0; 1, 2, , 6i i

w E iλ− ≤ =b …

This particular problem does not lend itself to use with the ε -constraint method.

As determined in Chapter 8, the ε -constraint method is only appropriate for a priori

articulation of preferences with problems that suggest physically significant constraint

limits or for a posteriori articulation of preferences in special cases. Consequently, this

method is not considered in this chapter and is reserved for use with the application in

Chapter 10.

206

9.4 Independent Function Minima

As a point of reference and for use as components of the utopia point, the

function-comparison matrix (Table 9.1) indicates the results when each error function is

minimized independently. This matrix is comparable to those used in previous chapters.

1E

2E

3E

4E

5E

6E

at 1*b 6.44472E+02 1.09050E+07 1.26576E+07 1.72028E+07 2.53309E+06 1.47890E+07

at 2*b 1.27455E+06 2.33989E+04 1.03320E+07 1.32487E+07 5.51764E+06 1.42074E+07

at 3*b 4.56590E+05 7.58594E+06 6.62455E+03 1.53035E+07 5.92429E+06 1.13890E+07

at 4*b 2.13529E+06 7.62369E+06 1.22524E+07 5.66785E+04 1.55235E+06 8.52363E+06

at 5*b 8.44348E+05 1.37686E+07 1.42703E+07 1.52967E+07 1.64286E+03 1.19213E+07

at 6*b 6.31957E+05 8.61994E+06 1.01826E+07 9.34399E+06 2.00624E+06 6.79912E+03

Table 9.1: Function-Comparison Matrix for System Identification Problem

The lightly shaded boxes indicate the maximum value of a function, when that function is

evaluated at each of the six points; this is the Pareto-maximum. The darker boxes

forming the diagonal of the table indicate the minimum of each function, i.e., the utopia

point, which is used in some multi-objective formulations. The different function-

minima correspond to significantly different points in the design space and in the

criterion space. Thus, the utopia point is unattainable, and these clearly are conflicting

objectives; what minimizes one objective results in a significant increase in other

objectives. We will show how the utopia point can be useful in evaluating other multi-

objective solutions.

207

9.5 Results: No Articulation of Preferences

In this section, the system identification problem is solved with no articulation of

preferences, providing a basis for comparison when preferences are incorporated and

providing initial insight into the multi-objective problem. We determine the general

nature of the Pareto optimal hypersurface and how the different objective functions are

related. In addition, we demonstrate the necessity for normalization with min-max

formulations. Note that the results that are determined using the objective sum method in

this section represent initial results obtained by Kim (2001).

9.5.1 Error Results

The results from the methods in section 9.3.3 are shown in Table 9.2 with respect

to the criterion space. The form of Table 9.2 is similar to that of Table 9.1.

1E

2E

3E

4E

5E

6E

Objective Sum 6.57680E+02 2.34034E+04 6.62609E+03 5.66874E+04 1.64374E+03 6.81131E+03

Distance Function 6.54815E+02 2.34031E+04 6.62756E+03 5.66898E+04 1.64510E+03 6.81066E+03

Min-max 5.23763E+04 4.94649E+04 4.15190E+04 5.66785E+04 5.63917E+04 5.55293E+04

Alternate Min-max 3.91229E+04 5.61206E+04 5.52465E+04 5.66785E+04 4.20225E+04 2.41501E+04

Table 9.2: Error-Function Results with No Articulation of Preferences

Each row indicates the values of the objective functions at the solution point determined

with a particular method. Each column indicates the values of a single objective function

when evaluated at the solution points obtained with different methods.

Table 9.2 provides information pertaining to the general form of the Pareto

optimal hypersurface, which is discussed as follows. The solution points for the

208

objective sum and the distance function are similar in the criterion space and in the

design space, which is a common occurrence when weights are not incorporated.

However, note that the objective-function values obtained with these methods are similar

to those for the utopia point, shown in Table 9.1. In fact, comparison of Tables 9.1 and

9.2 suggests that the utopia point is nearly attainable, though still outside the feasible

criterion space. However, as seen in Table 9.1, when one actually minimizes an

individual error function, other functions assume significantly high values. These two

observations suggest that the trade-offs (discussed in Chapters 4 and 8) between objective

functions, as one moves along the Pareto optimal hypersurface, are rather high. That is,

assuming one is interested only in Pareto optimal solutions, a small change in one

function can correspond to a relatively large change in another function. This scenario is

illustrated in Figure 9.3 for two general objective functions in the criterion space.

1 2 3 4 5 6 7

1

2

3

4

5

6

7

F1

F2

Pareto Optimal

Set

Z

Utopia

Point

Approximate

Solution to

Objective-

Sum Problem

High Trade-

off between

F1 and F2

A

B

Figure 9.3: Pareto Optimal Curve with High Trade-offs

209

The dashed and dotted lines, representing the boundary of the feasible criterion

space, indicate the Pareto optimal set. The portions of the Pareto optimal set that are

indicated with dashed lines rather than a dotted line, depict relatively high trade-offs

between the objectives and represent points that are nearly weakly Pareto optimal. For

example, points A and B both are Pareto optimal; it is not possible to move from one

point and improve an objective without resulting in a detriment to the other objective.

However, moving from A to B results in a significant reduction (improvement) in 2

F and

only a negligible increase in 1F . Consequently, for all practical purposes, there is no

significant advantage to using point A rather than point B. Only Pareto optimal points

that are in the neighborhood of the approximate solution to the objective sum are

significant, and assuming the objective functions have been transformed such that they

have similar ranges, these points are relatively close (compared to other Pareto optimal

points) to the utopia point. The idea that many of the points on the Pareto optimal

hypersurface for this problem are nearly weakly Pareto optimal with relatively high trade-

offs between objective functions is demonstrated further in following sections.

The number of necessary SQP iterations for the objective sum, global criterion,

min-max, and alternate min-max methods was 177, 226, 153, and 205, respectively. The

objective sum method requires the fewest number of iterations. Note that convergence in

the value for the objective-functions in (9.12) through (9.15) is relatively fast, while

additional iterations are used to refine design-variable values.

These results raise interesting issues with regards to the two min-max

formulations. Although the standard min-max formulation shown in (9.14) can be

210

discontinuous and computationally intractable, in this case it is successful and actually

requires fewer iterations and function evaluations than the alternate min-max

formulation. This could be a consequence of the additional constraints used in the

alternate min-max method. Such constraints can require more computation time.

Alternatively, it is possible that these results are a consequence of the unique form of the

Pareto optimal set. With this particular Pareto optimal set, it is possible to reduce one

objective function while having only a gradual increase in other functions. Consequently,

there may not necessarily be a significant discontinuity in the design space when the

objective function with the maximum value changes in the criterion space. Note that

minimizing (9.14) did not simply result in the minimization of just one function without

consideration of the other functions. We verified that during the optimization process,

the error function in (9.14) with the maximum value changed from iteration to iteration.

Further analysis of the results with the min-max approach yields additional

information concerning the solution points in terms of Pareto optimality. At the solution

to the alternate min-max formulation, the value of λ is 5.667851E+04, which is the same

as the value for 4

E only. Thus, the only active function-constraint is the 4

E constraint.

Comparing the min-max results with the utopia point in Table 9.1 indicates that the value

for 4

E when the min-max approach is used is the same as when 4

E is minimized

independently. However, the values of the design variables, the 126 expansion

parameters, are significantly different for the two cases. Thus, the minimum of 4

E and

the solutions to the min-max formulations are non-unique and consequently non-Pareto

optimal; they are only weakly Pareto optimal (or approximately weakly Pareto optimal).

211

Minimizing an objective sum provides a sufficient condition for Pareto

optimality; it always results in a Pareto optimal point. This is true for the distance

function as well but not for min-max methods. Solving the min-max formulations is

sufficient only for weak Pareto optimality. By definition of Pareto optimality, a point at

which each of the error functions increases relative to another point is not Pareto optimal,

and this is almost the case with the min-max results. When compared to the first two

solutions in Table 9.2, the min-max solutions show an increase in each error function

except for a minor decrease in 4

E . Therefore, if one views all four values for 4

E as

being approximately equal, the min-max solutions can be considered only weakly Pareto

optimal, comparable to the points on the dashed lines in Figure 9.3.

The above-mentioned results highlight the necessity for normalization, especially

with the min-max approach is used. When using a min-max method in general, reaching

the minimum of one function prevents the further reduction of any other functions, and

this is what happens with this problem. Essentially, 4

E is the only function that is

considered in the multi-objective problem. However, as discussed in Chapter 6, if the

functions are transformed such that they have the same range, then it is not possible to

achieve the minimum of any single function unless the utopia point is attainable.

Consequently, as we will show, using normalization with this problem is advantageous

even though the objective functions are inherently non-dimensional, have similar orders

of magnitude, and have the same physical significance.

9.5.2 Force Results

Whereas the previous section evaluates results in terms of the error functions, this

section concerns the approximated restoring forces. Recall that the nature of the plots for

212

the restoring forces reflects the solution in the design space, as discussed in section 6.3.1.

The original force-time data is shown for the six restoring forces in Figure 9.4. Results

for the objective sum method, the min-max method, and the alternate min-max method

are shown in Figures 9.5, 9.6, and 9.7 respectively. The results for the distance function

are nearly identical to those for the weighted sum, as suggested in the previous section.

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.4: Force-Time Data

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.5: Force-Time Results with the Objective Sum Method

Different

from Data

213

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.6: Force-Time Results with the Min-max Method

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.7: Force-Time Results with the Alternate Min-max Method

All of the multi-objective methods provide reasonable though slightly different

force-approximations. This is because although the error for the complete simulation

may appear significant, the error at each time step is acceptable. However, in most cases,

the approximated curves are not as smooth as those for the original data are. Areas where

Different

from Data

Different

from Data

214

an approximate-force curve is significantly different from the actual-force curve are

indicated with dotted ellipses.

The curve for force 4 is particularly poor. However, the error for force 4 is at its

individual minimum; it cannot be improved. Thus, in the case of force 4, the indicated

inaccuracies are a consequence of the general modeling approach, not the multi-objective

approach.

The curves for force 1 and force 5, determined with the alternate min-max

approach, are also particularly poor, especially during the initial stages of the simulation.

Especially in the case of force 1, these force results coincide with relatively high error,

indicated in Table 9.2.

The above-mentioned force results suggest two potential preference sets. First,

judging from Figures 9.5, 9.6, and 9.7, forces 2, 3, 4, and 6 have the highest values.

Thus, when preferences are articulated, the accuracy with which these forces are

approximated should be considered relatively important. In this case, the error associated

with force 1, which represents the sheet metal of the front fenders and occupant

compartment, and the error associated with force 5, which represents the connection

between the engine and the cross member, are viewed as being relatively insignificant.

Alternatively, a second set of preferences should be considered by noting that in Figure

9.2, forces 1, 2, and 3 all contact the passenger compartment. Therefore, one may

consider the error for these forces to be relatively important.

215

9.6 Normalization

In this section, we use the system identification problem to discover potential

computational issues when standard normalization is used. In addition, the advantages of

using normalization with the min-max approach are demonstrated.

In response to the above-mentioned results and in anticipation of assigning

weights to the error functions for articulation of preferences, the objective functions are

first normalized as follows:

( )max

o

i inorm

i o

i i

E EE

E E

−

=

−

b (9.16)

where norm

iE is the normalized version of ( )

iE b . Using (9.16) to normalize the error

functions such that their range is approximately between zero and one can not only

provide advantages when the min-max method is used, as discussed in the previous

results-section, but can also make selecting relevant weights easier as discussed in

Chapters 4 and 5. In this case, max

iE is determined as the Pareto maximum such that

( )max

1 6

max *i i j

jE E

≤ ≤

= b , where *j

b is the point that minimizes the jth

objective function.

Note that normalizing or scaling the objective functions does not alter the nature of the

Pareto optimal set.

When the normalization scheme shown in (9.16) was used, interesting

computational issues arose which supplement the discussion in Chapter 5 and

necessitated a minor alteration. (9.16) was tested with objective sum method.

Convergence, which is marked by a relatively low value for the gradient of the

Lagrangian, was achieved in just six iterations, and the results were significantly different

from those shown in Table 9.2. In addition, the solution clearly was not Pareto optimal;

216

all error-values were higher than those achieved with the original objective sum. Even

when the convergence parameter was decreased by five orders of magnitude, the solution

was not particularly accurate. This is a result of the normalization method and the range

of values for the objective functions. As shown in Table 9.1, all of the objective

functions have an approximate range between 3

10 and 7

10 . Consequently, when (9.16)

is applied, not only are the objective functions divided by a large number (7 3

10 10− ), but

the gradients of the objective functions (and the gradient of the aggregate objective

function) are as well. Computationally, this can result in premature convergence,

meaning the gradient may be relatively close to zero, although the theoretical solution

may not yet be determined. Although it is generally accepted that normalizing the

objective functions is advantageous, these results suggest that there can be consequent

computational difficulties.

To remedy this problem, norm

iE is multiplied by a factor of

710 . In this way, all of

the normalized objective functions still have the same orders of magnitude and the same

ranges. However, rather than being between zero and one, the values for norm

iE are

between zero and 7

10 .

Results using normalization and no articulation of preferences are shown in Table

9.3, which is comparable to Table 9.2. The results with the objective sum method and the

distance function method are similar to the results obtained without normalization. In

this case, the objective sum method requires the fewest number of iterations. The results

with the min-max methods, however, are improved significantly. Using normalization

resolves the issue of the min-max methods determining the independent minimum of 4

E .

In fact, the function-constraints for 1

E , 4

E , and 6

E are all active, and consequently the

217

normalized version of these functions all have approximately the same value at the

solution point.

1E

2E

3E

4E

5E

6E

Objective Sum 6.45137E+02 2.34108E+04 6.62830E+03 5.66953E+04 1.64333E+03 6.82052E+03

Distance Function 6.45740E+02 2.34075E+04 6.62985E+03 5.66972E+04 1.64488E+03 6.81586E+03

Min-max 6.46994E+02 2.34151E+04 6.64139E+03 5.66988E+04 1.64872E+03 6.81659E+03

Alternate Min-max 6.47010E+02 2.34141E+04 6.64124E+03 5.66989E+04 1.64953E+03 6.81669E+03

Table 9.3: Normalized Error-Function Results with No Articulation of Preferences

The two min-max methods provide similar results, but again, the alternate min-

max method is more demanding computationally. As suggested earlier, this can be

attributed, in part, to the additional constraints.

Compared to the results for the objective sum method and the distance function

method, all of the error-functions with the min-max method increase except for 4

E ,

which remains unchanged. Thus, as was the case with un-normalized functions, the min-

max methods result in weakly Pareto optimal solutions. Whether normalization is used

or not, the objective sum method provides a solution that is close to the utopia point and

is guaranteed to be Pareto optimal, and this method requires relatively few optimization

iterations.

The force results for the alternate min-max method with normalization, shown in

Figure 9.8, indicate an improvement over those shown in Figure 9.7, particularly with

respect to forces 1 and 5. Results for the standard min-max method indicated similar

218

improvements. Thus, as determined in Chapter 6, especially when using the min-max

approach, function-normalization is advantageous.

9.7 Results: A Priori Articulation of Preferences

In this section, the system identification problem is used to evaluate the

performance of different MOO methods when preferences are articulated a priori. In

addition, the postulate concerning the nature of the Pareto optimal hypersurface that is

discussed in section 9.5 is reinforced.

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.8: Force-Time Results with the Alternate Min-max Method and Normalization

9.7.1 Comparison of Preferences

First, the global criterion approach is used with a priori articulation of preferences

to incorporate and compare the two above-mentioned sets of preferences. In accordance

with the work in Chapters 7, the following global criterion, which is based on (7.2), is

minimized:

219

( )

1

6

1

ppnorm

i i

i

U w E

=

= ∑ (9.17)

As discussed in Chapter 6, the exponent in (9.17) generally indicates the amount of

emphasis that is placed on the objective function with the highest value. If the exponent

is one, then (9.17) reduces to the weighted sum method. Minimizing a weighted sum is

sufficient for Pareto optimality; theoretically, it always results in a Pareto optimal point.

With an exponent of infinity, (9.17) represents the weighted min-max method. Using the

weighted min-max method provides a necessary condition for Pareto optimally; it is able

to capture all Pareto optimal points with variation in the weights. We use 7p = , and this

provides a general, intermediate method that blends characteristics of the min-max

method and the weighted sum method, in terms of the global criterion contours in the

criterion space.

9.7.1.1 Error Results

(9.17) is minimized using different weighting vectors as general indications of

preference. The results for the error functions are shown in Table 9.4, where w indicates

the weighting vector. There are essentially two cases: one in which the accuracy of

forces 2, 3, 4, and 6 is emphasized, corresponding to the first four rows in Table 9.4, and

one in which the accuracy of forces 1, 2, and 3 is emphasized, corresponding to the last

three rows. In each case, the ratio of the highest weight value to the lowest weight value

increases. Computational time tends to increase as the difference between the weights

increases, and this will be emphasized in section 9.7.2.

220

w 1E

2E

3E

4E

5E

6E

(.1,.2,.2,.2,.1,.2) 6.48045E+02 2.34067E+04 6.63310E+03 5.66951E+04 1.64877E+03 6.81302E+03

(.01,.245,.245,.245,.01,.245) 6.69952E+02 2.34035E+04 6.62901E+03 5.66864E+04 1.69209E+03 6.80590E+03

(.0001,.24995,.24995,.24995,.0001,.24995) 5.36010E+03 2.34009E+04 6.63071E+03 5.66872E+04 1.62471E+04 6.80624E+03

(.00001,.249995,.249995,.249995,.00001,

.249995) 4.43607E+05 2.34115E+04 6.64229E+03 5.67246E+04 1.30340E+06 6.84349E+03

(.2222,.2222,.2222,.1111,.1111,.1111) 6.45499E+02 2.34058E+04 6.63151E+03 5.66981E+04 1.64730E+03 6.81595E+03

(.3332,.3332,.3332,.0001,.0001,.0001) 6.45345E+02 2.34137E+04 6.63803E+03 1.55842E+05 2.38773E+04 9.70560E+04

(.33332,.33332,.33332,.00001,.00001,.00001) 6.53350E+02 2.35118E+04 6.73639E+03 1.16585E+07 4.03418E+06 9.65751E+06

Table 9.4: Error Results with the Weighted Global Criterion Approach

As expected, error-function values become more distinguished as the difference

between the weights, increases. This is because the relative values of the weights, not the

absolute values, govern which Pareto optimal point is produced. As a weight is

decreased, the associated error increases, but there is no corresponding decrease

(improvement) in other error functions. This supports the idea of high trade-offs, which

is discussed in section 9.5 with respect to Figure 9.3. Despite increased error for

functions with lower weights, all of the solutions theoretically are Pareto optimal.

A key feature of these results is that even when some weights are twice as large as

others are, the results do not necessarily deviate significantly from those shown in Table

9.3. The fact that especially small weights are necessary in order to affect the final

solution suggests that the difference between the values of the components of a weighting

vector must be significant relative to the potential values of the objective functions. This

reinforces the findings in Chapter 4.

221

9.7.1.2 Force Results

Figure 9.9 shows the force results when (9.17) is minimized with

( )10.0001,0.24995,0.24995,0.24995,0.0001,0.24995=w . These results show only a

small degradation in the accuracy of the curves for forces 1 and 5, when compared to

Figures 9.4 and 9.5. Thus, the force results are relatively insensitive to alterations in

preference. However, using a weight as small as 510

− does reduce the significance of 1

E

and 5

E in the aggregated function to the point that unacceptable results for force 1 and

force 5 are achieved, even though the results are Pareto optimal. This is shown in Figure

9.10 where ( )20.00001,0.249995,0.249995,0.249995,0.00001,0.249995=w .

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.9: Force-Time Results with a Weighted Global Criterion using 1

w

222

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.10: Force-Time Results with a Weighted Global Criterion using 2

w

Despite the substantial degradation in the force-results associated with low

weights, there is no substantial improvement in other force approximations, as one might

expect. While Figures 9.9 and 9.10 reflect trends in the design space, this phenomenon

was seen in the criterion space as well, as discussed above with respect to Table 9.6.

Similar trends occur with the second set of preferences, which emphasizes forces 1, 2,

and 3. Thus, with this system identification problem, there is no distinguishable

advantage in specifying preferences with a weighted aggregate function. This reinforces

the idea illustrated in Figure 9.3 in section 9.5, where significant portions of the Pareto

optimal hypersurface involve high trade-offs between the different functions as one

moves along the hypersurface. Error functions associated with relatively high weights

are not reduced substantially enough to compensate for degradation in the error functions

with lower weights.

223

9.7.2 Comparison of Methods

In this section, different methods that involve a priori articulation of preferences

are studied. The same basic formulations are used here that are used with no articulation

of preferences. However, in this case, the objective functions are normalized, and

weights are incorporated by using

( )0.0001,0.24995,0.24995,0.24995,0.0001,0.24995=w . The weighted global criterion

is formulated as shown in (9.13), with 2p = , thus representing a weighted distance

function.

The error-function results for each weighted method are shown in Table 9.5,

which is comparable to Tables 9.2 and 9.3. For each method, when compared to the

results in Table 9.3, the results in Table 9.5 show an increase in the error for forces 1 and

5, as one would expect given the weighting vector. However, again, there is no

concurrent significant decrease in the error for the other functions.

1E

2E

3E

4E

5E

6E

Weighted Sum 7.78288E+02 2.33995E+04 6.62516E+03 5.66795E+04 7.68288E+03 6.79968E+03

Weighted Distance 1.48121E+03 2.33996E+04 6.62554E+03 5.66794E+04 7.28464E+03 6.79979E+03

Weighted Min-max 3.67636E+03 2.34067E+04 6.63261E+03 5.66881E+04 1.00553E+04 6.80752E+03

Alt. Weighted Min-max 5.57878E+03 2.34116E+04 6.63740E+03 5.66915E+04 1.53333E+04 6.81279E+03

Table 9.5: Error-Function Results with Weighted Methods

With this case, there was a significant increase in the computational demands for

all methods, suggesting that with this type of Pareto optimal set, the nature of the

weighting vector can have a significant affect on computational performance.

224

Nonetheless, the weighted sum method still requires the least number of iterations. As

was the case earlier, the min-max methods yield weakly Pareto optimal solutions.

Consequently, the weighted sum and the weighted distance function provide superior

results.

The force results for the different methods were indistinguishable, and the results

for the weighted sum are shown in Figure 9.11.

-2.0E+04

-1.0E+04

0.0E+00

1.0E+04

2.0E+04

3.0E+04

4.0E+04

0 20 40 60 80 100

Force 1

Force 2

Force 3

Force 4

Force 5

Force 6

Time Step

Fo

rce

(lb

f)

Figure 9.11: Force-Time Results with the Weighted Sum


In this chapter, we have demonstrated the advantages of viewing a system

identification problem as a MOO problem. We have provided further study of a system

identification problem used for the development of simplified dynamic crash model, and

we have demonstrated some of the findings concerning MOO methods that were

discussed in previous chapters. We have demonstrated the necessity for normalization,

especially with the min-max approach. However, we reveal and resolve computational

225

difficulties with standard normalization. We have identified and exploited a unique form

of Pareto optimal set, and we have used this form to identify the most advantageous

MOO method for this particular problem. Two different sets of preferences have been

incorporated using a global criterion with transformed objective functions, and the

relevance of preferences to the problem has been evaluated. In summary, we have shown

how prudent application of MOO methods and careful consideration of MOO theory can

yield significant insight that would otherwise go unnoted.

Even with more than three design variables and more than three objective

functions, when illustrating the Pareto optimal set directly is impossible, we have shown

that one can determine the nature of the Pareto optimal set through careful analysis. It

was found that the Pareto optimal hypersurface for this problem resembles the corner of a

hypercube, as illustrated in Figure 9.3. This means that many of the points on the

boundary of the feasible criterion space are weakly Pareto optimal or nearly weakly

Pareto optimal. At such points, one is able to improve at least one objective without any

significant detriment to any other objective. Thus, in this case, many of the points on the

Pareto optimal hypersurface are irrelevant. This kind of approach to MOO problems,

where careful theoretical analysis of the Pareto optimal set is used to extract practical

information beyond that of a single solution, is unseen in the current literature.

We discovered that there can be a trade-off in capabilities between the two min-

max methods, and this issue has not been discussed in the literature. The first method,

shown in (9.14), can be discontinuous and non-differentiable. However, in some

instances, it can be much less computationally demanding than the alternate min-max

method shown in (9.15), which is differentiable.

226

With objective functions like error functions, which all have the same physical

significance, one anticipates that the weighted sum may not provide the most useful

results. This is because when using an objective sum, although the total error is

minimized, it is possible that the error for one particular force could be relatively high. In

essence, the objective should be to avoid a highly inaccurate single subsystem (single

degree of freedom model) just as much as it is to minimize the error of the total system.

In such cases, the min-max method and the global criterion approach, in which one can

control the amount of emphasis placed on the function with the highest deviation from a

predetermined goal, seem to be most appropriate. However, we have shown that with

this problem, that is not the case.

In contrast to the min-max methods, the weighted sum method and the weighted

global criterion approach always yield Pareto optimal points. In fact, we recommend

them for problems with this type of Pareto optimal set when a priori articulation of

preferences is used, because there is no chance of obtaining a weakly Pareto optimal

solution. However, regardless of the method that is used, we have found that because of

the shape of the Pareto optimal set, there is little advantage in articulating different

preferences using weights. The superior solution is one that is closest to the utopia point

in some sense, and such a solution is provided by either the objective sum method or the

distance function method. Consequently, although methods with no articulation of

preferences theoretically provide an arbitrary Pareto optimal point (see section 3.8), we

have demonstrated that there are scenarios for which they are well suited.

Specific conclusions based on this work are:

227

1) Despite potential nonlinearities, in this case, the standard min-max method is more

efficient (fewer iterations and function calls) than the alternative min-max method.

2) Despite similar units and orders of magnitude, the error functions need to be

normalized, especially when the min-max approach is used. However, standard

normalization can result in premature convergence when the denominator large (i.e.

the Pareto-maxima and minima for the objective functions have significantly different

orders of magnitude and the Pareto-maxima is orders of magnitude larger than zero).

3) The value of a weight is significant relative to the other weights and relative to the

magnitude of its corresponding objective function.

4) Using weights with significantly different orders of magnitude can increase

computational demands.

5) Detrimental increases in the value of error functions that are multiplied by a relatively

small weight are not accompanied by beneficial decreases in the values of other error

functions. Thus, there is no substantial advantage in articulating preferences for

problems like this, which involve a Pareto optimal hypersurface with a specific form

(see Figure 9.3).

6) With this problem, the objective sum method efficiently yields a superior Pareto

optimal solution that is relatively close to the utopia point. In general, it requires

relatively few optimization iterations.

228

CHAPTER X

OPTIMIZATION-BASED HUMAN POSTURE PREDICTION

10.1 Introduction

As suggested in Chapter 9, up to this point, much of the development and analysis

has concerned theoretical and numerical aspects of different methods. This chapter

provides a second practical application for the aspects that are discussed in earlier

chapters. Whereas Chapter 9 focuses primarily on a priori articulation of preferences,

this chapter concerns, among other things, a posteriori articulation. We propose a multi-

objective optimization (MOO)-based approach to human posture prediction. Virtual

(computer-based) humans that act and look as real humans do, offer a means to evaluate

and test virtual prototypes without having to build an actual costly prototype. This can

save money and reduce the design cycle time for any product that eventually requires

human interaction. The posture prediction problem entails having an avatar contact a

specified target point with the end-effector, using a realistic, natural posture. An end-

effector is a point of interest (typically the end-point in a series of links) on a kinematic

system such as an arm. We consider only static posture, independent of time. The joint

angles for all degrees-of-freedom (DOFs) in the human model provide the design

variables and are determined by optimizing an objective function that represents a human

performance measure, such as discomfort or potential energy. In general, performance

measures are metrics that govern how and why a human model moves, given a particular

scenario. Different measures can result in slightly different postures.

229

However, currently, only a limited number of human performance measures are

available, and different components of these measures have not been viewed in terms of

MOO. In addition, when incorporating multiple performance measures simultaneously,

only the weighted sum method has been used, and analysis of the results in terms of

MOO is not provided. Various MOO methods and MOO theory in general has not be

used adequately to contribute to posture prediction. Consequently, an in-depth multi-

objective study can improve the posture prediction model and improve understanding of

how to predict human postures, determining which performance measure or which

combination of measures predict human postures most effectively. Use of different

MOO methods in this capacity can also provide a practical analysis of the methods.

Thus, the purpose of this chapter is to incorporate MOO into the posture

prediction problem in two ways: 1) the development of new human performance

measures, and 2) the combination of different performance measures. Concurrently, we

analyze the relative performance of the MOO methods in terms of computational

requirements and solution characteristics. This study draws on the work by Marler and

Yang (2004), and Yang et al (2004). It corresponds to objective 4 in section 1.4,

concerning significant applications for MOO methods.

As with Chapter 9, we elaborate on and further demonstrate the newly revealed

characteristics of the MOO methods that are discussed in previous chapters. Again, we

discuss the necessity for, and special circumstances concerning normalization. We also

demonstrate the advantages of using the Pareto-maximum, as suggested in Chapter 5.

We use the posture prediction model as an example that contrasts methods in terms of

their ability to capture non-convex portions of the Pareto optimal set. We also use the

230

posture prediction problem to demonstrate the advantages of the newly developed

modified weighted global criterion method presented in Chapters 6 and 7. We again

identify and exploit the characteristics of a special form of Pareto optimal set, a form that

surfaced in Chapter 9. We show how the ε -constraint method (discussed in Chapter 8)

performs poorly with this type of set, even in the case of a bi-criterion problem. In terms

of posture prediction, we demonstrate the advantages of incorporating MOO in posture

prediction, which has never been done before to any significant degree. We develop two

new human performance measures and highlight their advantages. Despite the presence

of infinitely many feasible spaces (one for every target point), we show how the Pareto

optimal set can be used as a tool for selecting weights that may be used to determine a

single solution regardless of the target point.


There is a variety of methods for predicting human postures and motion (Mi,

2004), and to discuss them all is beyond the scope of the present chapter. Rather, we

focus on the optimization-based approaches.

10.1.1.1 Motion Prediction

Although this chapter focuses on human posture prediction, much of the initial

development in this area stems from work with robotic and human motion prediction,

where motion essentially implies that a series of postures are predicted over time. Much

of the initial work with robotic motion prediction has been completed in the field of

controls with simple three-DOF robots (Kahn and Roth, 1971; Shin and Mckay, 1986).

These works seek to determine what torques needed to be applied to joint in order to

yield a specified motion (i.e., have the end-effector follow a specified parameterized

231

path) while minimizing operation time and/or energy loss. Typically, some type of path

descriptor is provided for the end- effector, where a path refers to Cartesian position-

history for a single point. Then, the first stage of the problem involves trajectory

planning, where the term trajectory refers to a time history, and this stage entails

determining the time histories for the joint angles and angular velocities of each robotic

joint. Then, trajectory tracking involves determining actuation that results in the motion

determined in the trajectory planning stage. In this chapter, we assume the term motion

prediction refers to trajectory planning, which is the focus of this work.

One of the earliest works with motion prediction outside the field of controls is

provided by Lin et al (1983). With this approach, first, a series of positions and

corresponding orientations are specified for an end-effector. Each set of specifications is

transformed to a set of joint angles, and this yields a series of points in joint-angle space

that represents a discretized path. Although not stated explicitly, the transformation is

presumably completed using the Denavit-Hartenberg method (DH-method) (Denavit and

Hartenberg, 1955). Time histories for the joint angles in a seven-DOF robot are

represented with splined cubic polynomials, and each polynomial is made to fit the series

of joint-angle values determined above. Thus, the joint angles are essentially constrained

by the above-mentioned specified points. The polynomials are written in terms of

angular acceleration and in terms of time intervals. The angular acceleration of a joint is

determined by solving a system of linear equations based on stipulations for continuity in

the time histories of the joint angular velocity and acceleration. Note that this process of

solving a system of equations can be cumbersome for larger models with a relatively high

number of DOFs. The time intervals provide the design variables for the optimization

232

problem. The sum of the time intervals for each spline segment is minimized, thus

maximizing the speed of operation for the robot. Limits (constraints) are placed on the

angular velocity, acceleration, and jerk for each joint. The problem is solved using

Nelder’s and Mead’s flexible polyhedron search. Saramago and Steffen (1998) use the

same approach and apply it to a three-DOF and a six-DOF robotic arm. However,

mechanical and potential energy are determined, and generalized forces are determined

based on Lagrangian mechanics. In addition, mechanical energy is also incorporated in

the objective function.

Drawing on the work of Gilbert and Johnson (1985), Chen (1991) outlines the

first use of B-splines for general robotic motion prediction and includes obstacle

avoidance capabilities. Details concerning the use of B-splines in this capacity are

discussed. Compared to using splined cubic polynomial, one of the advantages of using

B-splines is inherent continuity. A general objective function is presented in terms of

time, joint angles, angular velocity, and a general vector of control parameters, such as

applied joint torques. The joint angles are represented by B-splines, and the B-spline

coefficients provide the design variables for the optimization problem. The approach is

tested on basic two and three-DOF robots.

Saramago and Steffen (2000) expand on this approach. They also use the DH-

method for kinematic analysis. Only the initial and final values for the joint angles are

provided as input; no intermediate points are stipulated. Thus, no path is specified for the

end-effector. The objective function is a function of the total traveling time for the robot

and of generalized forces at the joints. The generalized forces are functions of angular

velocity and angular acceleration, which can be written as functions of the B-spline

233

coefficients. These coefficients provide design variables for the optimization problem

along with the total traveling time for the robot. The joint angles are determined from the

B-spline coefficients using a system of linear equations that incorporates user-specified

initial and final angular velocities. The coefficients are then used to determine angular

velocities and accelerations, which are used in the objective function and in the

constraints. A penalty term is also incorporated in the objective function, in order to

model collision avoidance and ensure that the distance between the end-effector and an

object is adequate. A similar approach is used for obstacle avoidance by Riffard and

Chedmail (1996). Saramago and Ceccarelli (2002, 2004) also use B-splines in this way,

and consider externally applied loads. Again, the method is tested on a six-DOF robot.

Alternatively, Zhang et al (1998) essentially solve a system identification problem

to determine the time histories for the joint angles in a seven-DOF robot. Parameters in

an approximating function for a joint angle are determined by minimizing the error

between the approximating function and experimental data.

Lo et al (2002) present one of the first works with optimization-based human

motion prediction. As with previous work, only the initial and final values for the joint

angles are provided as input. Joint angles, angular velocity, and angular acceleration are

written in terms of B-splines, and the B-spline coefficients provide the design variables.

Torque is determined using recursive dynamics. The objective function is the integral of

the Euclidean norm of joint torque over time. In addition to the specified initial and final

configurations, joint angles and joint torques are subject to limits. The calculated motion

is animated using Jack (Badler et al, 1993). This approach is applied to a ten-DOF

human model.

234

Abdel-Malek et al (2005) and Mi (2004) also use B-splines and the DH-method

with human motion prediction, but take a slightly different approach from that which is

describe above. The approach is used for a 15-DOF human torso model. The above-

mentioned system of linear equations is not used. Rather, the objective function and the

constraints are functions of joint angles only, which are directly related to the B-spline

coefficients, which in turn provide the optimization design variables. Discomfort is used

as the objective function and is determined as the sum of the change in the joint angles,

relative to a predetermined neutral position. The end-effector is constrained to remain on

a parametric path that is determined by minimizing the jerk of the end-effector during the

motion (Flash and Hogan, 1985). Note that although using minimum jerk to determine

the path for a human end-effector is common, Alexander (1997) uses metabolic energy

and suggests that most currently available models for human paths, including the

minimum jerk model, have significant deficiencies. Because this application concerns

human motion, additional mathematical terms are included in the objective function to

refine the motion, making it more realistic. These terms are necessary to ensure

consistency in the joint angle profiles, by reducing the amount of reversal in the sign of

the angular velocity. In addition, they enforce smoothness by reducing the magnitude of

the angular acceleration. Finally, they reduce the magnitudes of the initial and final

angular velocities. The simulation takes approximate 17 seconds on a 1.8 GHz Pentium4

CPU with 512M RAM, which constitutes near real-time operation.

235

10.1.1.2 Posture Prediction

The literature concerning optimization-based posture prediction is relatively

limited, since robots are not considered with this topic. However, one exception to this

trend is provided by Mi et al (2002a) who determine an optimum starting configuration

(posture) for a three-DOF robot, given a specified path and given the requirement that the

trajectory must be smooth. Multiple potential starting configurations are determined by

numerically solving a system of differential-algebraic equations. The ideal configuration

is determined by minimizing joint displacement, which is essentially the difference

between current joint angles and angles that constitute a predetermined neutral position.

One of the earliest and most extensive works with human posture prediction is

provided by Riffard and Chedmail (1996). The DH-method is used to model a seven-

DOF upper torso of a human. The optimum placement of the torso and the optimum

posture of an arm are determined such that the hand reaches a given target, there are no

collisions between the operator and objects in the environment, and the joint angles are

within specified limits. In addition, coupling between particular joint angles and variable

joint limits is modeled. This can be especially crucial with the shoulder joint. The

approach satisfies the requirement that the avatar sees the target point and the

requirement that the avatar is properly oriented relative to the target point. Finally, the

torques resulting from an externally applied load, are considered. Equations for each of

the above-mentioned components (target contact, collision, vision, orientation, and

torque) are essentially combined in a weighted sum, and the weights are used as scaling

factors for the different components. The final unconstrained problem is solved using

simulated annealing, which is a global optimization technique. Consequently, the

236

solution process is relatively slow (approximately two to six minutes on an Indigo II

SGI).

Yu (2001) uses the same fundamental approach but with joint displacement and

potential energy as objective functions for a three-DOF arm. The problem is solved

using a genetic algorithm, which is also a relatively slow global optimization technique.

Mi et al (2002b) and Mi (2004) extend the work of Yu (2001) to a 15-DOF model

with similar objective functions. A real-time optimization algorithm is developed that

combines genetic-algorithm results from a library of off-line computations, with results

from an unconstrained gradient-based BFGS algorithm. In addition, the results are

validated with the results from IKAN (Tolani et al, 2000) using a seven-DOF torso.

Results are also validated using HUMOSIM (Human Motion Simulation ate the Center

for Ergonomics of University of Michigan).

Although a significant amount of work has been completed concerning the

concept of optimization-based posture prediction, this approach depends heavily on the

objective function, and relatively little work has been conducted which implements

multiple and varied human performance measures. Thus, we investigate the development

and improvement of these objective functions. In addition, we provide a thorough study

of how to combine some of these functions by using MOO.

10.1.1.3 Multi-objective Optimization

Despite the current work with optimization-based motion prediction and posture

prediction, MOO has not been exploited thoroughly. In fact, use of MOO with human

modeling in general is minimal. It involves only the weighted sum method with no

indication of weight values, no discussion of normalization even with objective functions

237

that have significantly different orders of magnitude and physical significance, and no

analysis of the results in terms of MOO theory.

With respect to robotic motion prediction, Saramago and Steffen (1998)

essentially use the weighted sum method to minimize the travel time for a robot and the

mechanical energy of robotic actuators. Saramago and Steffen (2000) use the weighted

sum method to minimize travel time, mechanical energy, and a penalty parameter for

collision avoidance, but the values for the weights are not indicated. Saramago and

Ceccarelli (2004) use the weighted sum method to combine mechanical power and

gripper action energy, but only three sets of weights are used and the results are not

evaluated in the context of MOO.

With respect to human posture prediction, Riffard and Chedmail (1996) string

together a variety of criteria (discussed above) in a weighted sum. The weights are used

as scaling factors to reduce the difference in magnitude of the different criteria. As

discussed in Chapter 5, this practice does not necessarily constitute a reasonable approach

to normalization. Mi at al (2002b) mention MOO in the development of an optimization

algorithm. However, although many objective functions are described, only one is used

in the final formulation. Using a weighted sum, the objective function is combined with

the primary constraint, not with another objective. Thus, conceptually, the result is an

unconstrained optimization problem representing a penalty function approach to

constrained optimization. Mi (2004) briefly mentions the use of MOO to combine

discomfort, effort, and potential energy as human performance measures. However, there

is no indication as to which MOO method is used.

238

Marler and Yang (2004), and Yang et al (2004) present the first thorough studies

of MOO for use with human posture prediction, and this chapter is an extension of these

works.


The objectives for this study are stated as follows:

1) Use MOO to develop new human performance measures pertaining to discomfort and

potential energy.

2) Use MOO to combine different human performance measures and to provide insight

into the posture prediction problem. Determine to what extent delta-potential-energy

should be integrated with discomfort.

3) Determine the form of the Pareto optimal set and study the trade-offs between the two

performance measures.

4) Analyze the relative performance of MOO methods in terms of computational

requirements and solution characteristics.

5) Test the modified weighted global criterion method in its ability to approximate the

results of the min-max method and in its ability to depict the Pareto optimal set.

10.2 Overview of the Human Model

In this section, we describe the human model that provides the foundation for this

study. Details concerning the development of this model are provided by Farrell and

Marler (2004).

We depict the human skeleton as a kinematic system, a series of links connected

by revolute joints that represent musculoskeletal joints. In general, our approach to

posture prediction entails finding the rotational displacement for these joints necessary to

239

optimize one or more objective functions that represent human-performance measures,

which govern how and why a human moves in a particular way, given a particular

scenario.

In order to provide gross motion, a kinematic model for the upper body is

developed, constituting the torso, spine, shoulders, and arms. iq represents the rotation

of each joint, as shown for the general series of links in Figure 10.1.

...

Target Point

End-effector

Global Coordinate

System

Local Coordinate

System

1q

2q

nq

( )x q

X

Z

Y

Figure 10.1: General Kinematic Model

Each joint angle is associated with a local coordinate system. n

∈q R is the vector of n

joint angles iq in an n-DOF model and represents a specific posture. ( ) 3

R∈x q is the

position vector in Cartesian space that describes the location of the end-effector as a

function of the joint angles, with respect to the global coordinate system. In this case, the

end-effector is the tip of the index finger, and we are concerned with finding the values of

the joint angles when the position of the end-effector is constrained with respect to the

240

global coordinate system. Typically, the end-effector is required to contact a pre-

determined target point, as shown in Figure 10.1. The set of all target points that the end-

effector can contact, given a fixed global coordinate system, is called the reach envelope.

With this study, a 21-DOF model for the human torso and right arm is used, as

shown in Figure 10.2.

Clavicle

Shoulder

Spine

Elbow Wrist

X

Z

Y Global

Reference

Frame

z21

x21

y21

End-effector

Figure 10.2: 21-DOF Kinematic Model

1q through

12q represent the spine.

13q through

17q represent the shoulder and clavicle.

18q through

21q represent the right arm.

Although this study focuses on upper-body posture prediction, Figure 10.2

represents just one part of a complete virtual human model called SantosTM

(Abdel-Malek

et al, 2004; Yang et al, 2005), which is illustrated in Figure 10.3.

241

Figure 10.3: SantosTM

10.2.1 Denavit-Hartenberg Method

The position of the end-effector in Cartesian space ( ) 3R∈x q , for a given set of

joint angles n

∈q R , is determined using the DH-method. This method involves a matrix

notation and approach for relating the position of a point in one coordinate system to

another coordinate system, by using a unique transformation matrix. A local coordinate

system and a local transformation matrix are associated with each joint, describing its

configuration with respect to the previous joint and previous coordinate system. Multiple

transformation matrices can be combined to determine the position of any point on the

kinematic system with respect to any local coordinate system or with respect to a global

coordinate system, based on all of the joint angles.

Global

Reference Frame

Z

X

Y

Global

Reference Frame

Z

X

Y

Z

X

Y

242

The first step with the DH-method is to embed reference frames (coordinate

systems) at each DOF such that the local z-axis of each frame represents the axis of

motion (rotation in our case) for the corresponding DOF. The z-axis for each reference

frame, corresponding to a single degree of freedom, is shown in Figure 10.2. The x-axis

of each frame must be perpendicular to both the z-axis of the current frame and the z-axis

of the previous frame. Then, a transformation matrix 1i

i

−

T relates position and

orientation in the ith

frame to the ( )1i −th

frame. This matrix is expressed in terms of the

angle i

θ measured from the -axisix to

1-axis

ix

−

about the 1-axis

iz

−

, the distance i

d

from the -axisix to

1-axis

ix

−

along the 1-axis

iz

−

, the angle i

α measured from the

-axisiz to

1-axis

iz

−

about the -axisix , and the distance

ia from the -axis

iz to

1-axis

iz

−

along the -axisix . These four parameters are shown in Figure 10.4 (based on a similar

figure from Mi (2004)), where 1i

q+

is the displacement (rotation) within frame-i. Note

that the values for these parameters can depend on the joint angles.

The independent transformation matrices can be multiplied to form a cumulative

transformation matrix, which describes the position of a point in any reference frame

with respect to any other reference frame. For instance, the cumulative transformation

matrix 2 2 3 4

5 3 4 5=T T T T translates coordinates of a point that are given in terms of

reference frame-5, into coordinates in terms of reference frame–2.

243

Joint i

Joint i + 1

di

ai

α i

Link i

z i-1

xi-1

θ i

x i

zi

qi

q i +1

Figure 10.4: DH-Parameters

Using this idea, the global position-vector ( )x q for the end-effector of the human model

is given as follows:

( ) 0

n n=x q T x (10.1)

0 1

1

n

i

n i

i

−

=

= ∏T T (10.2)

1

cos cos sin sin sin cos

sin cos cos sin cos sin

0 sin cos

0 0 0 1

i i i i i i i

i i i i i i ii

i

i i i

a

a

d

θ α θ α θ θ

θ α θ α θ θ

α α

−

− − =

T (10.3)

where n

x is the position of the end-effector with respect to the nth

frame. n

x is actually

an augmented 4 1× vector written as [ ]1T

n n n nx y z=x . Note that the 0

th frame is

the global reference frame (global coordinate system). In addition, although there is an

244

nth

frame, shown as 21

x , 21

y , and 21

z in Figure 10.2, there is no joint angle associate with

the nth

frame. That is, there is no 22

q (recall that i

q is the joint angle from the xi−1

-axis

to the xi-axis).

The transformation matrix can be decomposed as follows:

3 3 3 1

1 30

× ×

×

=

R pT

0 (10.4)

As suggested earlier, this matrix transforms the coordinates of a point in reference frame-

i to coordinates in terms of frame- ( )1i − . The matrix R is responsible for the rotation of

frame-i with respect to frame- ( )1i − , and the vector p represents the translation of frame-

i with respect to frame- ( )1i − , as shown in Figure 10.5.

X

Z

Y

x

z

y

p

Reference

Frame-i

Reference

Frame-(i-1)

v

r

vz

vx vy

Figure 10.5: Reference Frame Transformation

245

One can relate the three vectors in Figure 10.5 as follows:

= +r p Rv (10.5)

Based on (10.3), R is written as follows:

cos cos sin sin sin

sin cos cos sin cos

0 sin cos

i i i i i

i i i i i

i i

θ α θ α θ

θ α θ α θ

α α

− = −

R (10.6)

The rows of R act to resolve each component of v into a specific component in frame-

( )1i − . For instance, the component ( )1,1R resolves vx into the X-direction (determines

the X-component of vx), ( )1,2R resolves vy into the X-direction, and ( )1,3R resolves vz

into the X-direction. The columns of R represent the direction of the axes for frame-i, in

terms of frame- ( )1i − . For instance, the x-axis of frame-i points in the direction

[ ]cos sin 0T

i iθ θ with respect to frame- ( )1i − .

10.3 Development of Human Performance Measures

The objective functions in the final optimization formulation represent human

performance measures. In this section, we present these performance measures, which

are incorporated in the final MOO problem in the following section. First, we describe

joint displacement, which is based on work by Jung et al (1994) and Yu (2001).

Although it is not used to obtain computational MOO results, it provides a basis for

comparison with proposed performance measures. We then present two newly developed

performance measures, which are studied in subsequent sections: delta-potential-energy,

and discomfort.

Literature suggests that each degree of freedom or each body segment should be

associated with an individual objective function or component that may be considered in

246

the final performance measure (Cruse, 1986; Zacher and Bubb, 2004). Following this

idea, we view the development of human performance measures in the context of MOO.

Smaller components of the performance measure are combined using different MOO

methods.

10.3.1 Joint Displacement

Joint displacement is determined based on the weighted sum method for MOO.

N

iq is the neutral position of a joint measured from the home configuration, which is

characterized by =q 0 . The neutral position N

q represents a relatively comfortable

position. Then, conceptually, the displacement from the neutral position for a particular

joint is given by N

i iq q− . However, to avoid numerical difficulties and non-

differentiability, the terms ( )2

N

i iq q− are used. Each of these 21 terms (one for each

degree of freedom) serves as an individual objective function, a component of a MOO

problem. The terms are combined using a weighted sum. The scalar weights wi are used

to stress the importance of particular joints. The consequent joint displacement function

is given as follows:

( ) ( )

1

2DOF

Jo int displacement i

i

N

i if w q q

=

= −∑q (10.7)

We have determined the values for the weights based on trial-and-error

experimentation with the 21-DOF model, and they are given in Table 10.1. Note that for

a single degree of freedom, because motion in different directions can entail different

degrees of difficulty, it is necessary to use discontinuous weighting values. This can

result in numerical difficulties.

247

Joint Variables Joint Weight

1 4 7 10, , ,q q q q 100

2 5 8 11, , ,q q q q

100 when 0N

i iq q− >

1000 when 0N

i iq q− <

3 6 9 12, , ,q q q q 5

13q 75

14 15 16, ,q q q 1

17q

50 when 0N

i iq q− >

1 when 0N

i iq q− <

18 19 20 21, , ,q q q q 1

Table 10.1: Joint Weights for Joint-

Displacement

For this model, the neutral position is chosen based on observation of the skinned

model in Figure 10.3 rather than a skeletal model like the one shown in Figure 10.2. The

resulting vector N

q is defined as

0; 1,...,12,19,20N

iq i= = (10.8)

13 14 15 16 17 18 21

15.0, 20.0, 100.0, 10.0, 80.0, 35.0, 15.0N N N N N N N

q q q q q q q=- = = =- =- =- =

This generally represents a posture with the arms straight down, parallel to the torso. The

avatar’s position gravitates towards the neutral position.

10.3.2 Delta-Potential-Energy1

In this section, we discuss the development of a new potential-energy human

performance measure, which is indirectly based on the weighted sum method for MOO.

Potential energy is a basic well-understood concept, and as suggested earlier, it has been

1 Dr. Jingzhou Yang provided much of the initial Fortran code for the concepts that are developed in this

section.

248

used successfully as an objective with basic robots. However, implementing it as a

human performance measure requires special considerations. This proposed performance

measure stems from difficulties with the above-mentioned joint displacement function

and from deficiencies in an existing performance measure that depends on the potential

energy of an arm (Abdel-Malek et al, 2001; Mi et al, 2002b; Mi, 2004).

With joint displacement, the weights are set based on intuition and

experimentation, and although the postures obtained by minimizing joint displacement

are acceptable, the question arises as to whether or not there are more practical, less ad

hoc approaches to setting the weights. The idea of potential energy provides one such

alternative. With potential energy, the weights are essentially based on the mass of

different segments of the body, and in a sense, an individual objective function is

developed for each segment.

Whereas the previous potential-energy function only incorporates the potential

energy of an arm, we consider the complete upper body. We represent the primary

segments of the upper body with six lumped masses: three for the lower, middle, and

upper torso, respectively; one for the upper arm; one for the forearm; and one for the

hand. We then determine the potential energy for each mass. The actual masses for the

segments are determined based on data from Chaffin and Anderson (1984). The heights

of the masses, rather than the joint displacements, provide the components of the human

performance measure. Mathematically, the weight (force of gravity) of a segment of the

upper body provides a multiplier for movement of the segment in the vertical direction.

The height of each segment is a function of the joint angles, so, in a sense, the weights of

249

the lumped masses replace the scalar multipliers, i

w , which are used in the joint

displacement function.

If potential energy is used directly, as is the case with the previous potential-

energy function, there is always a tendency for the avatar simply to bend over, thus

reducing potential energy. All of the lumped masses gravitate towards the same vertical

base line or datum where the potential is considered zero. Consequently, we introduce

the idea of minimizing the change in potential energy. It is calculated as follows. Each

segment in the human model has a specified center of mass as depicted in Figure 10.6.

X

Z

Y

Local

Coordinate

System

' 0 '

i i iP = Tr

0

i i iP = Tr

Segment

Center of

Gravity

ih∆

ir

Global

Coordinate

System

Figure 10.6: Potential Energy of a Body Segment

The vector from the origin of a link’s local coordinate system to its center of mass is

given by ir , where the subscript indicates the relevant local coordinate system. In order

to determine the position any part of the body, we use the DH transformation matrices

250

( 1)i

i

−

T . Note that ir is actually an augmented 4 1× vector with respect to local coordinate

system-i, rather than a 3 1× vector typically used with Cartesian space, as discussed in

section 10.2.1. [ ]0 0 0T

g= −g is the augmented gravity vector. When the avatar

moves from one configuration to another, '

iP represents the potential energy of the initial

configuration, and iP represents the potential energy of the current configuration. The

potential energy terms for the ith

body part are ' 0 ' 1 '

1i

T i

i i iP m

−

= g rT T� and

0 1

1i

T i

i i iP m

−

= g rT T� . In Figure 10.6, ih∆ is the y-component of the vector

0 ' 1 ' 0 1

1 1

i i

i i i i

− −

−r rT T T T� � . The final performance measure, which is minimized, is

defined as follows:

( ) ( )2

'

1

Delta potential energy i i

i

f P P

κ

− −

=

= −∑q (10.9)

Note that (10.9) can be written in the form of a weighted sum as follows:

( ) ( ) ( )2 2

1

Delta potential energy i i

i

f m g hκ

− −

=

= ∆∑q (10.10)

where ( )2

im g represent the weights and ( )

2

ih∆ act as the individual objective functions.

κ is the number of lumped masses. Potential energy is a relative quantity, and the term

ih∆ usually refers to the vertical distance between a mass and a plane of zero potential,

where the plane is the same for all masses. However, in this case, ih∆ is measured

relative to the neutral position for a particular mass. Essentially, each mass has a

different datum or plane of zero potential. In this case, the initial position is the neutral

position described in relation to joint displacement. Thus, with this performance

251

measure, the avatar again gravitates towards the neutral position, as was the case with

joint displacement. However, horizontal motion of the lumped masses has no affect.

10.3.3 Discomfort

The idea of modeling discomfort can be somewhat ambiguous; it is a subjective

quantity, the evaluation of which may vary from person to person. However, it is

possible to incorporate distinct factors that contribute to discomfort, though the actual

absolute value for discomfort may not be significant. In this section, we present a new

human performance measure for musculoskeletal discomfort that incorporates three such

factors: 1) the tendency to move different segments of the body sequentially, 2) the

tendency to gravitate to a reasonably comfortable neutral position, and 3) the discomfort

associated with moving while joints are near their respective limits. As suggested earlier,

we are concerned only with static discomfort, corresponding to an instantaneous posture.

In addition, we do not consider environmental of psychological factors; the emphasis is

on musculoskeletal discomfort.

Some experimental work has been completed with discomfort in an effort to

determine what factors contribute to discomfort, and the above-mentioned ingredients

surface in these studies. However, these components have not yet been collectively

incorporated in an effective optimization-based human performance measure for use with

virtual humans. Shen and Galer (1993) conduct a study with seating discomfort. As with

many studies, they determine that discomfort is multifaceted, depending on different

components, including the position of the body and the relative position of the body

joints with respect to the environment (i.e. a seat), the ability to alter the position of the

joints over time, duration of a fixed posture, and applied pressure.

252

Zhang (1996) also considers seating and suggests that comfort and discomfort

should be treated as different but complementary quantities. It is found that discomfort

tends to be associated with biomechanical factors, whereas as comfort is associated with

“feelings of relaxation and well-being.” In the context of seating comfort, Shen and

Vertiz (1997) refine the general concept of comfort, again contrasting it with discomfort.

It is suggested that comfort is a temporal quantity. However, as we are concerned only

with static posture prediction, we are concerned only with static, instantaneous comfort.

Da Silva (2002) provides a thorough review of more concrete factors that effect comfort,

all of which concern the environment (thermal conditions, air quality, noise, etc.) in a

vehicle.

Allread et al (1998) perform experiments to evaluate discomfort in manufacturing

environments. The authors find that overall total-body discomfort depends on external

loads and on the nature of tasks that are being completed (i.e. lifting and moving objects),

whereas kinematics of the torso area relates to discomfort in specific body parts.

Discomfort is often highest in the lumbar region and in the shoulder, implying that

different segments of the body should be considered separately. In addition, the authors

find that discomfort in different areas of the body depends on different planes of motion.

Motion in the lateral plane is associated with lower back discomfort; motion in the

sagittal plane is associated with middle back discomfort; and motion in the transverse

plane is associated with discomfort across the shoulder, middle back, and lower back.

Santos et al (2000) conduct experiments to correlate subjective indications of

discomfort with biomedical indices that are evaluated using a 54-DOF motion-capture

model. They find that discomfort increases as the distance between the human and the

253

intended target point increases. All of the targets that are used require movement in the

torso. It is found that reaching out from the body (as opposed to across the body) results

in the highest discomfort. In addition, the authors find a linear relationship between

discomfort and two biomechanical indices: 1) moments in the muscles that are necessary

to balance the effect of gravity, and 2) deviation from a neutral position.

Zacher and Bubb (2004) draw similar conclusions with respect to a proposed

force-based discomfort model. The authors find that discomfort is proportional to how

close a joint angle is to its limit. In addition, discomfort depends on the magnitude and

direction of forces at the joints. Finally, the authors suggest that overall discomfort is

highly dependent on the maximum discomfort for a single body part. Chung et al. (2002)

also distinguish between the discomfort at each joint and the total-body discomfort, and

they use a neural network to connect the two.

In addition to the above-mentioned experimentation, some work has been

completed with the development of discomfort-based human models, but the

representation of discomfort in this capacity and the models themselves are limited. One

of the first works that involves a mathematical metric for discomfort is provided by Jung

et al (1994). The authors draw on the work of Liegeois (1977), who uses inverse

kinematics for motion prediction of a six-DOF robot. Jung et al (1994) use a normalized

form of (10.7) with each component of the neutral position determined as the center angle

for each joint. They refer to this as discomfort (as opposed to joint displacement), and it

is based on the idea that discomfort for the arm reaches a minimum approximately when

each joint is at its center angle (Cruse et al, 1990). The weights are determined based on

subjective experimentation with humans. They are 5.3, 3.5, 1.3, and 4.2 for the hip,

254

shoulder, elbow, and wrist joints respectively. This form of discomfort is used with a

two-dimensional, four-DOF human model based on inverse kinematics. Jung and Choe

(1996) extend the work of Jung et al (1994) to a three-dimensional, seven-DOF human

model, with externally applied forces. Again, a regression model is used to create the

discomfort function. Yu (2001), Mi et al (2002a, 2002b), and Mi (2004) use a similar

concept for discomfort, although the term joint displacement is also used. In this case, as

suggested in section 10.3.1, the weights are set based on visual experimentation.

We propose a new discomfort function that does not require experimental data,

and we contend that discomfort is not the same as simple joint displacement, despite the

discrepancy in the literature. Our discomfort function is based on three biomechanical

concepts hypothesized to govern human posture. Consequently, we anticipate that this

function will provide a tool with which one can study how and why humans move in a

particular way. Here, we provide a conceptual foundation with which joint torque will be

incorporated eventually.

In order to incorporate the first factor (the tendency to move different segments of

the body sequentially), discomfort is based on the lexicographic method for MOO, which

is discussed in detail by Marler and Arora (2003). With this method, a priori articulation

of preferences is used as it is with the weighted sum, but preferences are articulated in a

slightly different format. Rather than assigning weights that indicate relative importance,

one simply prioritizes the objectives. Then, one objective at a time is minimized in a

sequence of separate optimization problems. After an objective has been minimized, it is

incorporated as a constraint in the subsequent problems. The solution is Pareto optimal if

it is unique.

255

Using the concept behind the lexicographic method, one is able to model the idea

that groups of joints are utilized sequentially. That is, in an effort to reach a particular

target point, one first uses one’s arm. Then, only if necessary, does one bends the torso.

Finally, if the target is still out of reach, one may extend the clavicle joint. The

lexicographic method for MOO is designed to incorporate this type of preference

structure. However, solving a sequence of optimization problems can be time consuming

and impractical for real-time applications. Although there is no currently available proof

or example demonstrating the idea, it has been suggested that the weighted sum method

can be used to approximate results of the lexicographic method if the weights have

infinitely different orders of magnitude (Miettinen, 1999; Romero, 2000). Using posture

prediction as an example, we substantiate this idea.

The weights used to approximate the lexicographic approach are shown in Table

10.2.

Joint Variables Joint Weight

1 12, ,q q… 4

1 10×

13 14,q q 8

1 10×

15 21, ,q q… 1

Table 10.2: Joint Weights for

Discomfort

Although weights are used here, they do not need to be determined as indicators of the

relative significance of their respective joints; they are simply fixed mathematical

parameters. In addition, the exact values of the weights are irrelevant; they simply have

256

to have significantly different orders of magnitude. As suggested earlier, some of the

weights in Table 10.1 (used with joint displacement) are discontinuous. These

discontinuities can lead to computational difficulties, but with this discomfort objective,

such discontinuities are avoided.

The weights in Table 10.2 are used in a function that is based on (10.7) with the

neutral position defined as shown in (10.8). In this way, the second factor of discomfort

(the tendency to gravitate to a reasonably comfortable neutral position) is incorporated.

Prior to applying the weights, each term in (10.7) is normalized as follows:

N

norm i i

U L

i i

q qq

q q

−∆ =

− (10.11)

With this normalization scheme, each term ( )2

norm

iq∆ acts as an individual objective

function and has values between zero and one. As suggested in Chapter 5, this can

improve performance and allow for the use of weights with more significance.

Generally, this approach works well but often results in postures with joints

extended to their limits, and such postures can be uncomfortable. To rectify this problem

and to incorporate the final factor of discomfort (the discomfort associated with moving

while joints are near their respective limits), specially designed penalty terms are added

to the discomfort function such that discomfort increases significantly as joint values

approach their limits. The final discomfort function is given as follows:

( ) ( )1

1DOF

norm

Discomfort i i i i

i

f q G QU G QLG

γ

=

= ∆ + × + × ∑q (10.12)

( )

100

5.00.5 1.571 1

U

i i

i U L

i i

q qQU sin

q q

− = + +

−

(10.13)

257

( )

100

5.00.5 1.571 1

L

i i

i U L

i i

q qQL sin

q q

− = + +

−

(10.14)

where G QU× is a penalty term associated with joint values that approach their upper

limits, and G QL× is a penalty term associated with joint values that approach their

lower limits. i

γ are the weights defined in Table 10.2. Each term varies between zero

and G, as ( ) ( )U U L

i i i iq q q q− − and ( ) ( )L U L

i i i iq q q q− − vary between zero and one.

Figure 10.7 illustrates the curve for the following function, which represents the basic

structure of the penalty terms:

( )( )100

0.5 5.0 1.571 1Q sin r= + + (10.15)

0.2 0.4 0.6 0.8 1.0

0.2

0.4

0.6

0.8

1.0

Q

r

Figure 10.7: Graph of Discomfort Joint-Limit Penalty Term

r represents either ( ) ( )U U L

i i i iq q q q− − or ( ) ( )L U L

i i i iq q q q− − . Thus, as Figure 10.7

illustrates, the penalty term has a value of zero until the joint value reaches the upper or

258

lower 10% of its range, where either ( ) ( ) 0.1U U L

i i i iq q q q− − ≤ or

( ) ( ) 0.1L U L

i i i iq q q q− − ≤ . The curve for the penalty term is differentiable, and reaches

its maximum of 610G = when 0r = . The final function in (10.15) is multiplied by 1 G

for the sake of presentation to the user, so discomfort does not have extremely high

values when compared to other performance measures.

10.4 Multi-objective Problem Statement

In this section, we formulate the posture prediction optimization problem using

the above-described human performance measures along with the model that is presented

in section 10.2. Then, various MOO methods are used to combine the performance

measures that serve as multiple objective functions in the final optimization formulation.

10.4.1 Posture Prediction Formulation

As suggested earlier, the design variables for the final MOO problem are iq ,

which indicate joint angles in units of degrees. The vector q represents the consequent

posture. Because listing values for all of the joint angles with each predicted posture can

be cumbersome and unrevealing, in this case, results in the design space are depicted with

actual pictures of the avatar.

The first constraint, called the distance constraint2, requires the end-effector to

contact the target point. In addition, each generalized coordinate is constrained to lie

within predetermined limits. U

iq represents the upper limit for

iq , and L

iq represents the

lower limit. These limits ensure that the virtual human does not assume an unrealistic

posture given the nature of actual human joints.

2 Dr. HyungJoo Kim provided the initial approach for programming the gradients for the distance-

constraint.

259

The optimum posture for the 21-DOF system shown in Figure 10.2 is determined

by solving the following MOO problem:

Find: DOFR∈q (10.16)

to minimize: ( )1f Discomfort=q

( )2f Delta-potential-energy=q

subject to: ( )end-effector target point

distance ε= − ≤x q x

; 1, 2, ,L U

i i iq q q i DOF≤ ≤ = …

where ε is a small positive number that approximates zero. [ ]1 2,

T

f f=f is a vector of

objective functions (human performance measures). All optimization problems are

solved using the software SNOPT (Gill et al, 2002), which uses sequential quadratic

programming (Arora, 2004). Analytical gradients are determined for all objective

functions and for all constraints.

10.4.2 Multi-objective Optimization Formulations

The multi-objective methods that are discussed in Chapters 4 through 8 are used

to solve the problem described above, and the basic formulations for these methods are

restated in this section in terms of the posture prediction problem. To begin with, the

weighted sum method (Chapters 4 and 5) is used, which entails minimizing the following

aggregate function:

( )2

1

i i

i

U w f=

=∑ q (10.17)

where i

w is a scalar weight.

A basic global criterion is shown as follows:

260

( )( )

1

2

1

p po

i i i

i

U w f f=

= − ∑ q (10.18)

where o

if is a component of the utopia point (the minimum of a single objective) and p is

a positive scalar. As explained in Chapter 9, the function in (10.18), with an exponent of

2, is also called a distance function, because it minimizes the Euclidean distance between

the solution point and the utopia point, in the criterion space. In addition to the weighted

distance function, we test the modified weighted global criterion presented in Chapters 6

and 7.

The alternate min-max formulation of Chapters 6 and 7, is also used and is

formulated with an additional design variable λ and additional constraints, as follows:

Find: λ , q (10.19)

to minimize: λ

subject to: ( ) 0; 1, 2i i

w f iλ− ≤ =q

Finally, we consider the ε -constraint method, which is discussed in Chapter 8

and reiterated as follows:

Find: q (10.20)

to minimize: ( )s

F q

subject to: ( ) ; 1, 2; i if i i sε≤ = ≠q

10.5 Independent Function Minima

In this section, we present the results when each human performance measure is

minimized independently in the context of (10.16). In doing so, we highlight the

advantages of the new discomfort function; we provide a point of reference for later

261

MOO results; and we determine the components of the utopia points associated with each

target point. We also discuss special circumstances concerning normalization. Finally,

we demonstrate the necessity for coupling delta-potential-energy with other performance

measures.

Each performance measure from section 10.3 is minimized using four different

target points that collectively represent the reach envelope. These target points are listed

in Table 10.3 and shown in Figure 10.8. The coordinates are given in units of cm with

respect to the global coordinate system, which is located at the hip of the human model

(at the point (0,0,0)).

Label Coordinates

Front Right (-38,31,34)

Front Left (25,27,25)

Front Lower (10,-43,55)

Back Right (-25,32,-35)

Table 10.3: Target Point

Coordinates

In anticipation of using MOO, the performance measures are normalized such that

they all have values between zero and one. Thus, no performance measure dominates the

aggregate functions that are used for MOO in later sections. As demonstrated in Chapter

5, normalization is crucial with scalarization methods for MOO. However, as in Chapter

9, the application in this chapter warrants special attention with respect to normalization.

262

Front Right

Back Right

Front Lower

Front Left

X

Y

ZZ

Y

Front Right

Back Right

Front Lower

Front Left

X

Y

Z

X

Y

ZZ

Y

Z

Y

Figure 10.8: Target Points

Note that the feasible space for the problem in (10.16) is variable; it depending on where

the target point is located. Consequently, the utopia point, which is incorporated in the

normalization approach recommended in Chapter 5, changes with each target point. This

makes using the recommended normalization approach impossible. Consequently, the

absolute maximum and minimum, considering all possible target points (the complete

reach envelope), are used for normalization. Each performance measure has an absolute

minimum of zero, achieved at the neutral position. Therefore, each performance measure

is normalized simply by dividing by its corresponding maximum. The maximum values

for joint displacement, discomfort, and delta-potential-energy are 287.7, 221.12, and

992.09, respectively. Note that Pareto-maxima (discussed in Chapter 5) are not

applicable with this approach, because if one considers the complete reach envelope, then

the utopia point (the neutral position) is attainable.

263

The function-comparison matrices for each target point are shown in Table 10.4.

The normalized performance measures are used here for single-objective optimization so

that the single-objective results are comparable to those obtained with MOO. However,

although normalization is used for computations, final values are reported in terms of

non-normalized functions.

Target

Point

Joint Displacement

Values

Discomfort

Values

Delta-Potential-

Energy Values

Minimized Displacement 1.7192 32.5539 10.9054

Minimized Discomfort 4.7659 0.3923 23.7974 Front

Right

Minimized Delta-Pot.-E. 150.5801 120.1475 7.4737



Left




Lower



Minimized Discomfort 25.3436 0.4012 116.1035 Back

Right


Table 10.4: Function-Comparison Matrices for Different Target Points

As with similar matrices in previous chapters, the shaded boxes indicate the individual

minima of the functions, i.e. the utopia point. However, as Figure 10.4 indicates, there is

a different utopia point for each target point. We call these local utopia points. As we

will show, with each target point, the different function-minima correspond to

significantly different points in the design space (postures) and in the criterion space.

Thus, the local utopia points are unattainable, and the objective functions (performance

264

measures) are conflicting; what minimizes one objective results in a significant increase

in other objectives. There is a different type of utopia point when all target points are

considered. Then, the utopia point is =f 0� at the neutral position, which is attainable.

We call this the global utopia point.

Note that the absolute values for the performance measures are not necessarily

significant in terms of quantifying the concepts that each performance measure

represents. Rather, we are concerned with the change in objective-function values as

different postures are assumed.

Minimizing delta-potential-energy can lead to particularly high values for other

performance measures, because it often results in substantial torso rotation about the y-

axis. Such movement can have an especially significant effect on discomfort, which

tends to penalize such movement.

The actual postures corresponding the minimum of each performance measure are

shown in Figures 10.9 through 10.12. Although SantosTM

is capable of predicting

postures with both arms, only the right arm is considered in this study. In evaluating the

visual results, we are concerned primarily with gross movement. The nuances of skin

deflection are not addressed. In fact, the abnormal deformation in the stomach (Figure

10.12) is not necessarily a result of a particular posture or an indication of discomfort. In

addition, the shoulder distortion shown in Figure 10.11 when delta-potential-energy is

used is not necessarily a consequence of the performance measure. Rather, it is a result

of inaccuracies in the skeletal model. However, discussing such inaccuracies is beyond

the scope of this study.

265

DiscomfortJoint Displacement Delta-Potential-EnergyDiscomfortJoint Displacement Delta-Potential-Energy

Figure 10.9: Posture Prediction for Front Right Target Point, using Performance

Measures Independently


Figure 10.10: Posture Prediction for Front Left Target Point, using Performance


266


Figure 10.11: Posture Prediction for Front Lower Target Point, using Performance



Figure 10.12: Posture Prediction for Back Right Target Point, using Performance


267

Based on Figures 10.9 through 10.12, using different human performance

measures as objective functions in (10.16) clearly yields significantly different results. In

fact, although all of the above-mentioned performance measures result in postures that

tend to gravitate towards the neutral position, each has its own set of advantages and

disadvantages.

The joint displacement function reflects a fundamental and common idea, and it

provides reasonable benchmark postures that are visually acceptable. However, it often

results in postures with the arm relatively close to (or actually intersecting) the torso. In

addition, there can be slightly more movement in the torso than one might anticipate.

The new discomfort function corrects the above-mentioned issues. In fact, using

the discomfort performance measure affords distinct improvements over the other

performance measures. For example, with the left front target point in Figure 10.10,

which requires the avatar to reach across his body, using joint displacement results in

interference between the arm and the torso. Of course, this can be remedied by

implementing complex collision detection and collision avoidance algorithms.

Alternatively, the new discomfort function provides a relatively simple approach to

avoiding postures where the elbow hugs or protrudes through the torso. This tendency

results from the penalty associated with joints that are near their limits. In some cases,

however, the discomfort function overcompensates when avoiding postures in which

joints angles are near their limits, as seen in Figure 10.9. We will show that this

overcompensation can be remedied by using MOO to couple discomfort with delta-

potential-energy. With the front lower target point in Figure 10.11, discomfort again

268

results in a more realistic posture when compared to other performance measures. In this

case, the benefit stems from the penalty associated with excessive torso motion. As

discussed earlier, the spine only bends if necessary, and this results in postures that are

more realistic especially when the target points are in front of the avatar. In essence, the

discomfort function provides an accurate approximation of the lexicographic approach to

MOO. With the back right target point in Figure 10.12, it may appear as if using

discomfort results in an unrealistic posture. However, in terms of skeletal mechanics, this

posture is relatively comfortable, because joints in the torso and shoulder are not forced

to approach their limits. The postures assumed when using joint displacement and delta-

potential-energy appear to be slightly more reasonable only because one typically strives

to see the target. In this sense, no form of musculoskeletal discomfort is ideal for

predicting postures that involve target points behind the avatar, although such

performance measures may predict discomfort accurately.

The new delta-potential-energy performance measure provides improvements

over previous human performance measures involving potential energy (Abdel-Malek et

al, 2001; Mi et al, 2002b; Mi, 2004; Marler and Yang (2004)), and it is relatively well

suited for target points behind the avatar. However, in general it often yields unrealistic

results. This is because potential energy is not affected by torso rotational about the y-

axis. Consequently, the delta-potential-energy performance measure tends to results in

excessive torso movement. In addition, it can result in excessive bending in the wrist.

The obvious lack of realism in postures obtained by using delta-potential-energy

independently suggests that human postures are not heavily influenced by potential

269

energy. This conclusion is unintuitive. One would suspect that potential energy effects

human posture significantly, but we find that this is not the case.

Recall that the original hypothesis with the new potential energy function was that

the masses of different body segments provide a natural weighting factor for the different

joint values, thus alleviating the need for the somewhat ad hoc weights in the joint

displacement function. Although the delta-potential-energy function does not provide a

replacement for the joint displacement function and should not be used independently, it

can provide useful results when coupled with other performance measures (i.e.,

discomfort), as we demonstrate in later sections. In fact, whereas the general feasibility

of combining different human performance measures is demonstrated by Yang et al

(2004), in this chapter we take a more extensive look at exactly how such performance

measures can be combined most effectively.

A primary premise supporting the use of optimization-based posture prediction is

that the selection of performance measures used to govern posture, changes depending on

the task that must be accomplished. Initial findings suggest that this hypothesis should be

augmented. Based on the current results with discomfort and delta-potential-energy, we

find that the appropriateness of a performance measure depends on where the task is

completed (where the target point is located) relative to the body.

10.6 Results: No Articulation of Preferences

In this section, the multi-objective posture prediction problem in (10.16) is solved

with no articulation of preferences, providing a basis for comparison when preferences

are incorporated and providing initial insight into the posture prediction problem. MOO

results in general are contrasted with the single-objective results of the previous section,

270

and the advantages of MOO are demonstrated. Whereas the intent of the previous section

was to evaluate the results when MOO was used to create performance measures, the

intent of this section is to evaluate results when MOO is used to combine different

performance measures. Consequently, only the front right target point is used. Joint

displacement is not considered in this study, because it is similar to discomfort and

because it has already been evaluated in previous studies (Marler and Yang, 2004; Yang

et al, 2004). We seek to determine to what extent it is advantageous to combine delta-

potential-energy with discomfort.

In addition to studying the use of MOO in the context of posture prediction, we

use this problem to scrutinize the different MOO methods presented in section 10.4.2

(with =w 1 ). These methods are studied in terms of their solution characteristics and in

terms of computational requirements. The ε -constraint method is not considered here,

because it is not applicable to problems with no articulation of preferences. However, it

is included in later sections with a posteriori articulation.

Results are shown in Table 10.5 with respect to the criterion space. Each row

indicates the values of the objective functions at the solution point determined with a

particular method. Each column indicates the values of a single objective function when

evaluated at the solution points obtained with different methods. The number of SQP

iterations and the number of objective-function evaluations for each method are shown in

Table 10.6. The postures corresponding to these results are shown in Figure 10.13, and

they essentially represent the results in the design space (in terms of iq ).

271

Discomfort

Values Delta-Potential-

Energy Values

Weighted Sum 0.3980 7.7017

Weighted Global Criterion 0.4118 7.6735

Weighted Min-max 1.6863 7.5658

Table 10.5: Performance-Measure Results with No

Articulation of Preferences

# Iterations# Function

Calls

Weighted Sum 299 369

Weighted Global Criterion 153 204

Weighted Min-max 119 169

Table 10.6: Computational Performance with No

Articulation of Preferences

When compared to Figure 10.9, Figure 10.13 suggests that the weighted sum

provides a suitable combination of the features that surface when discomfort and delta-

potential-energy are used independently. This blend is seen more clearly when Figure

10.9 is compared to Figure 10.14. Movement in the spine is negligible, and the arm falls

more naturally towards the side of the avatar. This supports the hypothesis of the last

section that delta-potential-energy should be combined with discomfort rather than used

independently. Thus, we find that in general, using MOO to combine different

performance measures is advantageous.

272

Weighted Global CriterionWeighted Sum Weighted Min-maxWeighted Global CriterionWeighted Sum Weighted Min-max

Figure 10.13: Posture Prediction with MOO and No Articulation of Preferences

Weighted SumWeighted Sum

Figure 10.14: Posture Prediction with Weighted Sum Method - View 2

273

Although the differences between the postures obtained using different MOO

methods are not substantial, Table 10.5 and Figure 10.13 do indicate an increase in

discomfort when the global criterion and min-max methods are used. In terms of the total

range of values for the objective functions, the results are similar, which is common with

these methods. However, the visual results reflect different postures; there is slightly

more bending of the torso compared to the posture obtained with the weighted sum

method. We find that these same trends surface even when the recommended

normalization scheme from Chapter 5 is used with the local utopia point for the front

right target point. Thus, when no preferences are articulated, using the weighted sum

method for MOO tends to provide the most natural postures.

10.6.1 Modified Global Criterion Approach

In this section, we test the modified global criterion, which is developed in

Chapters 6 and 7 to accommodate large exponent values with minimal computational

difficulties. It approximates results to the min-max approach without the use of

additional constraints or an additional design variable. It is formulated as follows:

( )

1

2

max

1

1

p po

i i

i o

i i i

f fU w

f f=

− = + − ∑

q (10.21)

Recall that in this case =w 1 and =f 0� . The results using (10.21) are shown in Table

10.7 with respect to the criterion space. The number of SQP iterations and the number of

objective-function evaluations for each method are shown in Table 10.8. Postures

corresponding these results are shown in Figure 10.15.

274

P-value Discomfort

Values Delta-Potential-

Energy Values

20 0.3986 7.6993

200 0.4078 7.6784

1500 1.5965 7.5965

Table 10.7: Performance-Measure

Results with the Modified Global

Criterion

P-value # Iterations # Function Calls

20 238 289

200 269 321

1500 258 315

Table 10.8: Computational Performance for the

Modified Global Criterion

P=200P=20 P=1500P=200P=20 P=1500

Figure 10.15: Posture Prediction with the Modified Global Criterion

275

Clearly, as the value of p increases the results begin to resemble those obtained with

the min-max method, described in Table 10.5 and Figure 10.13. Other advantages of eth

modified global criterion approach are discussed later in terms of accurately depicting the

Pareto optimal set. However, the modified global criterion approach requires more SQP

iterations and function calls. Nonetheless, with these results and the results in Chapter 5,

we have provided a feasibility study for the modified global criterion. We anticipate its

computational advantages (when compared to the min-max approach) surface when large

numbers of objective functions are considered, which lead to a large number of added

constraints with the min-max method. This issue is discussed later with respect to future

work.

Alternative forms of the global criterion do not function well with this problem.

For instance if the standard form of the global criterion in (10.21) is used without the

extra factor of 1, computational difficulties prevent SNOPT from converging when a

relatively high value for p is used; it becomes impossible to approximate min-max

results.

Furthermore, when the basic global criterion is used (see (6.3)), even with an

exponent value as low as five the solution is not Pareto optimal. Considering that the

formulation theoretically always provides a Pareto optimal solution, this suggests

numerical difficulties. When p is increased, the problem converges prematurely, as

suggested in Chapter 6, and the results become useless. Thus, using the modified global

criterion yields the most reliable and most computationally stable results when relatively

high values for p are used.

276

10.7 Results: A Posteriori Articulation of Preferences

10.7.1 Study of Posture Prediction

Having obtained MOO results with no articulation of preferences in the preceding

section and having determined that MOO can provide beneficial results with this

problem, we now investigate the Pareto optimal sets for a posteriori articulation of

preferences. Based on the definition for Pareto optimality, there is no reason to use a

design point (a set of q-values representing a posture) that is not Pareto optimal, and in

this sense, the Pareto optimal set is a subset of the feasible space, providing further

restrictions on the design points that should be considered. To select a single Pareto

optimal solution point requires some articulation of preferences; the user must make a

decision. By studying the Pareto optimal set as a whole, we can simplify that decision

and gain insight into posture prediction. In fact, we are able to determine to what extent

delta-potential-energy should be combined with discomfort.

First, we use the modified weighted global criterion approach to depict and study

the complete Pareto optimal set for the front right target point. The weights are selected

such that 0i

w ≥ and 1 2

1w w+ = , and a total of 1500 points are plotted. As 1

w is

incremented from zero to one, 2

w is reduced from one to zero. A single optimization

problem is solved for each set of weights. The consequent Pareto set reveals key

characteristics of the posture prediction problem. Then, we investigate the Pareto optimal

sets for the four target points discussed in section 10.5.

The Pareto optimal set is illustrated in Figure 10.16.

277

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

A

B

Pareto

Optimal Set

Figure 10.16: Pareto Optimal Set using the Modified Weighted Global Criterion Method

- Front Right Target Point

This problem provides the same type of Pareto optimal set that is discussed in Chapter 9.

Significant portions of the Pareto optimal set are flat with high trade-offs between

objectives. Theoretically, minimizing the global criterion always results in a Pareto

optimal point, so all of the points illustrated in the corresponding figures for these

methods are Pareto optimal. However, most of the points on the Pareto optimal curve are

approximately weakly Pareto optimal; the value of one of the objective functions is

approximately constant as one moves along the curve to reduce the other objective

function. This means that other than the corner-section circled in Figure 10.16, most

sections of the Pareto optimal set represent points at which one function can be decreased

(while remaining on the Pareto optimal curve) without any significant increase in the

other function. That is, the trade-offs (see Chapters 4 and 8) between objectives are high.

278

Consequently, there is no need to consider such points when searching for a solution

point.

Note that the solutions obtained with single-objective optimization represent the

endpoints of the Pareto optimal curve and thus can be improved. For example, in Figure

10.16, Point A represents the point (posture) that minimizes discomfort. However, by

moving to Point B, where delta-potential-energy is considered but discomfort is weighted

more heavily, delta-potential-energy is reduced significantly with negligible change in

discomfort. The consequent posture is shown in Figure 10.17, while the posture

corresponding to Point A is shown in Figure 10.9 (when discomfort is minimized).

Figure 10.17: Posture Prediction with the Modified Global Criterion and Heavily

Weighted Discomfort

279

As suggested in Table 10.4, at Point A, discomfort is 0.3923, and delta-potential-energy

is 23.7974. At Point B, discomfort is 0.3930 and delta-potential-energy is 7.83340. Note

that the posture in Figure 10.17 is similar to the posture in Figure 10.9 (when discomfort

is minimized), but the potential energy of the arm has been reduced, resulting in a more

natural pose. There is no significant advantage to using Point A rather than Point B, and

this scenario applies to large portions of the Pareto optimal set.

Assuming we view the Pareto set as a new, reduced feasible space, the Pareto

optimal curve indicates the sensitivity of one function to the other. The flat portions

indicate points at which the sensitivities are nearly zero; the value of one function does

not change given a change in the other function. Solution points on these sections of low

sensitivity are provided when one of the weights in a weighted method is significantly

higher than the other weight. Thus, one should avoid weighting vectors towards the

periphery of the weight space; meaning weights with significantly different magnitudes

should not be used when combining discomfort with delta-potential-energy.

Thus far, we have studied the Pareto optimal set as a whole, and we find that

using Pareto optimal points that are far away from the corner section highlighted in

Figure 10.16 is not advantageous. Figure 10.18 zooms in on the corner of the Pareto

optimal set, and we study this section of the set more closely. In Figure 10.18, Points C,

D, and E represent three different points in the criterion space, and their values are given

respectively as ( )0.3984,7.7001 , ( )0.9538,7.6001 , and ( )3.5768,7.5500 . The postures

that correspond to these points are slightly different from each other, as shown in Figure

10.19. The posture obtained at Point C is clearly the most realistic. As with the posture

corresponding to Point B, the posture at Point C is similar to the posture in Figure 10.9

280

(when discomfort is minimized). However, the potential energy of the arm is reduced,

and this results in a more natural pose. These results, along with the results in Figures

10.17 and 10.18, suggest that when combining discomfort and delta-potential-energy, one

should focus on minimizing discomfort as much as possible as long as delta-potential-

energy does not increase to a point where the final solution is nearly weakly Pareto

optimal.

7.45

7.5

7.55

7.6

7.65

7.7

0 1 2 3 4 5 6

7.45

7.5

7.55

7.6

7.65

7.7

0 1 2 3 4 5 6

Discomfort

Energy

C

D

E

Utopia

Point

Non-convex

Portion of the

Pareto Optimal

Set

Figure 10.18: Pareto Optimal Set Corner using the Modified Weighted Global Criterion

Method - Front Right Target Point

The above-mentioned conclusion is only conceptual, and the question remains as

to exactly how the two functions should be aggregated. One option is to use the ε -

constraint method, minimizing discomfort with a constraint on delta-potential-energy.

281

However, if one considers other potential target points, it becomes impossible to predict

an appropriate value for ε .

Point DPoint C Point EPoint DPoint C Point E

Figure 10.19: Postures at Different Points in the Criterion Space, using the Modified

Weighted Global Criterion Method - Front Right Target Point

Alternatively, we find that it is possible to identify an appropriate weighting vector that

can be used in the scalarization methods with any target point. In obtaining the posture

with Point C, the weight for discomfort is approximately 0.7352, and the weight for

delta-potential energy is approximately 0.2648. This set of weights loosely corresponds

to the point where the Pareto optimal curve begins to flatten. Of course, identifying this

point precisely is subjective. However, the weights associated with Point C give a

general guideline indicating how to achieve the most realistic postures. In fact, using

these same weights with the other three target points, results in solutions that are similar

282

to Point C in terms of their position relative to the complete Pareto optimal set. In

conclusion, when combining discomfort and delta-potential-energy with a priori

articulation of preferences, it is best to use the modified global criterion approach with

weights that yield a convex combination of arguments and with the weight for discomfort

set at approximately 0.75. Note that based on the aforementioned results, it would be

more detrimental to use a weight for discomfort that is too small than it would be to use a

weight that is too large. In the extreme, minimizing discomfort independently (using a

weight of one) provides better results than minimizing delta-potential-energy

independently.

The above-mentioned results concern postures that are assumed when reaching

for the front right target point. Next, we use the weighted global criterion method to

evaluate the Pareto optimal sets for the other three target points (discussed in section

10.4). Figures 10.20 through 10.22 illustrate these sets. The general structure of the

Pareto optimal sets is the same as it is for the front right target point. Thus, the

aforementioned conclusions concerning the front right target point apply to each of the

target points. In summary, the overall shape of the Pareto optimal set tends to be

independent of the target point and indicates that many of the Pareto optimal solution

points are essentially useless for all practical purposes.

283

0

2

4

6

8

10

12

14

16

0 5 10 15 20

Pareto

Optimal Set

Figure 10.20: Pareto Optimal Set using the Weighted Global Criterion Method - Front

Left Target Point

0

20

40

60

80

100

120

0 5 10 15 20 25

Pareto

Optimal Set


Lower Target Point

284

0

20

40

60

80

100

120

140

0 10 20 30 40 50

Pareto

Optimal Set

Figure 10.22: Pareto Optimal Set using the Weighted Global Criterion Method – Back

Right Target Point

10.7.2 Study of MOO Methods

Whereas the previous section uses MOO to study the posture prediction problem,

in this section we use the posture prediction problem to reveal contrasting characteristics

of the MOO methods that are presented in section 10.4.2. We study these methods in

terms of their ability to depict the complete Pareto optimal set in the criterion space and

in terms of their ability to capture non-convex portions of the Pareto set. Again, with the

weighted methods, weights are selected such that 0i

w ≥ and 1 2

1w w+ = . With the ε -

constraint method, (8.4) is used to set the limit on ε . A total of 1500 points are used for

each plot. Figures 10.23 through 10.27 illustrate the Pareto optimal set for the front right

target point when different methods are used.

285

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

Pareto

Optimal Set

Figure 10.23: Pareto Optimal Set using the Weighted Sum Method - Front Right Target

Point

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

Pareto

Optimal Set


Right Target Point

286

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

Pareto

Optimal Set

Figure 10.25: Pareto Optimal Set using the Weighted Min-max Method - Front Right

Target Point

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

Pareto

Optimal Set

Figure 10.26: Pareto Optimal Set using the ε -constraint Method with Discomfort as the

Primary Objective - Front Right Target Point

287

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

Pareto

Optimal Set

Figure 10.27: Pareto Optimal Set using the ε -constraint Method with Energy as the

Primary Objective - Front Right Target Point

The weighted sum method provides particularly poor results when it comes to

depicting the complete Pareto optimal set, as shown in Figure 10.16. This is due, in part

to the normalization approach that is used. As demonstrated in Chapter 5, using (5.5)

with the Pareto-maximum can improve the results. In fact, the Pareto optimal points

spread out more evenly across the Pareto optimal set when (5.5) is used with the local

utopia point for the front right target point, as shown in Figure 10.28. Nonetheless, the

complete Pareto optimal set is not clearly depicted.

288

0

5

10

15

20

25

0 20 40 60 80 100 120 140

Discomfort

Energy

Pareto

Optimal Set

Figure 10.28: Pareto Optimal Set using the Weighted Sum Method with Improved

Normalization - Front Right Target Point

As shown in Figure 10.25, the weighted min-max method also performs poorly.

In fact, only 1159 solution points were determined (as apposed to 1500), because the

optimization engine could not satisfy the constraints. When (5.5) is used with this

approach, the computational difficulties are avoided, but the consequent Pareto optimal

set is still incomplete. Thus, with respect to depicting the complete Pareto optimal set,

the modified weighted global criterion method provides significant improvement over the

min-max method. This improvement is shown in Figure 10.16.

With the ε -constraint method, solution points tend to gravitate towards the

minimum of the primary objective function, as determined in Chapter 8 and illustrated in

Figures 10.26 and 10.27. The Pareto optimal set is not well represented. This is a

consequence of the form of the Pareto optimal set for this problem; it is characterized by

relatively large segments that are nearly vertical or nearly horizontal. Based on Figure

289

8.1, if any portion of Pareto optimal curve is nearly parallel to the ε -constraint, it

becomes difficult to capture points on that portion. This condition arises with the posture

prediction problem. In addition, when compared to the other methods for a posteriori

articulation of preferences, the ε -constraint method requires considerably more CPU

time, as suggested in Chapter 8. Generally, with this problem, the ε -constraint provides

week results.

10.7.2.1 Non-convex Portions of the Pareto Optimal Set

Figure 10.18 reveals an important characteristic of the Pareto optimal set, which

in turn demonstrates significant qualities in the various MOO methods with respect to

non-convex Pareto optimal sets. The gap in the Pareto optimal curve marks a non-

convex portion of the Pareto optimal set, and different methods demonstrate varying

degrees of success in capturing points on this portion of the curve. Theoretically, the

weighted sum method is not able to capture points on non-convex portions of a Pareto

optimal set, and as explained in Chapters 6 and 7, global criterion methods with a fixed p-

value may not be able to capture all of the points on non-convex portions of Pareto

optimal sets. We show that the modified global criterion method minimizes the number

of Pareto points that are skipped with non-convex Pareto optimal sets. It essentially

blends features of the global criterion method and the min-max method. Additional

detailed Pareto optimal sets are shown in Figures 10.29 through 10.32.

Theoretically, the min-max formulation provides a necessary condition for Pareto

optimality; it is able to capture all Pareto optimal points, although it may also yield

weakly Pareto optimal points. Consequently, as seen in Figure 10.31, it captures all of

the points on the non-convex portion of the Pareto optimal curve.

290

Non-convex

Portion of the

Pareto Optimal

Set

7.45

7.5

7.55

7.6

7.65

7.7

0 1 2 3 4 5 6

Discomfort

Energy

Utopia

Point

Figure 10.29: Pareto Optimal Set Corner using the Weighted Sum Method with Improved


Non-convex

Portion of the

Pareto Optimal

Set

7.45

7.5

7.55

7.6

7.65

7.7

0 1 2 3 4 5 6

Discomfort

Energy

Utopia

Point

Figure 10.30: Pareto Optimal Set Corner using the Global Criterion Method - Front Right

Target Point

291

Non-convex

Portion of the

Pareto Optimal

Set

7.45

7.5

7.55

7.6

7.65

7.7

0 1 2 3 4 5 6

Discomfort

Energy

Utopia

Point

Figure 10.31: Pareto Optimal Set Corner using the Min-max Method with Improved


Non-convex

Portion of the

Pareto Optimal

Set

7.45

7.5

7.55

7.6

7.65

7.7

0 1 2 3 4 5 6

Discomfort

Energy

Utopia

Point

Figure 10.32: Pareto Optimal Set Corner using the ε -constraint Method with Energy as

the Primary Objective - Front Right Target Point

292

By zooming in on Figure 10.31 further, it becomes clear that no part of the Pareto optimal

curve is horizontal or vertical. Thus, none of the solution points is weakly Pareto

optimal. The ε -constraint method also captures the points on the non-convex portion of

the Pareto optimal curve. However, as demonstrated previously, these two methods do

not effectively depict the complete Pareto optimal set. Alternatively, the modified global

criterion method depicts the complete Pareto set and skips relatively few (compared to

the weighted sum method and the global criterion method) points, as seen in Figure

10.16.


In this chapter, we have demonstrated the advantages of using MOO to predict

human posture. We have used MOO to create new, effective human performance

measures and to combine these measures. To date, its use in this capacity has been

minimal. MOO has been implemented with no articulation of preferences and with a

posteriori articulation of preferences (depicting the Pareto optimal set). As with Chapter

9, we have identified a special form of the Pareto optimal set that surfaces with this

problem. The majority of the points in this set correspond to high trade-offs between the

two objective functions. By exploiting the structure of this set, we were able to

determine the ideal value for weights used with scalarization methods. We have shown

that using these weights yields improved results in terms of realistic postures. This idea

of leveraging theoretical analysis of a Pareto optimal set in order to determine weights

that provide preferable results with general problems is new. In fact, the process of

293

employing the Pareto optimal set to extract practical information rather than just a palette

of potential solutions is unseen in the current literature.

In addition to contributing to human posture prediction capabilities, we have

studied a variety of MOO methods. We have demonstrated that the normalization

scheme recommended in Chapter 5, which involves the Pareto-maximum, provides

improvements when trying to depict the Pareto optimal set. However, we have also

presented a special problem with a variable feasible space that does not lend itself to the

implementation of standard normalization approaches.

We have demonstrated the benefits of the modified weighted global criterion

approach that is developed in Chapters 6 and 7. We have shown that it provides a

successful blend of the capabilities to depict the complete Pareto optimal set and to

capture points on non-convex portions of the set. In addition, we have provided an

example of how the modified weighted global criterion method can approximate the

results of the min-max method. However, in this case, computational demands are

higher. Thus, problems with more objective functions (potential constraints) should be

considered.

With regards to the new discomfort function, we have considered three primary

factors that are involved in mechanical-based discomfort. The next step is to incorporate

joint torque. Then, other types of discomfort should be considered as well (i.e.

environmental, physiological, etc.). The discomfort attributed to joint displacement (one

of the factors incorporated in discomfort) may depend on displacement only in an indirect

way. In actuality, discomfort may directly depend on other factors that in turn depend on

displacement, such as energy.

294

Using weights with joint displacement and trial-and-error experiments is actually

a means of fitting a curve to data inherent in the developers mind. This approach

essentially designs the performance measure around data or experience. It does not

design a performance measure based on a biomechanical concept, and therefore does not

produce performance measures that are well suited to studying how and why humans

move. Discomfort and delta-potential-energy take steps towards providing human

performance measures that can actually be used to study the nature of human posture.

Although they all provide significantly different postures, all three of the

performance measures in this chapter depend on the neutral position. This neutral

position can be adjusted for various types of postures (i.e. standing, sitting, etc.). In this

way, the performance measures developed in this chapter can reflect the tendency of

humans to assume different types of postures for different types of scenarios.

A primary question that arises with any type of human modeling concerns

validation. There are essentially two steps to the validation process: 1) visual inspection

to filter out unacceptable results, and 2) experimental evaluation to precisely evaluate

techniques. This chapter constitutes the completion of the first step.

While many finding are highlighted (italicized) throughout this chapter, the most

significant conclusions are listed as follows:

1) The location of a task, in addition to the general nature of a task, should be factored

into the process of task-based posture prediction. Different performance measures are

more appropriate for different zones around the human.

2) Although most of the human performance measures gravitate to the same neutral

position, they each result in significantly different postures. In general, the new

295

discomfort function yields superior results when compared to other performance

measures.

3) In general, potential energy does not have a significant bearing on human posture.

Nonetheless, the new delta-potential-energy function provides improvements over

previous potential energy functions and is well suited for target points behind the

avatar.

4) Using MOO to combine human performance measures improves the realism of

predicted postures.

5) When discomfort and delta-potential-energy are considered, significant portions of

the consequent Pareto optimal set are flat with high trade-offs between objectives.

This suggests that many of the Pareto optimal solution points are essentially useless.

The overall shape of the Pareto optimal set tends to be independent of the target point.

6) Delta-potential-energy should be combined with discomfort, especially for target

points in front of the human, but its significance is relatively small; weights with

significantly different orders of magnitude should be avoided. In fact, the weight for

discomfort should be approximately 0.75 (with a weight of 0.25 for delta-potential-

energy).

7) Although the ε -constraint method is relatively well suited for bi-criterion problems,

when it comes to depicting the Pareto optimal set (as discussed in Chapter 8), it is

particularly ill suited to problems that have a Pareto optimal set like the one discussed

in this Chapter.

8) The modified weighted global criterion method performs better than the other

methods in this study in terms of coupling the ability to accurately depict the

296

complete Pareto optimal set with the ability to capture points on non-convex portions

of the set. It is also shown to approximate effectively the results of the min-max

method without using extra constraints or an extra design variable.

297

CHAPTER XI

SUMMARY AND CONCLUSIONS

11.1 Summary

Multi-objective optimization (MOO) is becoming an industry focus within

engineering and the managerial (decision-making) sciences, and it has potential for

application to problems in a variety of fields. Widespread popularity across different

social, mathematical, and engineering environments has led to the significant and on-

going development of algorithms. However, such progress also has led to the duplication

of effort without consolidation and to extraneous variations on methods. Method-

analysis has been incomplete on three fronts: 1) individual algorithms, 2) broad classes of

methods with commonalties, and 3) multi-objective optimization as a whole.

Understanding of how methods work, and of the pros and cons for these methods has

been insufficient. Critical information concerning the significance of method-parameters,

the effect of formulation variations, and guidelines for effective use of methods, has been

overlooked and thus underutilized. Finally, attention paid to significant applications has

surfaced only recently, and this trend needs to be developed further. Although MOO has

been used to obtain solutions to various problems, such problems typically are not studied

thoroughly by leveraging both the practical and theoretical aspects of MOO.

With this dissertation, we have provided new analysis revealing practical insight

into how methods work; we have provided new methods and method enhancements that

result in improved performance; and we have provided new understanding of solutions

for significant engineering problems. This dissertation has addressed the above-

298

mentioned deficiencies with three phases. Phase 1 involves a comprehensive qualitative

analysis of methods. Phase 2 entails a focused analysis of, development and

enhancement of, and guidelines for specific common methods. It addresses the

significance of method-parameters, the nature of the solutions, the effectiveness of

different formulations for a given method, and the benefits of different function-

transformation schemes. Phase 3 involves the application of various methods to a system

identification problem for a simplified crash model and to human posture prediction. The

progress that has been made with these three phases is summarized as follows.

Phase 1 of this research, which is provided by Marler and Arora (2003, 2004b)

and is summarized in Chapter 3, responds to the deficiencies in current qualitative

method-analysis. It entails a review of predominant, continuous, and nonlinear, multi-

objective optimization methods, with consideration of engineering applications.

However, this work is more than just a literature review, although it does provide a

comprehensive survey of the currently available MOO methods. Rather, it is an

extensive monograph, providing a thorough outline, analysis, and comparison of

methods. A vast amount of information is funneled into significant and unifying

conclusions concerning specific methods as well as broader groups of methods and the

field of MOO as a whole. Seemingly different methods and terminologies are

consolidated and related. In many cases, subtle differences between supposedly

equivalent ideas are brought to light. Variants of each algorithm are discussed, and new

insight as to how and why algorithms work is given. Methods are evaluated and

compared, and commentary is provided concerning the advantages and pitfalls of

individual methods and different classes of methods. The programming complexity,

299

software-use complexity, and computational complexity are rated. The resulting study

provides a MOO guide for practicing engineers and analysts. In addition, potential for

future research is gleaned from deficiencies in the current literature and results in

motivation for subsequent portions of the dissertation.

Phase 2 of this research involves the study of transformation methods, the

weighted sum method, and the global criterion method (and min-max method), the

weighted global criterion method (and weighted min-max method), and the ε -constraint

method. We have provided new guidelines concerning the most effective use of these

methods. We have provided new understanding of the theory behind these methods.

And, we have provided modifications that enhance the performance of these methods

with respect to depicting the Pareto optimal set and with respect to computational

difficulties. These contributions are summarized as follows, with respect to specific

methods.

We have studied the weighted sum method on a more fundamental level than

what is provided in the literature, and we have presented a unifying explanation of how

the weights should be viewed. We have isolated the factors that govern the solution to

the weighted sum problem. We have solidified the idea that a weight is significant

relative to the values of other weights and relative to the magnitude of its corresponding

objective, and we have provided a lucid explanation of why this is. In terms of a priori

articulation of preferences, we have identified two general classes of algorithms for

setting weights. We find that with rating methods, a convex combination of functions

should be used, whereas with paired comparison methods, the functions should retain

their original units and weights should be unrestricted. We develop the idea that the

300

weighted sum only provides the most basic linear approximation of one’s preference

function and thus is incapable of accurately representing complex preference information,

no matter how the weights are determined. In addition, as a second fundamental

deficiency, we highlight that it can be difficult if not impossible to discern between

setting weights as scaling factors to compensate for differences in the relative magnitudes

of the objective functions and setting weights as indications of preferences.

Drawing on the work by Marler and Arora (in press), we provide a unique study

of transformation methods. Different function-transformation schemes are evaluated

with consideration of imposed function-values and potential computational difficulties.

Using the weighted sum method, transformation schemes are contrasted in terms of their

effect on yielding an even spread of Pareto optimal solutions. With respect to a posteriori

articulation of preferences, we demonstrate potential improvement in the performance of

the weighted sum method, resulting from a convex combination of transformed functions.

In fact, we demonstrate that using the upper-lower-bound approach with the Pareto-

maximum provides superior results. We develop criteria for indicating when other

approaches will yield particularly poor results.

As far as the global criterion is concerned, we have clarified and simplified a

relatively large and potentially confusing set of method-parameters and formulation

variations. From this set, we have derived guidelines for improved performance. We

have revealed characteristics of two general formulations and of two approaches to

incorporating weights, thus providing new understanding of when each variation is most

advantageous. In fact, we have shown that certain formulations are more effective than

others are. We have studied thoroughly the significance of the method-parameters

301

(weights, p-exponent, and aspiration point). We have found that because the exponent p

can represent a type of preference, it should not be limited to a limited number of discrete

values as it is in the current literature, and we have proposed measures to avoid

computational difficulties with higher values of p.

As far as the min-max method is concerned, we have proposed new conditions for

irregularity. We have determined special advantages to using normalized objective

functions with this approach. We have shown that although the weighted min-max

method is touted for providing a necessary condition for Pareto optimality, it is not

particularly well suited for depicting the Pareto optimal set.

We have developed a modified weighted global criterion method that can be used

to approximate the results of the min-max approach without additional constraints or an

additional design variable and with minimal computational difficulties. In addition, this

approach allows for a continuous range of p-values, which can be used to articulate

preferences more effectively. The approach entails a specific formulation, a special

normalization scheme, and a relatively high value for p. In addition, we show that this

approach can provide an attractive blend of the abilities to depict the complete Pareto

optimal set and capture points on non-convex portions of the Pareto optimal set.

The final method we study is the ε -constraint method. We discover its

weaknesses despite the suggestion in the literature that it be preferred to weighted

methods. In fact, it is particularly ill suited for a posteriori articulation of preferences

with problems that involve more than two objective functions. Based on these

discoveries, we propose general guidelines to minimize deficiencies. We provide new

analysis concerning the primary objective, normalization, and the function-constraint

302

limits. We find that there is minimal benefit in using normalization with this approach,

which contrasts sharply with findings for other approaches. We find that when depicting

the Pareto optimal set, solution points may cluster around the minimum of the primary

objective. In this sense, selecting the proper primary objective is important. We have

demonstrated the potential for infeasible problems and duplicate solutions, and we have

provided new understanding as to why these deficiencies surface.

Phase 3 involves the application of the above-mentioned methods to significant,

practical problems. MOO has been applied to problems that had previous been solved

only with single-objective optimization. MOO methods are used not just to provide new

solutions but also to gain new insight into the nature of the problems. In addition, the

problems provide foundations for additional analysis of the methods. First, we

considered a system identification problem for crash models. Much of the work with this

problem is presented by Marler and Arora (2004a). We have identified a special form of

Pareto optimal set and consequently determined that in this case, using no articulation of

preferences can actually be ideal. Interesting results were obtained with the two min-max

methods, indicating that the standard formulation can actually be faster than the alternate

formulation. We suggest that this performance is a consequence of the form of the Pareto

optimal set. We have found that using weights of significantly different magnitudes can

increase computational demands. We also show how standard normalization can result in

computational difficulties if the denominator of the normalization scheme is too large.

As a second application, we consider an optimization-based approach to human

posture prediction. This work is based on studies by Marler and Yang (2004), and Yang

et al (2004). MOO is used to create two new human performance measures and to

303

combine these measures. We test a new discomfort function and find that it successfully

yields more realistic postures than other performance measures. We find that using delta-

potential-energy alone does not provide suitable results, suggesting that potential energy

does not play a significant roll in governing human postures. Nonetheless, we find that it

can be beneficial when used in moderation. In comparing the results of individual

performance measures, we find that different measures are more appropriate for different

zones around the human. Thus, we contend that the concept of task-based posture

prediction should be augmented to reflect this finding.

We show that using MOO to combine human performance measures is, in

general, beneficial. Again, special issues arise with normalization, because with this

problem, the feasible space, which is defined by the target point, is variable. Thus, the

Pareto-maximum cannot be used, and an alternate approach is implemented. As with the

system identification problem, a special form of the Pareto optimal set surfaces and is

exploited to determine how the new performance measures should be combined.

We demonstrate that although the ε -constraint method is relatively well suited

for bi-criterion problems and tends to be used in this capacity in the literature, it is

particularly ill suited for problems involving the form of the Pareto optimal set that we

discover with posture prediction. Furthermore, we determine why such poor

performance occurs. We find that the modified weighted global criterion performs

particularly well, offering both the ability to capture a substantial amount of points on

non-convex portions of the Pareto optimal set and the ability to clearly depict the

complete set.

304

Overall, this research has been successful in revealing and enhancing the

functional intricacies of MOO methods, as well as creating a new method. Phase 1 has

produced a much-needed comprehensive study of the field of multi-objective

optimization, the likes of which had been unavailable in the literature. Phase 2 has shed

new light on the use of function transformations and common MOO methods, all of

which are techniques that have been used with undue naiveté. Finally, phase 3 has

introduced the advantages of MOO studies to complex practical problems.

11.2 Conclusions

Conclusions concerning various classes of methods and MOO in general, are

discussed in Marler and Arora (2003). Specific conclusions concerning the various

studies in this dissertation are discussed in detail with each chapter. In this section, rather

than reiterate previously mentioned ideas and contributions, we discuss recurring,

comprehensive, and unifying lessons that are extracted from this work.

In general, function transformations yield benefits and should always be

considered with MOO problems. However, as with the problems in Chapters 9 and 10,

transforming the objective functions may not be a straightforward process. Nonetheless,

the normalization approach suggested in Chapter 5 typically performs well. An

exception to the rule of using normalization is the use of the weighted sum method with

paired comparison methods for determining weights. In such a case, unmodified weights

should be used without transformed objective functions. In general, the limits for

weights should be similar to the limits for the arguments that the weights modify

(correspond to).

305

In essence, transformations represent a type of formulation variation; the two

ideas are closely related. Both topics concern the magnitudes of the components of the

scalar utility function. Along these same lines, as discussed in Chapters 4 and 5,

mathematically, there is a fine line between using weights as transforming coefficients

and using weights as a means to articulate preferences. However, conceptually, there is a

significant difference between these two intentions. This distinction is not made in the

current literature and is critical for the proper use of the weighted sum method.

We have demonstrated that for a given MOO method, different formulation

variations can provide significantly different results. Essentially, there are often many

methods falling under a single title. This is especially apparent with the global criterion

approach. However, the existence of and consequences of using various formulations for

a single method have been overlooked in the literature.

Much of the work in this dissertation has pertained to depicting the Pareto optimal

set. As computational capabilities increase, using a posteriori articulation of preferences

becomes a more viable approach to MOO problems. While some relatively complex

specialized methods have been developed for a posteriori articulation of preferences

(Marler and Arora, 2003), we have shown that basic, relatively simple, and ease-to-use

scalarization methods also can be effective in depicting the Pareto optimal set if used

properly. To date, their use in this capacity has generally been overlooked. In addition,

we have shown that Pareto optimal sets can be used for more than just presenting a

palette of potential solutions from which one selects a final single point. They may also

be used to extract otherwise unrevealed characteristics of complex problems.

306

In terms of Pareto optimality, this work pertains to locally Pareto optimal

solutions. In fact, except where noted, all of the problems in this dissertation are non-

convex. A multi-objective optimization problem is non-convex if the feasible space and

all of the objective functions are non-convex (Miettinen, 1999). Consequently, it is

possible that different locally Pareto optimal sets (or portions of sets) exist, and different

starting points for the optimization engine can result in different solution points. In fact,

Marler and Yang (2004) provide an example in which two slightly different Pareto

optimal sets surface, in the context of an optimization-based posture prediction problem.

Of course, the potential difficulty of determining locally optimal solution rather than

globally optimal solutions applies to single-objective optimization as well.

Concerning the articulation of preferences, the final goal of a MOO problem is of

course to determine a solution, as it is with any engineering problem. However, with

MOO, it is necessary for the user to articulate preferences in some capacity in order to

determine a final solution point. The user must provide input, typically by approximating

the utility function. Each of the primary methods discussed in this dissertation (i.e.

weighted sum, global criterion, and ε -constraint) involve articulating preferences in a

different way. The modified weighted global criterion method allows for the most

latitude in this respect. The weights, the aspiration point, and the exponent can all be

used to represent a different form of preference. Essentially, this method allows the user

the most latitude in designing an accurate utility function (an approximation of the

preference function). However, preferences tend to be inherently fuzzy or indistinct.

Thus, the precise Pareto optimal solution that is provided may be of little consequence to

the user. That is, the user may not be able to distinguish preferentially Pareto optimal

307

solution points that are mathematically similar. This in turn suggests that the precise

value assigned to a method-parameter is irrelevant to a user, although precision may be

significant mathematically. In terms of practical use, a user need only be concerned with

approximate values for method-parameters.

When forming a utility function, one typically manipulates the objective functions

directly, while the variables are manipulated indirectly. In addition, when making

decisions about the acceptability of the final solution, one usually works with the

criterion space (objective function values). This trend is apparent in the current literature.

However, with the applications in this dissertation, especially with the posture prediction

problem, we have shown that one must also consider the solutions in the design space

(force-time curves and postures) when making decisions.

11.3 Future Work

In addition to the broad directions for future research, which are discussed by

Marler and Arora (2003), this dissertation has flushed out exciting opportunities for

future work.

In Chapter 4, it is shown that a weight is significant relative to the values of other

weights and relative to the values of its corresponding objective function. In Chapter 5, it

is shown that using a convex combination of normalized objective functions is the most

appropriate approach to depicting the Pareto optimal set. However, the results of Chapter

4 suggest that a convex combination of normalized function is sufficient for improved

results, but it is not necessary. If standard normalization is not feasible and the range of

values for the objective functions is no longer between zero and one, as is the case in

308

Chapter 9, what guidelines should be followed when altering the weights in order to

depict the Pareto optimal set? This question needs to be answered.

Although it is significant, the discussion in Chapter 4 surrounding the weighted

sum method in terms of preferences is primarily conceptual. It should be validated. In

particular, the idea that the weighted sum only provides a linear approximation of the

preference function, with which preferences are articulated based on the initial point in

the criterion space, should be verified using interactive human experiments. This would

involve subjective questioning of the users in order to determine their preferences, their

preferred weights, and their opinion of the final solution (in terms of objective function

values).

As suggested earlier, we have demonstrated the ability of basic scalarization

methods to depict the Pareto optimal set effectively. While considering the

recommended enhancements, methods in this dissertation should be compared to other

methods (genetic multi-objective algorithms, physical programming, normal boundary

intersection method, normal constraint method, etc.) that have been designed specifically

for a posteriori articulation of preferences. Methods should be evaluated in terms of

computational efficiency and in terms of the effectiveness with which the Pareto optimal

set is presented (i.e. how even and how complete is the spread of Pareto optimal points).

With the modified global criterion, we have provided a successful feasibility

study. As suggested earlier, further computational advantages could be realized if this

method were used with problems involving a high number of objective functions, which

would result in a high number of function-constraints with the alternate min-max

formulation. One such potential problem is optimization-based human kinematic motion

309

prediction. Currently, the objective function for such a problem is similar to the joint

displacement objective function for posture prediction, but function values are summed

over a sequence of time steps. However, it would be advantageous to restrict the

maximum joint displacement (or discomfort) over time, rather than the total joint

displacement over time. Formulating this idea would entail using the min-max method

with an additional constraint for each time step. This would result in a significant amount

of nonlinear constraints. The modified global criterion could provide a much more

elegant approach to solving this problem.

In addition to making improvements in motion prediction, advancements with

posture prediction can be made by incorporating joint torque and vision into discomfort,

as suggested in Chapter 10. Visual discomfort itself involves the use of an inverse utility

function approach to MOO, for incorporating acuity and distance comfort, which are two

primary components of visual discomfort (Kim et al, 2004). Improvements in modeling

visual discomfort may be possible by considering other MOO methods. In addition,

human motion and posture tends to be variable. For instance, when contacting a specific

target point, people rarely strike the exact same pose twice. Consequently, modeling

such variability in posture prediction, and possibly motion prediction, should be

investigated. Furthermore, a motion-capture system should be employed to 1) test how

accurately the discomfort function actually measures discomfort, and 2) to validate that

the human performance measures accurately govern how people pose. This work is on

going.

The study in Chapter 10 involves four target points that generally represent the

reach envelope. However, the modified weighted global criterion and the proposed

310

approach to combining discomfort and delta-potential energy should be tested with

additional target points. This work also is on going.

A fundamental premise with optimization-based posture and motion prediction is

that human posture and motion are governed in different ways, given different tasks.

That is, they are task-based. Different human performance measures (or combinations of

measures) can be associated with different tasks, and Mi (2004) has laid the groundwork

for this concept. The work of actually associating various tasks or type of tasks with

various performance measures should be initiated. As suggested earlier, inherent in this

association should be the idea that the appropriate performance measure depends on

where the task is completed relative to the body. In addition to requiring a thorough

understanding of MOO, developing a rigorous task-based engine will involve

psychological studies as well and engineering and biomedical studies. We also anticipate

that various neutral positions can be used for different fundamental posture forms (i.e.

standing, sitting, lying down, etc.).

With the system identification problem, there is also a potential for extended

work. When the modified global criterion was used with this problem, the optimization

engine did not complete a single iteration; it seemed to enter an infinite loop. This

occurred even after various function-transformation schemes were used. We suspect that

this is a result of the software rather than the MOO formulation, and this issue should be

investigated further. The issue of premature convergence should also be studied further.

In fact, initial research has begun concerning the dependence of various software

packages (Idesign, Excel Solver, SNOPT, and IMSL) on objective function scales, and

the tendency to provide significantly inaccurate results when the objective function is

311

multiplied times a scalar. The results that were obtained with the two min-max

formulations should be investigated further, to understand when, in general, the standard

formulation can be more efficient than the alternate formulation. Finally, new progress

with methods for illustrating multi-dimensional Pareto optimal sets should be leveraged

with this problem in order to validate and to understand more thoroughly the nature of the

complete Pareto optimal set.

With both the posture prediction problem and the system identification problem,

we have taken advantage of the special form for the Pareto optimal set. To determine

under what conditions this form of Pareto optimal set surfaces could be useful in

analyzing multi-objective problems.

312

REFERENCES

Abdel-Malek, K., Wei, Yu, Mi, Z., Tanbour, E., and Jaber, M. (2001), “Posture

Prediction Versus Inverse Kinematics”, Proceedings of the 2001 ASME Design

Engineering Technical Conferences and Computers and Information in Engineering

Conference, September, Pittsburgh, Pennsylvania, American Society of Mechanical

Engineers, New York, 37-45.

Abdel-Malek, K., Yang, J., Kim, J., Marler, T., Beck, S., and Nebel, K. (2004), "Santos:

A Virtual Human Environment for Human Factors Assessment", 24th Army Science

Conference, November, Orlando, FL, Assistant Secretary of the Army, (Research,

Development and Acquisition), Department of the Army, Washington, DC.

Abdel-Malek, K., Yang, J., Mi, Z., Patel, V. C., and Nebel, K. (2004), “Human Upper

Body Motion Prediction”, Conference on Applied Simulation and Modeling (ASM),

June, Rhodes, Greece.

Adali, S., Richter, A., and Verijenko, V. E. (1995), “Multiobjective Design of Laminated

Cylindrical Shells for Maximum Pressure and Buckling Load”, Microcomputers in

Civil Engineering, 10, 269-279.

Alexander, R. McN. (1997), “A Minimum Energy Cost Hypothesis for Human Arm

Trajectories”, Biological Cybernetics, 76, 97-105.

Allread, W. G., Marras, W. S., and Fathallah, F. A. (1998), “The Relationship, Between

Occupational Musculoskeletal Discomfort and Workplace, Personal, and Trunk

Kinematic Factors”, Proceedings of the Human Factors and Ergonomics Society 42nd

Annual Meeting, October, Chicago, IL, Human Factors and Ergonomics Society,

Santa Monica, CA, 896-900.

Arora, J. S. (2004), Introduction to Optimal Design, 2nd

ed., Elsevier, San Diego, CA.

Arora, J. S. (1989), IDESIGN User’s Manual: Version 3.5.2, Optimal Design Laboratory,

College of Engineering, The University of Iowa, Iowa City, Iowa, 52242, U.S.A.

Athan, T. W., and Papalambros, P. Y. (1996), “A Note on Weighted Criteria Methods for

Compromise Solutions in Multi-Objective Optimization”, Engineering Optimization,

27, 155-176.

Badler, N. I., Phillips, C. B., and Webber, B. L. (1993), Simulated Humans: Computer

Graphics Animation and Control, Oxford University Press, New York.

313

Ballestero, E., and Romero, C. (1991), “A Theorem Connecting Utility Function

Optimization and Compromise Programming”, Operations Research Letters, 10, (7),

421-427.

Bendsoe, M. P., Olhoff, N., and Taylor, J. E. (1984), “A Variational Formulation for

Multicriteria Structural Optimization”, Journal of Structural Mechanics, 11, (4), 523-

544.

Bowman, V. J. (1976), “On The Relationship of The Tchebycheff Norm and The

Efficient Frontier of Multiple-Criteria Objectives”, in Thiriez, H., and Zionts, S.

(eds.), Multiple Criteria Decision Making, Proceedings of a Conference, May, Jouy-

en-Josas, France, 1975, Lecture Notes in Economics and Mathematical Systems, No.

130, Springer-Verlag, Berlin, 76-85.

Carmichael, D. G. (1980), “Computation of Pareto Optima in Structural Design”,

International Journal for Numerical Methods in Engineering, 15, 925-952.

Chaffin, D. B., and Anderson, G. B. J. (1984), Occupational Biomechanics, John Willey

and Sons, New York.

Chankong, V. and Haimes, Y. Y. (1983), Multiobjective Decision Making Theory and

Methodology, Elsevier Science Publishing, New York.

Chen, H., Xi, N., and Chen, Y. (2003), “Multi-objective Optimal Robot Path Planning in

Manufacturing”, Proceedings of the 2003 IEEE/RJS International Conference on

Intelligent Robots and Systems, October, Las Vegas, NV, Institute of Electrical and

Electronics Engineers, New York, 1167-1172.

Chen, W., Sahai, A., Messac, A., and Sundararaj, G. (2000), “Exploration of the

Effectiveness of Physical Programming in Robust Design”, Journal of Mechanical

Design, 122, 155-163.

Chen, W., Wiecek, M. M., and Zhang, J. (1999), “Quality Utility – A Compromise

Programming Approach to Robust Design”, Journal of Mechanical Design, 121, 179-

187.

Chen, Y. –L. (1998), “Weighted-Norm Approach for Multiobjective VAr Planning”,

IEEE Proceedings: generation, Transmission and Distribution, 145, (4), 369-374.

Chen, Y. –C. (1991), “Solving Robot Trajectory Planning Problems with Uniform Cubic

B-Splines”, Optimal Control Applications and Methods, 12, 247-262.

Chung, M. K., Lee, I., Lee, D., Kim, S. H. (2002), “A Postural Workload Evaluation

System Based on a Macro-postural Classification”, Human Factors and Ergonomics

in Manufacturing, 12, (3), 267-277.

314

Cohon, J. L. (1978), Multiobjective Programming and Planning, Academic Press, New

York.

Cohon, J. L., and Marks, D. H. (1973), “Multiobjective Screening Models and Water

Resource Investment”, Water Resources Research, 9, (4), 826-836.

Cohon, J. L., and Marks, D. H. (1975), “A Review and Evaluation of Multiobjective

Programming Techniques”, Water Resources Research, 11, (2), 208-220.

Cruse, H. (1986), “Constraints for Joint Angle Control of the Human Arm”, Biological

Cybernetics, 54, 125-132.

Cruse, H., Wischmeyer, E., Bruwer, M., Brockfeld, P., and Dress, A. (1990), “On the

Cost Functions for the Control of the Human Arm Movement”, Biological

Cybernetics, 62, 519-528.

Das I. (1999) “An Improved Technique for Choosing Parameters for Pareto Surface

Generation Using Normal-Boundary Intersection”, ISSMO/UBCAD/AIASA, Third

World Congress of Structural and Multidisciplinary Optimization, March, Buffalo,

NY, University of Buffalo, Center for Advanced Design.

Das, I., and Dennis, J. E. (1997), “A Closer Look at Drawbacks of Minimizing Weighted

Sums of Objectives for Pareto Set Generation in Multicriteria Optimization

Problems”, Structural Optimization, 14, 63-69.

Das, I., and Dennis, J. E. (1998), “Normal-Boundary Intersection: A New Method for

Generating The Pareto Surface in Nonlinear Multicriteria Optimization Problems”,

SIAM Journal on Optimization, 8, (3), 631-657.

Da Silva, M. C. G. (2002), “Measurements of Comfort in Vehicles”, Measurement

Science and Technology, 13, (6), R41-R60.

Denavit, J., and Hartenberg, R. S. (1955), “A kinematic Notation for Lower-Pair

Mechanisms Based on Matrices”, Journal of Applied Mechanics, 22, 215-221.

Dhingra, A. K., Rao, S. S., and Kumar, V. (1992), “Nonlinear Membership Functions in

Multiobjective Fuzzy Optimization of Mechanical and Structural Systems”, AIAA

Journal, 30, (1), 251 – 260.

Eckenrode, R. T. (1965), “Weighting Multiple Criteria”, Management Science, 12, (3),

180-192.

Eschenauer, H., Koski, J., and Osyczka, A. (eds.) (1990), Multicriteria Design

Optimization Procedures and Applications, Springer-Verlag, Berlin.

315

Farrell, K., and Marler, R. T. (2004), “Optimization-Based Kinematic Models for Human

Posture”, University of Iowa, Virtual Soldier Research Program, Technical Report

Number VSR-04.11.

Flash, T., and Hogan, N. (1985), “The Coordination of Arm Movements: An

Experimental Confirmed Mathematical Model”, The Journal of Neuroscience, 5, (7),

1688-1703.

Freimer, M., and Yu, L. (1976), “Some New Results on Compromise Solutions for Group

Decision Problems”, Management Science, 22, (6), 688-693.

Gembicki, F. W. (1974), “Performance and Sensitivity Optimization: A Vector Index

Approach”, Ph.D. Dissertation, Case Western Reserve University, Cleveland, OH.

Geoffrion, A. M. (1968), “Proper Efficiency and the Theory of Vector Maximization”,

Journal of Mathematical Analysis and Applications, 22, (3), 618-630.

Gilbert, E. M., and Johnson, D. W. (1985), “Distance Functions and Their Application to

Robot Path Planning in the Presence of Obstacles”, IEEE Journal of Robotics and

Automation, 1, 21-30.

Gill, P., Murray, W., and Saunders, A. (2002), “SNOPT: An SQP Algorithm for Large-

Scale Constrained Optimization”, SIAM Journal of Optimization, 12, (4), 979-1006.

Goicoechea, A., Duckstein, L., and Fogel, M. M. (1976), “Multiobjective Programming

in Watershed Management: A Case Study of the Charleston Watershed”, Water

Resources Research, 12, (6), 1085-1092.

Goicoechea, A., Hansen, D. R., and Duckstein, L. (1982), Multiobjective Decision

Analysis with Engineering and Business Applications, John Wiley and Sons, New

York.

Gupta, A. K., and Sivakumar, A. I. (2002), “Simulation Based Multiobjective Scheduling

Optimization in Semiconductor Manufacturing”, Proceedings of the 2002 Winter

Simulation Conference, December, San Diego, CA, Association for Computing

Machinery, New York, 1862-1870.

Haimes, Y. Y., Lasdon, L. S., and Wismer, D. A. (1971), “On a Bicriterion Formulation

of the Problems of Integrated System Identification and System Optimization”, IEEE

Transactions on Systems, Man and Cybernetics, SMC-1, 296-297.

Haug, E. J., and Arora, J. S. (1979), Applied Optimal Design: Mechanical and Structural

Systems, Wiley and Sons, New York.

Hobbs, B. F. (1980), “A Comparison of Weighting Methods in Power Plant Siting”,

Decision Sciences, 11, 725-737.

316

Hollowell, W. T. (1986), “Adaptive Time Domain, Constrained System Identification of

Nonlinear Structures”, Symposium on Vehicle Crashworthiness Including Impact

Biomenchanics, ASME, AMD-Vol. 79/BED-Vol. 1, 105-123.

Holtzman, J. M., and Halkin, H. (1966), “Directional Convexity and the Maximum

Principle for Discrete Systems”, SIAM Journal on Control, 4, (2), 263-275.

Hsiao, Y. –T., and Chien, C. –Y. (2001), “Optimisation of Capacitor Allocation Using an

Interactive Trade-Off Method”, IEE Proceedings: Generation, Transmission and

Distribution, 148, (4), 371-374.

Hunt, B. J., Wiecek, M. M., Fadel, G. (2004), “Matrices as Preference Modeling Tools in

Bi-criterioa Engineering Design”, 10th AIAA/ISSMO Multidisciplinary Analysis and

Optimization Conference, August, Albany, NY, American Institute of Aeronautics

and Astronautics, Washington, DC.

Hwang, C. -L., and Md. Masud, A. S., in collaboration with Paidy, S. R. and Yoon, K.

(1979), Multiple Objective Decision Making, Methods and Applications: A State-of-

the-Art Survey, Beckmann, M., and Kunzi, H. P. (eds.), Lecture Notes in Economics

and Mathematical Systems, No. 164, Springer-Verlag, Berlin.

Jung, E. S., and Choe, J. (1996), “Human Reach Posture Prediction Based on

Psychophysical Discomfort”, International Journal of Industrial Ergonomics, 18,

173-179.

Jung, E. S., Choe, J., and Kim, S. H. (1994), “Psychophysical Cost Function of Joint

Movement for Arm reach Posture Prediction”, Proceedings of the Human Factors

and Ergonomics Society 38th

Annual Meeting, October, Nashville, TN, Human

Factors and Ergonomics Society, Santa Monica, CA, 636-640.

Kahn, M. E., and Roth, B. (1971), “The Near-Minimum-Time Control of Open-Loop

Articulated Kinematic Chains”, Journal of Dynamic System, Measurement, and

Control, 93 Ser G, (3), 164-172.

Kaliszewski, I. (1995), “A Theorem on Nonconvex Functions and Its Application to

Vector Optimization”, European Journal of Operational Research, 80, 439-449.

Karamardian, S. (1965), “Duality in Mathematical Programming”, Ph.D. Dissertation,

University of California, Berkly, CA.

Kassaimah, S. A., Mohamed, A. M., and Kolkailah, F. A. (1995), “Bi-criteria Optimum

Design of Laminated Plates Under Uniform Load and Shear”, Proceedings of the 27th

International SAMPLE Technical Conference, Albuquerque, NM, 27, 731-737.

317

Kim, C., -H. (2001), “Development of Simplified Models for Automotive

Crashworthiness Simulation and Design Using Optimization”, Ph.D. Dissertation,

University of Iowa, Iowa City, IA.

Kim, C. -H., and Arora, J. S. (2003a), “Development of Simplified Dynamic Models

using Optimization: Application to Crushed Tubes”, Computer Methods in Applied

Mechanics and Engineering, 192, 2073-2097.

Kim, C. -H., and Arora, J. S. (2003b), “Nonlinear Dynamic System Identification for

Automotive Crash Using Optimization: A Review”, Structural and Multidisciplinary

Optimization, 25, (1), 2-18.

Kim, C. -H., Mijar, A., R., and Arora, J. S. (2001), “Development of Simplified Models

for Design and Optimization of Automotive Structures for Crashworthiness”,

Structural and Multidisciplinary Optimization, 22, (4), 307-321.

Kim, J. H., Abdel-Malek, K., Mi, Z., Nebel, K. (2005), “Layout Design using an

Optimization-Based Human Energy Consumption Formulation”, SAE 2004 Digital

Human Modeling Symposium, June, Oakland, CA, Society of Automotive Engineers,

Warrendale, PA, SAE paper 2004-01-2175.

Korhonen, P., Salo, S., and Steuer, R. E. (1997), “A Heuristic for Estimating Nadir

Criterion Values in Multiple Objective Linear Programming”, Operations Research,

45, (5), 751-757.

Koski, J. (1984), “Multicriterion Optimization in Structural Design”, in Atrek, E.,

Gallagher, R. H., Ragsdell, K. M., and Zienkiewicz, O. C. (eds.), New Directions in

Optimum Structural Design, John Wiley and Sons, New York, 483-503.

Koski, J. (1985), “Defectiveness of Weighting Method in Multicriterion Optimization of

Structures”, Communications in Applied Numerical Methods, 1, 333-337.

Koski, J., and Silvennoinen, R. (1987), “Norm Methods and Partial Weighting in

Multicriterion Optimization of Structures”, International Journal for Numerical

Methods in Engineering, 24, 1101-1121.

Kumar, N., and Tauchert, T. R. (1992), “Multiobjective Design of Symmetrically

Laminated Plates”, Journal of Mechanical Design, 114, 620-625.

Lewandowski, A., and Wierzbicki, A. P. (1989), “Decision Support Systems Using

Reference Point Optimization”, in Lewandowski, A., and Wierzbicki, A. P. (eds.),

Aspiration Based Decision Support Systems – Theory, Software and Applications,

Lecture Notes in Economics and Mathematical Systems, No. 331, Springer-Verlag,

Berlin, 3-20.

318

Liegeois, A. (1977), “Automatic Supervisory Control of the Configuration and Behavior

of Multibody Mechanism”, IEEE Transactions on Systems, Man and Cybernetics,

SMC-17, (2), 868-871.

Lin, C. -S., Chang, P. -R., and Luh, J. Y. S. (1983), “Formulation and Optimization of

Cubic Polynomial Joint Trajectories for Industrial Robots”, IEEE Transactions on

Automatic Control, AC-28, (12), 1066-1074.

Lin, J. G. (1975), “Three Methods for Determining Pareto-Optimal Solutions of Multiple-

Objective Problems”, in Directions in Large-Scale Systems, Ho, Y. C., and Mitter, S.

K. (eds.), Plenum Press, New York, 117-138.

Lin, J. G. (1976), “Maximal Vectors and Multi-Objective Optimization”, Optimization

Theory and Application, 18, (1), 41-64.

Lo, J., Huang, G., and Metaxas, D. (2002), “Human Motion Planning Based on Recursive

Dynamics and Optimal Control Techniques”, Multibody System Dynamics, 8, 433-

458.

Luc, D. T., and Schaible, S. (1997), “Efficiency and Generalized Concavity”, Journal of

Optimization Theory and Applications, 94, (1), 147-153.

Majid, K. I. (1974), Optimum Design of Structures, Buterworths, London.

Makaraci, M. (2000), “A Multi-Objective Optimization Process for a Generalized

Control System”, Proceedings of Space 2000: The Seventh International Conference

and Exposition on Engineering, Construction, Operations, and Business in Space,

February, Albuquerque, NM, American Society of Civil Engineers, Reston, Virginia,

377-383.

Mansfield, E. (1985), Microeconomics Theory/Applications, 5th

Edition, W. W. Norton

and Company, New York.

Marler, R. T., and Arora, J. S. (2003), “Review of Multi-Objective Optimization

Concepts and Methods for Engineering”, Technical Report Number ODL-01.01,

University of Iowa, Optimal Design Laboratory, Iowa City, IA.

Marler, R., and Arora, J. (2004a), "Study of Multi-Objective Optimization Using

Simplified Crash Models", 10th AIAA/ISSMO Multidisciplinary Analysis and

Optimization Conference, August, Albany, NY, American Institute of Aeronautics

and Astronautics, Washington, DC.

Marler, R. T., and Arora, J. S. (2004b), "Survey of Multi-Objective Optimization

Methods for Engineering", Structural and Multidisciplinary Optimization, 2), (6),

369-395.

319

Marler, R. T., and Arora, J. S. (in press), “Function-Transformation Methods for Multi-

objective Optimization”, Engineering Optimization.

Marler, R. T., and Yang, J. (2004), "Study of a Bi-Criterion Posture Prediction Problem

Using Pareto Optimal Sets", University of Iowa, Virtual Soldier Research Program,

Technical Report Number VSR-04.07.

Masri, S. F., and Caughey, T. K (1979), “A Nonparametric Identification Technique for

Nonlinear Dynamic Problems”, Journal of Applied Mechanics, ASME, 46, 433-447.

Masri, S. F., Chassiakos, A. G., and Caughey, T. K. (1993), “Identification of Nonlinear

Dynamic System Using Neural Networks”, Journal of Applied Mechanics, ASME,

60, 123-133.

Matsumoto, M., Abe, J., and Yoshimura, M. (1993), “Design Optimization for Dynamic

Response of Machine Structures Using a Mini-Max Method”, JSME International

Journal, Series A, 36, (4), 402-409.

Messac, A., and Mattson, C. A. (2002), “Generating Well-Distributed Sets of Pareto

Points for Engineering Design using Physical Programming”, Optimization and

Engineering, 3, 431-450.

Messac, A., Sukam, C. P., and Melachrinoudis, E. (2000a), “Aggregate Objective

Functions and Pareto Frontiers: Required Relationships and Practical Implications”,

Optimization and Engineering, 1, 171-188.

Messac, A., Sukam, C. P., and Melachrinoudis, E. (2001), “Mathematical and Pragmatic

Perspectives of Physical Programming”, AIAA Journal, 39, (5), 885-893.

Messac, A., Sundararaj, G., J., Tappeta, R., V., and Renaud, J., E. (2000b), “Ability of

Objective Functions to Generate Points on Nonconvex Pareto Frontiers”, AIAA

Journal, 38, (6), 1084-1091.

Messac, A., and Yahaya, A., I. (2001), “Required Relationship between Objective

Function and Pareto Frontier Orders: Practical Implications, AIAA Journal, 39, (1),

2168-2173.

Mi, Z. (2004), Task-Based Prediction of Upper Body Motion, Ph.D. Dissertation,


Mi, Z., Yang, J., Abdel-Malek, K., and Jay, L. (2002a), “Planning for Kinematically

Smooth Manipulator Trajectories”, Proceedings of the 2002 ASME Design

Engineering Technical Conferences and Computer and Information in Engineering

Conference, 5B, September, Montreal, Canada, American Society of Mechanical


320

Mi, Z., Yang, J., Abdel-Malek, K., Mun, J. H., and Nebel, K. (2002b), “Real-Time

Inverse Kinematics for Humans”, Proceedings of the 2002 ASME Design

Engineering Technical Conferences and Computer and Information in Engineering

Conference, 5A, September, Montreal, Canada, American Society of Mechanical


Miettinen, K. (1999), Nonlinear Multiobjective Optimization, Kluwer Academic

Publishers, Boston.

Ni, Y. Q., Ko, J. M., and Wong, C. W. (1999), “Nonparametric Identification of

Nonlinear Hsteretic Systems”, Journal of Engineering Mechanics, 125, (2), 206-215.

Osyczka, A. (1978), “An Approach to Multicriterion Optimization Problems for

Engineering Design”, Computer Methods in Applied Mechanics and Engineering, 15,

309-333.

Osyczka, A. (1981), “An Approach to Multicriterion Optimization for Structural Design”,

International Symposium on Optimal Structural Design: 11th

ONR Naval Structural

Mechanics Symposium, Tucson, AZ, University of Arizona, College of Engineering,

Tucson, AZ, 10.37-10.40.

Osyczka, A. (1984), Multicriterion Optimization in Engineering with Fortran Programs,

John Wiley and Sons, New York.

Osyczka, A. (1985), “Computer Aided Multicriterion Optimization Method”, Advances

in Modeling and Simulation, 3, (4), 41-52.

Osyczka, A. (1989), “Multicriterion Decision Making with Min-Max Approach”, in

Optimization Methods in Structural Design, Eschenauer, H., and Olhoff, N. (eds.),

Euromech-Colloquium 164, Wissenschaftsverlag.

Pareto, V. (1906), Manuale di Economica Politica, Societa Editrice Libraria, Milan;

translated into English by A. S. Schwier as Manual of Political Economy, Edited by

A. S. Schwier and A. N. Page, A. M. Kelley, New York, 1971.

Proos, K. A., Steven, G. P., Querin, O. M., and Xie, Y. M. (2001), “Multicriterion

Evolutionary Structural Optimization Using the Weighted and the Global Criterion

Methods”, AIAA Journal, 39, (10), 2006-2012.

Radwan, R. A., and Hollowell, W. T. (1990), “System Identification of Vehicle

Structures in Crash Loading Environments”, SAE Special Publication n 807 on

Vehicle Crashworthiness and Occupant Protection in Frontal Collisions, SAE Paper

900415, 41-56.

321

Rajurkar, K. P., Navelkar, V. V., and Kamelian, Y. (1987), “Application of Multi-

Objective Optimization Techniques to Manufacturing Systems”, Proceedings of the

1987 IEEE International Conference on Systems, Man and Cybernetics, Alexandria,

VA, Institute of Electrical and Electronics Engineers, New York, 513-516.

Rao, J. R., and Roy, N. (1989), “Fuzzy Set Theoretic Approach of Assigning Weights to

Objectives in Multicriteria Decision Making”, International Journal of Systems and

Science, 20, (8), 1381-1386.

Rao, S. S. (1987), "Game Theory Approach for Multiobjective Structural Optimization”,

Computers and Structures, 25, (1), 119-127.

Rao, S. S. (1996), Engineering Optimization: Theory and Practice, 3rd

Edition, John

Wiley and Sons, New York.

Rao, S. S., and Freiheit, T. I. (1991), "A Modified Game Theory Approach to

Multiobjective Optimization”, Journal of Mechanical Design, 113, 286-291.

Rao, S. S., Sundararaju, K., Prakash, B. G., and Balakrishna, C. (1992), “Multiobjective

Fuzzy Optimization Techniques for Engineering Design”, Computers and Structures,

42, (1), 37-44.

Rao, S. S., Venkayya, V. B., and Khot, N. S. (1988), "Game Theory Approach for the

Integrated Design of Structures and Controls”, AIAA Journal, 26, (4), 463-469.

Riffard, V., and Chedmail, P. (1996), “Optimal Posture of a Human Operator and CAD in

Robotics”, Proceedings of the 1996 IEEE International Conference on Robotics and

Automation, April Minneapolis, MN, Institute of Electrical and Electronics


Romero, C. (2000), “Bi-criteria Utility Functions: Analytical Considerations and

Implications in the Short-run Labor Market”, European Journal of Operations

Research, 122, (1), 91-100.

Romero, C., Tamiz, M., and Jones, D. F. (1998), “Goal Programming, Compromise

Programming and Reference Point Method Formulations: Linkages and Utility

Interpretations”, Journal of the Operational Research Society, 49, (9), 986-991.

Ruiz-Canales, P., and Rufian-Lizana, A. (1995), “A Characterization of Weakly Efficient

Points”, Mathematical Programming, 68, (2), 205-212.

Saaty, T. L. (1977), “A Scaling Method for Priorities in Hierarchies, Multiple Objectives

and Fuzzy Sets”, Journal of Mathematical Psychology, 15, (3), 234-281.

Saaty, T. L. (2003), “Decision-making with the AHP: Why is the Principal Eigenvalue

Necessary”, European Journal of Operational Research, 145, 85-91.

322

Saaty, T. L., and Hu, G. (1998), “Ranking by Eigenvector Versus Other Methods in the

Analytic Hierarchy Process”, Applied Mathematical Letters, 11, (4), 121-125.

Sakawa, M. (1993), Fuzzy Sets and Interactive Multiobjective Optimization, Plenum

Press, New York.

Salukvadze, M. E. (1971a), “Optimization of Vector Functionals, I, Programming of

Optimal Trajectories (in Russian), Avtomatika i Telemekhanika, (8), 5-15.

Salukvadze, M. E. (1971b), “Optimization of Vector Functionals, II, The Analytic

Construction of Optimal Controls” (in Russian), Avtomatika i Telemekhanika, (9), 5-

15.

Salukvadze, M. E. (1979), Vector-Valued Optimization Problems in Control Theory,

Academic Press, New York.

Santos, R., Verriest, J. P., and Silva, K. (2000), “Relationship Between Discomfort

Perception and Biomechanical Criteria During A Reaching Task”, Proceedings of the

XIVth Triennial Congress of the International Ergonomics Association and 44th

Annual Meeting of the Human Factors and Ergonomics Association, July, San Diego,

CA, Human Factors and Ergonomics Society, Santa Monica, CA, 333-336.

Saramago, S. F. P., and Ceccarelli, M. (2002), “An Optimum Robot Path Planning with

Payload Constraints”, Robotica, 20, 395-404.

Saramago, S. F. P., and Ceccarelli, M. (2004), “Effect of Basic Numerical Parameters on

a Path Planning of Robots Taking into Account Actuating Energy”, Mechanism and

Machine Theory, 39, 247-260.

Saramago, S. F. P., Steffen, V. S. (1998), “Optimization of the Trajectory Planning of

Robot Manipulators Taking into Account the Dynamics of the System”, Mechanism

and Machine Theory, 33, (7), 883-894.

Saramago, S. F. P., Steffen, V. S. (2000), “Optimal Trajectory Planning of Robot

Manipulators in the Presence of Moving Obstacles”, Mechanism and Machine

Theory, 35, 1079-1094.

Saravanos, D. A., and Chamis, C. C. (1992), “Multiobjective Shape and Material

Optimization of Composite Structures Including Damping”, AIAA Journal, 30, (3),

805-813.

Shabeer, S., and Wang, M. Y. (2000), “Multi-objective Optimization of Sequential

Brakeforming Processes”, Journal of Materials Processing Technology, 102, 266-

276.

323

Shen, W., and Galer, A. R. (1993), “Development of A Pressure Related Assessment

Model of Seating Discomfort”, Proceedings of the Human Factors and Ergonomics

Society 37th

Annual Meeting, October, Seattle, WA, Human Factors and Ergonomics

Society, Santa Monica, CA, 831-835.

Shen, W., and Vertiz, A. M. (1997), “Redefining Seat Comfort”, SAE Special

Publications, v 1242, Progress with Human Factors in Automotive Design: Seating

Comfort, Visibility, and Safety, 161-168.

Shimoda, M., Azegami, H., and Sakurai, T. (1996), “Multiobjective Shape Optimization

of Linear Elastic Structures Considering Multiple Loading Conditions (Dealing with

Mean Compliance Minimization Problems)”, JSME International Journal, 39, (3),

407-414.

Shin, K. G., and Mckay, N. D. (1986), “A Dynamic Programming Approach to

Trajectory Planning of Robotic Manipulators”, IEEE Transactions on Automatic

Control, AC-31, (6), 491500.

Stadler, W. (1988), “Fundamentals of Multicriteria Optimization”, in Stadler, W. (ed.),

Multicriteria Optimization in Engineering and in the Sciences, Plenum Press, New

York, 1-25.

Stadler, W. (1995), “Caveats and Boons of Multicriteria Optimization”, Microcomputers

in Civil Engineering, 10, 291-299.

Stadler, W., and Dauer, J. P. (1992), "Multicriteria Optimization in Engineering: A

Tutorial and Survey" in Kamat, M. P. (ed.), Structural Optimization: Status and

Promise, American Institute of Aeronautics and Astronautics, Washington, DC, 211-

249.

Steuer, R. E. (1989), Multiple Criteria Optimization: Theory, Computation, and

Application, Robert E. Krieger Publishing, Malabar, Florida.

Steuer, R. E., and Choo, E., –U. (1983), “An Interactive Weighted Tchebycheff

Procedure for Multiple Objective Programming”, Mathematical Programming, 26,

326-344.

Surdacki, P., and Montusiewicz, J. (1996), “Approach to Multicriterion Optimization of

Quench Performance of Superconducting Winding”, IEEE Transactions on

Magnetics, 32, (3), 1266-1269.

Tabucanon, M. T. (1988), Multiple Criterion Decision Making in Industry, Elsevier

Science Publishers, Amsterdam.

324

Tappeta, R. V., Renaud, J. E., Messac, E., and Sundararaj, G., J. (2000), “Interactive

Physical Programming: Tradeoff Analysis and Decision Making in Multicriteria

Optimization”, AIAA Journal, 38, (5), 917-926.

Tind, J., and Wiecek, M. M. (1999), “Augmented Lagrangian and Tchebycheff

Approaches in Multiple Objective Programming”, Journal of Global Optimization,

14, 251-266.

Tolani, D., Goswami, A., and Badler, N. (2000), “Real-Time Inverse Kinematics

techniques for Anthropomorphic Limbs”, Graphical Models, 62, (5), 353-388.

Tseng, C. H., and Lu, T. W. (1990), “Minimax Multiobjective Optimization in Structural

Design”, International Journal for Numerical Methods in Engineering, 30, 1213-

1228.

Vincent, T. L. (1983), "Game Theory as a Design Tool”, ASME Journal of Mechanisms,

Transmissions, and Automated Design, 105, 165-170.

Vincent, T. L., and Grantham, W. J. (1981), Optimality in Parametric Systems, John

Wiley and Sons, New York.

Weistroffer, H. R. (1985), “Careful Usage of Pessimistic Values is Needed in Multiple

Objectives Optimization”, Operations Research Letters, 4, (1), 23-25.

Wierzbicki, A. P. (1980), “The Use of Reference Objectives in Multiobjective

Optimization”, in Fandel, G., and Gal, T. (eds.), Multiple Criteria Decision Making

Theory and Application, Lecture Notes in Economics and Mathematical Systems, No.

177, Springer-Verlag, Berlin, 468-486.

Wierzbicki, A. P. (1982), “A Mathematical Basis for Satisficing Decision Making”,

Mathematical Modelling, 3, 391-405.

Wierzbicki, A. P. (1986), “A Methodological Approach to Comparing Parametric

Characterizations of Efficient Solutions”, in Fandel, G., Grauer, M., Kurzhanski, A.,

and Wierzbicki, A. P. (eds.), Large-Scale Modeling and Interactive Decision

Analysis, Lecture Notes in Economics and Mathematical Systems, No. 273, Springer-

Verlag, Berlin, 27-45.

Yang, R. J., Tseng, L., and Cheng, J. (1994), “Feasibility Study of Crash Optimization”,

Advances in Design Automation, Proceedings of the 1994 ASME Design Technical

Conference, 20th

Design Automation Conference, Vol. 2, September, Minneapolis,

MN, American Society of Mechanical Engineers, New York, 549-556.

325

Yang, J., Marler, R., Kim, H., Arora, J., and Abdel-Malek, K. (2004), "Multi-objective

Optimization for Upper Body Posture Prediction", 10th AIAA/ISSMO

Multidisciplinary Analysis and Optimization Conference, August, Albany, NY,

American Institute of Aeronautics and Astronautics, Washington, DC.

Yang, J., Marler, T., Kim, H. J., Farrell, K., Mathai, A., Beck, S., Abdel-Malek, K.,

Arora, J., and Nebel, K. (2005), "Santos: A New Generation of Virtual Humans", SAE

2005 World Congress, April, Detroit, MI, Society of Automotive Engineers,

Warrendale, PA.

Yoon, K. P., and Hwang, C. -L. (1995), Multiple Attribute Decision Making, an

Introduction, Sage Publications, London.

Yu, P. -L. (1973), “A Class of Solutions for Group Decision Problems”, Management

Science, 19, (8), 936-946.

Yu, P. -L. (1985), Multiple-Criteria Decision Making Concepts, Techniques, and

Extensions, Plenum Press, New York.

Yu, P. -L., and Leitmann, G. (1974), “Compromise Solutions, Domination Structures,

and Salukvadze’s Solution”, Journal of Optimization Theory and Applications, 13,

(3), 362-378.

Yu, W. (2001), Optimal Placement of Serial Manipulators, Ph.D. Dissertation,


Zacher, I., and Bubb, H. (2004), “Strength Based Discomfort Model of Posture and

Movement”, SAE Digital Human Modeling for Design and Engineering, June,

Rochester, MI, SAE International, Warrendale, PA, SAE paper 2004-01-2139.

Zadeh, L. A. (1963), “Optimality and Non-Scalar-Valued Performance Criteria”, IEEE

Transactions on Automatic Control, AC-8, 59-60.

Zeleny, M. (1973), “Compromise Programming” in Cochrane, J. L., and Zeleny, M.

(eds.), Multiple Criteria Decision Making, University of South Carolina Press, South

Carolina.

Zeleny, M. (1982), Multiple Criteria Decision Making, McGraw Hill, New York.

Zhang, L. (1996), “Identifying Factors of Comfort and Discomfort in Sitting”, Human

Factors, 38, (3), 377-389.

Zhang, W. H., and Yang, H. C. (2002), “Efficient Gradient Calculation of the Pareto

Optimal Curve in Multicriteria Optimization”, Structural and Multidisciplinary

Optimization, 23, 311-319.

326

Zhang, X., Kuo, A. D., and Chaffin, D. B. (1998), “Optimization-Based Differential

Kinematic Modeling Exhibits A Velocity-Control Strategy for Dynamic Posture

Determination in Seated Reaching Movements”, Journal of Biomechanics, 31, 1035-

1042.

Zhao, J., and Bai, S. –X. (1999), “Load Distribution and Joint Trajectory Planning of

Coordinated Manipulation for Two Redundant Robots”, Mechanism and Machine

Theory, 34, 1155-1170.

a study of multi-objective optimization ......a study of multi-objective optimization methods for...

Documents