a genetic algorithm approach to multiple response optimization

A Genetic Algorithm Approach A Genetic Algorithm Approach to Multiple Response to Multiple Response

OptimizationOptimization

Francisco Ortiz Jr.

James R. Simpson

Joseph J. Pignatiello, Jr.

Alejandro Heredia-Langner

Department of Industrial EngineeringFlorida A&M and Florida State University

• Many industrial problems involve many response solutions – “It is not unusual to find RSM applications with 20 or more

responses…” (Montgomery, JQT, 1999)

• Current methods don’t always work

• Work by Heredia-Langner, and suggestions from Carlyle, Montgomery and Runger (JQT, 2000)

Motivation


CompositesProduction

INPUTS (Factors) OUTPUTS (Responses)

PROCESS:

Resin Transfer Molding

Resin Flow Rate

Mold Complexity

Gate Location

Type of Resin

Fiber Weave

Fiber Weight

Resin Flow Properties

Tensile Strength

Fiber Permeability

Dynamic Mechanical Analysis

Compression Strength

Flexural Strength

Acoustic Properties


Charge Control Agents

Charge Voltage

Colorants

Release Agents

Transfer Voltage

Surface Additives

Laser Toner Development

INPUTS (Factors) OUTPUTS (Responses)

PROCESS:

Toner Transfer

Background

Developer Roll Mass/Area

Spitting

Powder Flow

Isopel Optical Density

Toner-to-Cleaner

Film Onset

24 responses


• As the number of response and decision variables increases– The combined response function typically can be highly

nonlinear, multi-modal and heavily constrained – Conventional optimization methods can get trapped at a

local optimum and even fail to find feasible solutions

Challenges for Current Multiple Response Optimization Methods


• This paper develops and evaluates a multiple response solution technique using a genetic algorithm in conjunction with an unconstrained desirability function

Purpose


• A desirability function for the genetic algorithm

• “Robust-ising” the genetic algorithm for multiple response optimization

• Performance evaluation and comparison

Topics to Cover


• Most methods convert multiple responses with different units of measurements into a single commensurable objective

• Distance or Loss Functions– Khuri and Conlon, 1981– Pignatiello, 1993– Vining, 1998

• Desirability Methods– Derringer and Suich, 1980– Del Castillo, Montgomery, and McCarville, 1996

• Desirability approach converts individual response ŷi into individual desirability di(ŷi)

• Used in combination with optimization algorithm to locate a single solution

Combining Responses


• For example, target as the goal response• Constraint developed based on response’s utility• 0 < di(ŷi) < 1

• Overall desirability

• Optimization using Nelder-Mead simplex or GRG

Desirability Approach

ˆˆ,

ˆˆ ˆ( ) ,

0 , otherwise

i

i

s

i ii i i

i i

t

i ii i i i i

i i

y LL y T

T L

y Hd y T y H

T H

si ti

1/

1 1 2 2ˆ ˆ ˆm

m mD d y d y d y x


• A genetic algorithm (GA) is a search technique that is based on the principles of natural selection

– “survival of the fittest”• Uses the objective function magnitude directly in the search• The GA generates and maintains a population pool of

solutions throughout • It then evaluates the quality of each individual chromosome

using a fitness function – In this case the desirability function = fitness function

Genetic Algorithm


• A designed experiment produces an empirical system model

• A coding scheme widely used is to transform the natural variables into coded variables that fall between -1 and 1

• Here we have multiple fitted responses

),...,(ˆ 1 kxxfy

x1 x2 x3 x4 x5 x6 x7 x8

-0.451 0.346 0.518 0.701 -0.573 -0.634 0.937 -0.448

1 2ˆ ˆ ˆ ˆ[ , ,..., ]my y y y

Using the GA in Response Surface Studies

Chromosome

Gene


• Multiplicative desirability functions– Overall desirability is

– If any di (x) = 0, D(x) = 0

– The GA is unable to compare infeasible solutions

– Perhaps we could extend the D(x) function

Limitations of Current Desirability for the GA

1/

1 1 2 2ˆ ˆ ˆm

m mD d y d y d y x


• GA can be designed to handle constraint violations by using a penalty method

• where

Formulating a Desirability Function

*( ) ( ) ( )DSD D P x x x

21/

1 1 2 2ˆ ˆ ˆ( ) ( )... ( )m

m mP p y p y p y c x


• Every fitted response has an individual desirability di(ŷ) and a penalty pi(ŷ)

• where c is a small constant (e.g. c= 0.0001)


ˆˆ,

ˆˆ ˆ( ) ,

0 , otherwise

i

i

t

i ii i i

i i

s

i ii i i i i

i i

y HT y H

T H

y Ld y L y T

T L

ˆˆ ,

ˆ ˆ( ) ,

ˆˆ ,

i ii i

i i

i i i i i

i ii i

i i

y Lc y L

T L

p y c L y H

y Hc H y

T H


• Hence, the overall unconstrained desirability function is

• The penalty function enables the GA to find feasible solutions

• After feasible solutions are found, P(x) = 0, and no effect on the D*(x)


21/ 1/*

1 1( ) ( ) ( ) ( ) ( )m m

m mD d y d y p y p y c x


• Overall GA desirability function for a case with one response and linear weights (s1 = t1= 1) for the d(y1)

Unbounded Desirability Function

D*


• Proposed desirability function D*(x) vs. DDS (x)

Desirability Function Investigation

Responses Value Low Target High d i d i p i

ŷ1 60 50 60 70 1 1 0.000001

ŷ2 75 50 75 100 1 1 0.000001

ŷ3 50 47 50 53 1 1 0.000001ŷ4 30 40 45 50 0 0 2.000001

0

Responses Value Low Target High d i d i p i (penalty)

ŷ1 60 50 60 70 1 1 0.000001

ŷ2 45 50 75 100 0 0 0.200001

ŷ3 60 47 50 53 0 0 2.333334ŷ4 30 40 45 50 0 0 2.000001

0

Case # 1 Chromosome #1

Overall Desirability -1.34E-09

Case # 1 Chromosome #2

Overall Desirability -9.66E-04

)(xDSD )(* xD

)(xDSD )(* xD

)(xDSD

)(xDSD

)(* xD

)(* xD


• Proposed desirability function D*(x) vs. DDS (x)

Desirability vs. Generations

-0.2

0

0.2

0.4

0.6

0.8

1

1.2

0 5 10 15 20 25 30 35 40 45 50

Generations

De

sira

bil

ity

GA DF

D-S DF

GA Performance Investigation


• The GA can be sensitive to the many parameter choices that must be made for its application

• The GA parameters considered for this study are– Population size– Parent-to-Offspring ratio– Selection type– Mutation type– Mutation rate– Crossover rate

Tuning the GA Parameters

x1 x2 x3 x4 x5 x6 x7 x8

-0.257 -0.914 0.028 0.383 0.429 -0.111 -0.207 -0.812

x1 x2 x3 x4 x5 x6 x7 x8

-0.451 0.346 0.518 0.701 -0.573 -0.634 0.937 -0.448


• Done by incorporating four problem environment parameters incorporated into a designed experiment framework

• Considered noise variables for the purposes of robust design

Factor Problem Parameters Low High A Number of Decision Variables 4 8B Number of Responses 4 16C Percent of 2nd order Models 25 75D Constraints (%of target) 5 15

Creating Multiple Response Problem Scenarios


• Certain rules were used to ensure that the problems generated mimic what typically is found in a real world situation

• Specifications – The decision variables that appear in each response are

chosen randomly– For second order models, ½ of the terms are linear (main

effects), ¼ of the terms are interactions, ¼ of the terms are pure quadratic

– The exact values of the regression coefficients are selected from a uniform probability distribution U(5, 20)

– Model hierarchy is always maintained



• For example, a case with 4 responses and 4 decision variables, 75% of the responses are second order models


21 1 2 1 2 1

2 2 3 2 3

23 1 2 1 2 3

24 3 4 3 4 4

ˆ 50 16 6 12 7

ˆ 50 19 8 13

ˆ 50 19 18 8 11

ˆ 50 7 19 10 14

y x x x x x

y x x x x

y x x x x x

y x x x x x


Factor GA Parameters Low High TypeA Parent Pop. Size 20 50 ControlB Parent/Offspring Ratio 1:1 1:07 ControlC Selection Type Ranking Tournament ControlD Crossover Rate 0.5 0.85 ControlE Mutation Type Uniform Gaussian ControlF Mutation Rate 0.1 0.4 Control

Problem Parameters Low High G Number of Decision Variables 4 8 NoiseH Number of Responses 4 16 NoiseJ Percent of 2nd order Responses 25 75 NoiseK Constraints (% of target) 5 15 Noise

Robust Designed Experiment

• Robust parameter design using a combined array 210-4 resolution IV (80 runs includes 16 pseudo-centers)


Robust Designed Experiment Results

• GA performance metrics were the number of evaluations until– Feasible– Within 10% of optimal, or D*(x) > 0.90

• Response model of the form

• Significant effects included– Noise factor main effects– Control x control interactions– Control x noise interactions

6 6 6 4 6 4

01 1 1, 1 1 1

ˆ , i i ij i j i i ij i ji i j j i i i j

y b b x b x x c z d x z

x z


Robust Designed Experiment Results

• Response: Achieving D*(x) > 0.90

Term Term Type FindingAC Control x Control For parent population = 20, selection type does not matter, whereas if the

parent population = 50, choose tournament selectionAE Control x Control Better performance is achieved with parent population = 20 and Gaussian

mutationAF Control x Control Performance is enhanced using the parent population = 20 and a mutation

rate = 0.4BK Control x Noise An offspring ratio of 1:7 is best with tight constraints (5%), but as the

constraints widen, the offspring ratio is unimportantEG Control x Noise Gaussian mutation works better with only 4 decision variables and both

mutation methods perform equally with 8 decision variablesEK Control x Noise Gaussian performs better with tight constraints (5%), but as the constraints

widen, the mutation type is insignificantAEK Control x Noise For mutation rates of 0.1, use a parent population = 20 and Gaussian

mutation. For all other mutation rates, factor AE is not significantCFK Control x Noise For tournament selection, use mutation rate = 0.4 regardless of constraint

setting. For ranking selection, the FK interaction is not significant


Robust GA Parameter Settings

• All but one determined by response surface models

Factor Parameters Setting Rationale

A Parent Population 20 Designed experiment result

B Parent/Offspring Ratio 1:7 Designed experiment result

C Selection Type Tournament Designed experiment result

D Crossover Rate 0.85 Diversify offspring pool

E Mutation Type Gaussian Designed experiment result

F Mutation Rate 0.4 Designed experiment result


Performance Evaluation

• Evaluate and compare proposed GA method• Considered the multiple response problem scenarios

– Dropped percentage of second order models– 23 factorial plus a center point

Factor A Factor B Factor CCase # No. of Decision Variables No.of Responses Constraint Range

1 4 4 52 8 4 53 4 16 54 8 16 55 4 4 156 8 4 157 4 16 158 8 16 159 6 10 10


• Results of investigation using 30 starting point locations

GRG Performance

Case # No. of Variables No.of Responses ConstraintsPct of GRG Solutions

Optimal1 4 4 5 0.302 8 4 5 0.073 4 16 5 0.074 8 16 5 0.005 4 4 15 0.836 8 4 15 0.277 4 16 15 0.578 8 16 15 0.009 6 10 10 0.07


• Performance of proposed GA using four replicates

Case # No. of Variables No.of Responses Constraints Mean St. Dev. High Low1 4 4 5 0.989 0.004 0.994 0.9852 8 4 5 0.969 0.011 0.979 0.9583 4 16 5 0.997 0.002 0.998 0.9954 8 16 5 0.987 0.005 0.991 0.9825 4 4 15 0.998 0.001 0.998 0.9976 8 4 15 0.994 0.003 0.997 0.9927 4 16 15 0.987 0.004 0.991 0.9838 8 16 15 0.958 0.027 0.984 0.9319 6 10 10 0.979 0.016 0.995 0.962

(95.0% )Confidence Intervals

GA Performance Investigation


• Current study shows consistent, effective performance

• Can be effectively combined with direct search methods (e.g. GA with GRG) to improve run-time performance

• Flexible to handle a host of objective functions– Distance or loss functions can all be applied as long as

covariance information is available

• Able to perform reasonable mapping of response function over design space

Why Use the GA for Multiple Response Optimization?


• 2.4 Genetic Algorithm

– Coding

00001010000110000000011000101010001110111011

is partitioned into 2 halves

0000101000011000000001 and 1000101010001110111011

these strings are then converted from base 2 to base 10 to yield:

x1 = 165377 and x2 = 2270139

The following is an example of real-value coding

0.125 -1.000 0.525


• Recombination – Exchange information/genes between parent chromosome to make new offspring

Parent 12500 85 1780 103 1200 95 1900 98 2500 168 500 40

Parent 21850 53 2200 99 785 67 1000 102 750 75 900 69

Offspring2500 85 1780 103 1200 95 1900 98 750 75 900 69

Crossover Point

• GA cannot rely on recombination alone.


• Mutation – Uniform mutation– Multiple uniform mutation – Guassian mutation

• All entities of the chromosome are mutated such that the resulting chromosome lies somewhere within the neighborhood of it parent.

X1

X2

a genetic algorithm approach to multiple response optimization

Documents

florida state universitytopics

response solutions

covera desirability

number of response

optimization algorithm

multiple responses

individual response

genetic algorithm approach