the analysis of response surfaces

7 The Analysis of Response Surfaces

• Goal: the researcher is seeking the experimental conditions which are most desirable, that is,determine optimum design variable levels.

• Once the researcher has determined the experimental design region by selecting ranges ofthe experimental variables, an experimental design is chosen and the experimental data iscollected. The coefficients in the model are then estimated. We are considering the fittedsecond-order model or the fitted response surface:

y = b0 +k∑

i=1

bixi +∑ k∑

i<j

bijxixj +k∑

i=1

biix2i (9)

where the b’s are the estimated coefficients. Eq (9) is used to predict the response y for givenx1, x2, . . . , xk and enables the researcher to conduct an analysis of the fitted response surface.

• In matrix notation Eq (9) is given by

y = (10)

where x =

x1x2· · ·xk

b =

b1b2· · ·bk

B =

b11 b12/2 · · · b1k/2

b22 · · · b2k/2· · · · · ·· · · bk−1,k/2

bkk

Note: B is a symmetric matrix.

• Suppose the goal of the experimenter is to estimate the conditions on x1, x2, . . . , xk whichmaximize the response y. Then, the maximum, if it exists, will be a set of conditions on(x1, x2, . . . , xk) such that the partial derivatives

∂y/∂x1, ∂y/∂x2, . . . , ∂y/∂xk

are simultaneously zero.

• This point, say x′s = [x1,0, x2,0, . . . , xk,0], is called the stationary point of the fitted responsesurface.

• We apply the rules for differentiating with respect to a column vector and then equate theresult to zero:

∂y

∂x=

∂

∂x

[b0 + x′b + x′Bx

]=

Solving for x yields the stationary point xs =

• This point may or may not be the point which maximizes the response. The point falls intoone of three categories:

1. xs is the point at which the response surface attains a maximum.

2. xs is the point at which the response surface attains a minimum.

3. xs is a saddle point. That is, the response could either increase or decrease as youmove away from xs (depending on what direction you move).

114

7.1 Canonical Analysis

• The goal is to determine the nature of the stationary point and the entire response surface.

• The nature of the stationary point is determined by the signs of the eigenvalues of B.

• The type of response surface is determined by examining the magnitudes of the eigenvalues.

• Let P be the k×k matrix whose columns are the normalized eigenvectors associated with theeigenvalues λ1, λ2, . . . , λk of B. Then

where Λ = Diag(λ1, λ2, . . . , λk) is a diagonal matrix of the eigenvalues of B.

• To understand a response surface, it is necessary to translate the matrix form in (10) intoanother form which is easier to interpret.

• The analysis begins with a translation of response function in (9) from the origin (x1, x2, . . . , xk) =(0, 0, . . . , 0) to a new origin at the stationary point xs. Let z = (z1, z2, . . . , zk) be the vectorof re-centered x coordinates:

z = x− xs (Shift)

• Now we want to rotate the axes to align with the contours of the response surface. We dothis by use of the eigenvalues λ1, λ2, . . . , λk and the matrix of eigenvectors P.

• The response function is expressed in terms of new variables w1, w2, . . . , wk which correspondto the principal axes. That is, rotate the original axes about x1, x2, . . . , xk by transforming zto w = (w1, w2, . . . , wk ) via the matrix of normalized eigenvectors:

w = P′z (Rotation)

115

• In terms of predicted values, this translation yields

y = b0 + x′b + x′Bx

where ys is the predicted response at the stationary point xs.

• Because PP′ = I, we see P−1 = P′, and because w = P′w, we have z = Pw. Therefore,

y = ys + z′Bz

(11)

• The w-axes are the principal axes of the contour system. The variables w1, w2, . . . , wk arecalled the canonical variables.

• The canonical form of the response surface is obtained by rewriting Eq (11) as:

y = ys +k∑

i=1

λiw2i (12)

• If λi > 0, then y will increase as you move along wi in either direction from xs.

• If λi < 0, then y will decrease as you move along wi in either direction from xs.

• The analysis after reduction of the response surface to canonical form is called a canonicalanalysis.

116

7.2 Interpretation of the Response Surface

• The canonical form in (12) describes the nature of the stationary point and the nature of theresponse surface about the stationary point:

1. If λ1, λ2, . . . , λk are all negative, then the stationary point is a point of .

Note that if all λi < 0 thenk∑

i=1

λiw2i < 0 for any nonzero w. Thus, if we move away from xs

in any direction, y decreases.

2. If λ1, λ2, . . . , λk are all positive, then the stationary point is a point of .

Note that if all λi > 0 then

k∑i=1

λiw2i > 0 for any nonzero w. Thus, if we move away from xs

in any direction, y increases.

3. If λ1, λ2, . . . , λk are mixed in sign, then the stationary point is a . Note

that if the λi are mixed in sign then

k∑i=1

λiw2i can be positive or negative. Thus, an increase or

decrease in y depends upon the direction traveled away from xs.

• We now consider three other types of response surfaces:

1. The stationary ridge system indicates that there is a region, rather than a singlestationary point , where there is approximately a maximum or minimum response. Fig-ure 1a shows a maximum stationary ridge along w1 and Figure 1b shows a minimumstationary ridge along w1.

118

2. The rising ridge system has its stationary point remote from the experimental regionand the estimated response increases as you move along the axis toward the stationarypoint . See the dotted line in the Figure 2a.

3. The falling ridge system has its stationary point remote from the experimental regionand the estimated response decreases as you move along the axis toward the stationarypoint . See the dotted line in the Figure 2b.

• To determine whether a stationary, rising, or falling ridge describes the response surface inthe experimental region, we need to simultaneously study the location of the stationary pointand the magnitude of the eigenvalues.

• The magnitude of the eigenvalues provides information about the shape of the response sur-face. This is best exemplified in response surfaces involving k = 2 variables (and, hence, twoeigenvalues):

– Case I: Suppose λ1 and λ2 are both negative and |λ2| is considerably greater than |λ1|. SeeFigure 3a. The stationary point is a maximum but note the sensitivity of y with respect to thetwo canonical variables w1 and w2. As you move along the w1 axis away from the stationarypoint in either direction, there is less of a change (decrease) in the response relative to thechange (decrease) that occurs when moving a similar distance along the w2 axis away from thestationary point in either direction.

– Case II: Suppose λ1 and λ2 are both positive and |λ2| is considerably greater than |λ1|. SeeFigure 3b. The stationary point is a minimum but note the sensitivity of y with respect to thetwo canonical variables w1 and w2. As you move along the w1 axis away from the stationarypoint in either direction, there is less of a change (increase) in the response relative to thechange (increase) that occurs when moving a similar distance along the w2 axis away from thestationary point in either direction.

In other words, for Cases I & II, the response surface is elongated in the w1 direction and isinsensitive to small changes in movement along the w1 axis.

119

– Case III: Suppose λ1 and λ2 have different signs such that λ1 < 0 and λ2 > 0, and |λ2|is considerably greater than |λ1|. See Figure 4a. The stationary point is a saddle point butnote the sensitivity of y with respect to the two canonical variables w1 and w2. As you movealong the w1 axis away from the stationary point in either direction, there is less of a change(decrease) in the response relative to the change (increase) that occurs when moving a similardistance along the w2 axis away from the stationary point in either direction.

– Case IV: Suppose λ1 and λ2 have different signs such that λ1 < 0, and λ2 > 0 and |λ2|is considerably smaller than |λ1|. See Figure 4b. The stationary point is a saddle point butnote the sensitivity of y with respect to the two canonical variables w1 and w2. As you movealong the w2 axis away from the stationary point in either direction, there is less of a change(increase) in the response relative to the change (decrease) that occurs when moving a similardistance along the w1 axis away from the stationary point in either direction.

• In general, for any response surface with k eigenvalues, if |λj| is considerably greater than |λi|for two eigenvalues λi and λj, the response surface contours are elongated along wi and arenarrower along wj.

• In general, for any response surface with k eigenvalues, if |λi| ≈ |λj| for two eigenvalues λiand λj, the response surface contours are nearly evenly spaced along wi and wj. That is,

|change in y along w1| ≈ |change in y along w2|

when moving similar distances along the axes away from the stationary point .

Near-Stationary Ridges

• The most extreme case with two variables is the stationary ridge. An exact stationary ridgeoccurs when one of the eigenvalues is zero. For the stationary ridges shown in Figures 1a and1b, we have λ1 = 0.

Consider the stationary ridge systems in Figures 1A and 1B. If λ1 ≈ 0, then

y = ys + λ1w21 + λ2w

22 ≈ ys + λ1w

22.

Therefore, changing w1 will have little effect on the predicted value y.

• In practice, if one of the eigenvalues is very small, then the response surface approximates astationary ridge.

120

• Mathematically, the stationary ridge condition arises as a limiting eigenvalue case from amaximum, minimum, or saddle point condition. Thus, it is unlikely that an exact stationaryridge condition (some λi = 0) would occur in practice.

• The practical implications of a near-stationary ridge (some λi ≈ 0) are important to theresearcher.

• For example, if the stationary point is a maximum and λ1 is near zero, then for all practicalpurposes the stationary point does not clearly define optimum operating conditions. In suchcircumstances, the researcher has a range of potential operating conditions along the w1 axis,all of which yield a near-maximum response.

Approximate Rising and Falling Ridges

• Suppose that for k = 2 variables that the stationary point is not near the experimental regionand that λ1 < 0 and λ2 is near zero. The response surface approaches a rising ridge condition.See Figure 5a. The boxed area is the experimental design region.

Figure 5a

• Suppose that for k = 2 variables that the stationary point is not near the experimental regionand that λ1 > 0 and λ2 is near zero. The response surface approaches a falling ridge condition.See Figure 5b. The boxed area is the experimental design region.

Figure 5b

121

7.2.1 Three Examples of Data Analysis

EXAMPLE 1: Two types of fertilizers, a standard N-P-K combination and the other a nutri-tional supplement, were applied to experimental plots to assess the effects on the yield of peanutsmeasured in pounds per plot. The level of the amount (lb/plot) of each fertilizer applied to a plotwas determined by the coordinate settings of a central composite rotatable design. The data belowrepresent the harvested yields of peanuts taken from two replicates of each experimental combina-tion. (Note: the uncoded fertilizer levels corresponding to coded

√2 levels are rounded to 1 decimal

place.)

Fertilizer Fertilizer Coded Coded Yields (lb/plot)

1 2 x1 x2 Rep 1 Rep 2

50 15 -1 -1 6.180 7.080120 15 1 -1 13.455 12.66050 25 -1 1 15.225 13.425120 25 1 1 19.620 17.880

35.5 20 −√

2 0 7.845 9.060

134.5 20√

2 0 16.230 13.755

85 12.9 0 −√

2 9.750 8.895

85 27.1 0√

2 19.635 21.00085 20 0 0 18.495 18.900

DM ’LOG; CLEAR; OUT; CLEAR;’;

ODS PRINTER PDF file=’C:\COURSES\ST578\SAS\canon1.pdf’;

ODS LISTING;

OPTIONS LS=72 NODATE NONUMBER;

**********************;

***** EXAMPLE #1 *****;

**********************;

DATA IN; INPUT F1 F2 YIELD @@; LINES;

50 15 6.180 50 15 7.080

120 15 13.455 120 15 12.660

50 25 15.255 50 25 13.425

120 25 19.620 120 25 17.880

35.5 20 7.845 35.5 20 9.060

134.5 20 16.230 134.5 20 13.755

85 12.9 9.750 85 12.9 8.895

85 27.1 19.635 85 27.1 21.000

85 20 18.495 85 20 18.900

;

PROC RSREG DATA=IN PLOTS=ALL ;

MODEL YIELD = F1 F2 / LACKFIT ;

TITLE ’EXAMPLE 1 -- USING UNCODED DATA IN PROC RSREG’;

RUN;

122

EXAMPLE 1 -- USING UNCODED DATA IN PROC RSREG

The RSREG Procedure

Fit Diagnostics for YIELD

0.996Adj R-Square0.9972R-Square0.9037MSE

12Error DF6Parameters

18Observations

Proportion Less0.0 0.4 0.8

Residual

0.0 0.4 0.8

Fit–Mean

-5

0

5

-2.1 -0.9 0.3 1.5

Residual

0

5

10

15

20

25

30

Perc

ent

0 5 10 15

Observation

0.00

0.05

0.10

0.15

0.20

0.25

Coo

k's

D5 10 15 20

Predicted Value

5

10

15

20

YIE

LD

-2 -1 0 1 2

Quantile

-1

0

1

Res

idua

l

0.3 0.4 0.5 0.6

Leverage

-2

-1

0

1

2

RSt

uden

t

7.5 10.0 12.5 15.0 17.5 20.0

Predicted Value

-2

-1

0

1

2

RSt

uden

t7.5 12.5 17.5

Predicted Value

-1.5

-1.0

-0.5

0.0

0.5

1.0

Res

idua

l

124


The RSREG Procedure

Residual Plots for YIELD

12.5 15.0 17.5 20.0 22.5 25.0 27.5

F2

40 60 80 100 120 140

F1

-1.5

-1.0

-0.5

0.0

0.5

1.0

Res

idua

l


The RSREG Procedure

Residual Plots for YIELD

12.5 15.0 17.5 20.0 22.5 25.0 27.5

F2

40 60 80 100 120 140

F1

-1.5

-1.0

-0.5

0.0

0.5

1.0

Res

idua

l

125

EXAMPLE 2: Data taken from Response Surface Methodology by Raymond Myers, pages 78-79.In this experiment, the researcher wants to gain an insight into the influence of sealing temper-

ature (x1), cooling bar temperature (x2), and % polyethylene additive (x3) on the seal strength ingrams per inch of a breadwrapper stock. The actual levels of the variables are coded as follows:

x1 =seal temp− 255

30x2 =

cooling temp− 55

9x3 =

% polyethylene− 1.1

0.6

On the uncoded scale, five levels of each variable were used:

−1.682 −1 0 1 1.682x1 204.5 225 255 285 305.5x2 39.9 46 55 64 70.1x3 0.09 0.5 1.1 1.7 2.11

The design variables and responses are:

x1 x2 x3 y

-1 -1 -1 6.61 -1 -1 6.9-1 1 -1 7.91 1 -1 6.1-1 -1 1 9.21 -1 1 6.8-1 1 1 10.41 1 1 7.3

x1 x2 x3 y

-1.682 0 0 9.81.682 0 0 5.0

0 -1.682 0 6.90 1.682 0 6.30 0 -1.682 4.00 0 1.682 8.6

0 0 0 10.10 0 0 9.90 0 0 12.20 0 0 9.70 0 0 9.70 0 0 9.6

• From the canonical analysis, the stationary point is xs = (x1, x2, x3) = (−1.01, .26, .68).

• The uncoded levels are

ξ1 = 255 + (30)(−1.01) = 224.7◦ Seal Temperatureξ2 = 55 + (9)(.26) = 57.3◦ Cool Temperatureξ3 = 1.1 + (.6)(.68) = 1.5 Polyethelene Additive

SAS Code for Example 2

******************************************************;*** EXAMPLE #2 --- 3 FACTOR CCD -- MYERS PP.78-84’ ***;******************************************************;

DATA DSGN; INPUT X1 X2 X3 STRENGTH @@; LINES;-1 -1 -1 6.6 1 -1 -1 6.9 -1 1 -1 7.9 1 1 -1 6.1-1 -1 1 9.2 1 -1 1 6.8 -1 1 1 10.4 1 1 1 7.3-1.682 0 0 9.8 1.682 0 0 5.0 0 -1.682 0 6.90 1.682 0 6.3 0 0 -1.682 4.0 0 0 1.682 8.60 0 0 10.1 0 0 0 9.9 0 0 0 12.20 0 0 9.0 0 0 0 9.7 0 0 0 9.6;PROC RSREG DATA=DSGN

PLOTS=(ALL SURFACE(AT(X1=-1 -.5 0 .5 1)));MODEL STRENGTH = X1 X2 X3 / LACKFIT ;

TITLE ’EXAMPLE 2 -- USING CODED DATA IN PROC RSREG’;RUN;

126

EXAMPLE 2 -- USING CODED DATA IN PROC RSREG

The RSREG Procedure

Fit Diagnostics for STRENGTH

0.9824Adj R-Square0.9908R-Square1.2919MSE

10Error DF10Parameters20Observations


Residual

0.0 0.4 0.8

Fit–Mean

-3

-2

-1

0

1

2

-2.8 -1.2 0.4 2

Residual

0

10

20

30

40

Perc

ent

0 5 10 15 20

Observation

0.0

0.1

0.2

0.3

0.4

Coo

k's

D

4 6 8 10 12

Predicted Value

4

6

8

10

12

STR

EN

GT

H

-2 -1 0 1 2

Quantile

-1

0

1

2

Res

idua

l

0.2 0.4 0.6 0.8 1.0

Leverage

-2

-1

0

1

2

RSt

uden

t

5 6 7 8 9 10

Predicted Value

-2

-1

0

1

2

RSt

uden

t5 6 7 8 9 10

Predicted Value

-1

0

1

2

Res

idua

l

128


The RSREG Procedure


The RSREG Procedure

129

EXAMPLE 3: From Response Surfaces by Andre Khuri and John Cornell, pages 188-190.

In this experiment, the researcher wants to investigate the effects of three fertilizer ingredientson the yield of snap beans under field conditions. The fertilizer ingredients and actual amountsapplied were (i) nitrogen (N) from 0.94 to 6.29 lb/plt, (ii) phosphoric acid (P2O5) from 0.59 to 2.97lb/plot, and (iii) potash (K2O) from 0.60 to 4.22 lb/plot. The response of interest is the averageyield in pounds per plot of snap beans. When coded the levels of nitrogen, phosphoric acid, andpotash are:

x1 =N− 3.62

1.59x2 =

P2O5 − 1.78

0.71x3 =

K2O− 2.42

1.07

On the uncoded scale, five levels of each variable were used:

−1.682 −1 0 1 1.682x1 0.94 2.03 3.62 5.21 6.29x2 0.59 1.07 1.78 2.49 2.97x3 0.60 1.35 2.42 3.49 4.22

The design variables and responses are:

x1 x2 x3 y-1 -1 -1 11.281 -1 -1 8.44

-1 1 -1 13.191 1 -1 7.71

-1 -1 1 8.941 -1 1 10.90

-1 1 1 11.851 1 1 11.03

x1 x2 x3 y-1.682 0 0 8.261.682 0 0 7.87

0 -1.682 0 12.080 1.682 0 11.060 0 -1.682 7.980 0 1.682 10.43

x1 x2 x3 y0 0 0 10.140 0 0 10.220 0 0 10.530 0 0 9.500 0 0 11.530 0 0 11.02

SAS Code for Example 3:

DM ’LOG; CLEAR; OUT; CLEAR;’;

* ODS PRINTER PDF file=’C:\COURSES\ST578\SAS\canon3p.pdf’;

ODS LISTING;

OPTIONS PS=54 LS=72 NODATE NONUMBER;

TITLE ’3 FACTOR CCD -- KHURI AND CORNELL (PAGE 190)’;

DATA DSGN; INPUT X1 X2 X3 YIELD @@; LINES;

-1 -1 -1 11.28 1 -1 -1 8.44 -1 1 -1 13.19 1 1 -1 7.71

-1 -1 1 8.94 1 -1 1 10.90 -1 1 1 11.85 1 1 1 11.03

0 0 0 10.14 0 0 0 10.22 0 0 0 10.53

0 0 0 9.50 0 0 0 11.53 0 0 0 11.02

-1.682 0 0 8.26 1.682 0 0 7.87

0 -1.682 0 12.08 0 1.682 0 11.06

0 0 -1.682 7.98 0 0 1.682 10.43

;

PROC RSREG DATA=DSGN PLOTS=(ALL SURFACE(AT(x2=-1 -.5 -.364 0 .5 1)));

MODEL YIELD = X1 X2 X3 / LACKFIT ;

RUN;

130

3 FACTOR CCD -- KHURI AND CORNELL (PAGE 190)

The RSREG Procedure


The RSREG Procedure

Coding Coefficients for theIndependent Variables

Factor Subtracted off Divided by

X1 0 1.682000

X2 0 1.682000

X3 0 1.682000

Response Surface for VariableYIELD

Response Mean 10.198000

Root MSE 0.995974

R-Square 0.7861

Coefficient of Variation 9.7664

Regression DFType I Sumof Squares R-Square F Value Pr > F

Linear 3 7.788261 0.1679 2.62 0.1088

Quadratic 3 13.386268 0.2886 4.50 0.0303

Crossproduct 3 15.290950 0.3297 5.14 0.0209

Total Model 9 36.465478 0.7861 4.08 0.0193

Residual DFSum of

Squares Mean Square F Value Pr > F

Lack of Fit 5 7.380042 1.476008 2.91 0.1333

Pure Error 5 2.539600 0.507920

Total Error 10 9.919642 0.991964


The RSREG Procedure

Parameter DF EstimateStandard

Error t Value Pr > |t|

ParameterEstimate

fromCoded

Data

Intercept 1 10.462435 0.406210 25.76 <.0001 10.462435

X1 1 -0.573718 0.269495 -2.13 0.0591 -0.964993

X2 1 0.183359 0.269495 0.68 0.5117 0.308410

X3 1 0.455468 0.269495 1.69 0.1219 0.766098

X1*X1 1 -0.676356 0.262311 -2.58 0.0275 -1.913495

X2*X1 1 -0.677500 0.352130 -1.92 0.0833 -1.916732

X2*X2 1 0.562543 0.262311 2.14 0.0576 1.591505

X3*X1 1 1.182500 0.352130 3.36 0.0073 3.345439

X3*X2 1 0.232500 0.352130 0.66 0.5240 0.657771

X3*X3 1 -0.273404 0.262311 -1.04 0.3218 -0.773495

Factor DFSum of


X1 4 25.949121 6.487280 6.54 0.0075

X2 4 9.125901 2.281475 2.30 0.1302

X3 4 15.529964 3.882491 3.91 0.0364

131


The RSREG Procedure

Parameter DF EstimateStandard

Error t Value Pr > |t|

ParameterEstimate

fromCoded

Data

Intercept 1 10.462435 0.406210 25.76 <.0001 10.462435

X1 1 -0.573718 0.269495 -2.13 0.0591 -0.964993

X2 1 0.183359 0.269495 0.68 0.5117 0.308410

X3 1 0.455468 0.269495 1.69 0.1219 0.766098

X1*X1 1 -0.676356 0.262311 -2.58 0.0275 -1.913495

X2*X1 1 -0.677500 0.352130 -1.92 0.0833 -1.916732

X2*X2 1 0.562543 0.262311 2.14 0.0576 1.591505

X3*X1 1 1.182500 0.352130 3.36 0.0073 3.345439

X3*X2 1 0.232500 0.352130 0.66 0.5240 0.657771

X3*X3 1 -0.273404 0.262311 -1.04 0.3218 -0.773495

Factor DFSum of


X1 4 25.949121 6.487280 6.54 0.0075

X2 4 9.125901 2.281475 2.30 0.1302

X3 4 15.529964 3.882491 3.91 0.0364


The RSREG ProcedureCanonical Analysis of Response Surface Based on Coded Data


The RSREG ProcedureCanonical Analysis of Response Surface Based on Coded Data

Critical Value

Factor Coded Uncoded

X1 -0.234408 -0.394274

X2 -0.216598 -0.364317

X3 -0.103796 -0.174585

Predicted value atstationary point: 10.502377

Eigenvectors

Eigenvalues X1 X2 X3

1.841262 -0.268043 0.962084 -0.050462

0.367301 0.527320 0.190348 0.828071

-3.304048 0.806280 0.195349 -0.558347

Stationary point is a saddle point.

132


The RSREG Procedure

Fit Diagnostics for YIELD

0.9911Adj R-Square0.9953R-Square

0.992MSE10Error DF10Parameters20Observations


Residual

0.0 0.4 0.8

Fit–Mean

-2

0

2

-2.4 -1.2 0 1.2 2.4

Residual

0

10

20

30

Perc

ent

0 5 10 15 20

Observation

0.0

0.2

0.4

0.6

0.8

Coo

k's

D8 10 12

Predicted Value

8

10

12

YIE

LD

-2 -1 0 1 2

Quantile

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Res

idua

l

0.2 0.4 0.6 0.8 1.0

Leverage

-2

-1

0

1

2

RSt

uden

t

7 8 9 10 11 12

Predicted Value

-2

-1

0

1

2

RSt

uden

t

7 8 9 10 11 12

Predicted Value

-1.0

-0.5

0.0

0.5

1.0

Res

idua

l

133


The RSREG Procedure


The RSREG Procedure

134

the analysis of response surfaces

Documents