design optimization under parameter uncertainty for general black-box models

Design Optimization under Parameter Uncertainty for GeneralBlack-Box Models

Ipsita Banerjee and Marianthi G. Ierapetritou*

Department of Chemical and Biochemical Engineering, Rutgers University, Piscataway, New Jersey 08854

Accurate knowledge of the effect of parameter uncertainty on process design and operation isessential for optimal and feasible operation of a process plant. Existing approaches for dealingwith uncertainty at the design and process operations level assume the existence of a well-defined model to represent process behavior and, in almost all cases, convexity of the involvedequations. However, most of the realistic case studies are described by nonconvex and, moreimportantly, poorly characterized models. Thus, a new approach is presented in this paper thatis based on the development of input-output mapping with respect to a system’s flexibility anda design’s optimality. High-dimensional model reduction is used as our basic mappingmethodology, although different mapping techniques can be employed. The result of the proposedapproach is a parametric expression of the optimal objective with respect to uncertain parameters.The proposed approach is general to accommodate black-box models as it does not rely on thenature of the mathematical model of the process. Various examples are presented to illustratethe applicability of the proposed approach.

1. Introduction

Uncertainties in chemical plants appear for a varietyof reasons, both internal, such as fluctuations of valuesof reaction constants and physical properties, or exter-nal, such as quality and flow rates of feed streams. Theneed to account for uncertainty in various stages ofplant operations has been identified as one of the mostimportant problems in chemical plant design andoperations.1-3

Two main problems are associated with the consid-eration of uncertainty in decision making: quantifica-tion of the feasibility and flexibility of a process designand incorporation of the uncertainty within the decisionstage. The quantification of process feasibility is mostcommonly addressed by utilizing the feasibility functionintroduced by Swaney and Grossmann1 that requiresconstraint satisfaction over a specified uncertaintyspace, whereas the flexibility evaluation is associatedwith a quantitative measure of the feasible space. Theflexibility index, as introduced by Swaney and Gross-mann,1 represents the determination of largest hyper-rectangle inscribed within the feasible region of thedesign. Other existing approaches to the quantificationof flexibility involve deterministic measures such as theresilience index (RI) proposed by Saboo et al.4 andstochastic measures such as the design reliabilityproposed by Kubic and Stein5 and the stochastic flex-ibility index proposed by Pistikopoulos and Mazzuchi6

and Straub and Grossmann.7 Recently, Ierapetritou andco-workers8 introduced a new approach to the quanti-fication of process feasibility based on the approximationof the feasible region by a convex hull. Their approachresults in an accurate representation of process feasibil-ity. However, it also relies on the utilization of processmodel and specific convexity assumptions.

Existing approaches for quantifying the effects ofuncertainty in process optimization include sensitivity

analysis and parametric programming techniques. Sen-sitivity analysis refers to post-optimality analysis thatdefines a range of parameter variation for which theidentified solution remains optimal, whereas the theoryof parametric programming provides a systematic methodof analyzing the effect of parameter changes on theoptimal solution of a mathematical programming model.For process engineering problems, sensitivity analysishas been widely used for the analysis of both linear andnonlinear continuous models.9,10 A review of parametricprogramming in linear models is given by Gal.11 Jongenand Weber12 reviewed the theoretical results of para-metric nonlinear programming.

Regarding design under parametric uncertainty, oneof the proposed procedures is the deterministic ap-proach,2,3,13,14 where the description of uncertainty isprovided by specific bounds or via a finite number offixed parameter values. An alternative approach is thestochastic approach,6,15 where uncertainty is describedby probability distribution functions. A combined mul-tiperiod/stochastic optimization formulation has recentlybeen presented by Ierapetritou et al.16 and Hene et al.17

that combines the parametric and stochastic program-ming approaches to deal with synthesis/planning prob-lems.

Most of the existing approaches to modeling uncer-tainty in design/planning problems rely heavily on thenature of model equations and are restricted by theassumption of convexity or the specification of a numberof uncertain parameters in the process model. What islacking is a single method that can successfully predictuncertainty propagation in all models, irrespective oftheir nature or complexity, that is, a method that treatsthe process model as a black box and, given a set ofinput uncertainties, can predict the effect on the output.In this paper, the problem of design under uncertaintyis viewed in a completely different way than it is inexisting techniques. First, the system model is treatedas a black box, and flexibility analysis is performed toevaluate the system’s feasible region using input-

* To whom correspondence should be addressed. Tel.: 732-4452971. E-mail: [email protected].

6687Ind. Eng. Chem. Res. 2002, 41, 6687-6697

10.1021/ie0202726 CCC: $22.00 © 2002 American Chemical SocietyPublished on Web 12/02/2002

output mapping. The second step is the solution of thedesign under uncertainty problem, which is treatedsimilarly to the black-box model where feasibility andoptimization analysis is performed iteratively using amapping method.

In this paper, the high-dimensional model represen-tation (HDMR) technique is used as the mappingmethod between the variations in the input and thechanges in the output. The HDMR method is a familyof tools18,19 that prescribe systematic sampling proce-dures to map out relationships between sets of inputand output model variables. The HDMR technique findsits application mostly in complex kinetics modeling,20

atmospheric chemistry,21 photochemical reaction model-ing, etc., where it efficiently reduces the computationalburden of the original model. The high-dimensionalmodel representation is based on the realization that,for most physical systems, only relatively low ordercorrelations of the input variables will have an impacton the output. Note that other mapping techniques suchas the surface response method can also be utilized inthe proposed methodology.

The paper is organized as follows. Section 2 providesa brief description of the mapping method utilized,whereas section 3 presents a detailed description of theproposed methodology. A number of example problemspresented in section 4 are used to illustrate the ap-plicability, efficiency, and generality of the proposedapproach, whereas section 5 summarizes the work andpresents future directions.

2. Mapping Method: High-Dimensional ModelRepresentation

A traditional approach20 to mapping the behavior ofa system with n input variables x1, ..., xn consists ofsampling each variable at s points to assemble aninterpolated look-up table that requires computationaleffort on the order of sn. Realistically, the number ofsample points is approximately in the range of 10-20,and the number of variables n is 10-100 or larger.Viewed from this perspective, attempts to create a look-up table would be prohibitive. Furthermore, the evalu-ation of a new point by interpolation in an n-dimen-sional space would be exceedingly difficult. However,this analysis implicitly assumes that all n variables areimportant and, most significantly, that correlationsamong all variables at all orders jointly affect thesystem’s output. Hence, the output g(x) can be expressedas a hierarchical correlated function expansion in termsof the input variables as follows

where f0 is a constant; fi(xi) is the function describingthe independent action of the variable xi on the output;fi,j(xi,xj) gives the correlated impact of xi and xj on theoutput; and so on, until the last term f1,2,3,...,n(x1,x2,...,xn),which incorporates any residual correlated behaviorover all of the system variables. This expansion is offinite order and is always an exact representation of themodel output.

The fundamental principle underlying HDMR is that,using the model expansion (eq 1), the order of correla-tions between the independent variables that affect the

system’s output diminishes rapidly. Traditional statisti-cal analysis of model behavior has revealed that avariance and covariance analysis of the output inrelation to the input variables often adequately describethe physics (i.e., only low-order correlations describe thedynamics). On the basis of this observation, a second-order HDMR expression has been shown to exhibitexcellent performance for high-dimensional systems.19

The presence of only low-order variable cooperativitydoes not necessarily imply a small set of significantvariables, nor does it limit the nonlinear nature of theinput-output relationship. However, HDMR might notbe of practical utility for systems where higher-ordercooperativity among variables become significant, therebyrequiring the inclusion of higher-order terms.18 Theexpression for second-order HDMR takes the form

The critical feature of HDMR expansion is that itscomponent functions, f0, fi(xi), fi,j(xi,xj), are optimalchoices tailored to the specific function f(x) over theentire domain of x such that the high-order terms inthe expansion are negligible. There are two commonlyused HDMR expansions, cut HDMR and RS (randomsampling) HDMR, which are two extreme cases ofdifferent HDMR expansions. Cut HDMR depends on thevalue of f(x) at a specified reference point xj, whereasRS HDMR depends on the average value of f(x) overthe entire domain of x. In the present study, cut HDMRhas been used, where the expansion is evaluated rela-tive to the nominal point xj ) (xj1, xj2, ..., xjn) in the overallvariable space. The f0 term is the model output evalu-ated at the nominal point. The higher-order terms areevaluated as cuts in the input variable space throughthe nominal point. Each first-order function fi(xi) isevaluated along its variable axis through the nominalpoint. Each second-order function fi,j(xi,xj) is evaluatedin a plane defined by each binary set of input variablesthrough the nominal point. The functions are definedas18

where the notation xj i indicates that all variables areat their nominal values except xi. Subtracting the lower-order functions removes their dependence to ensure aunique contribution from the new expansion function.As a result, the expansion functions contain informationonly of the specified level of interaction, and they satisfythe null-point criteria

These criteria ensure that the functions in eq 1 areorthogonal using a special inner product defined withrespect to the nominal point. The functions in eqs 1 and3 yield exact information about g(x1,x2,...,xn) along thecut lines, surfaces, and subvolumes for 2-D, 3-D, and4-D cases, respectively, through the nominal points. Theoutput response at any point x, away from the cuts, canbe obtained by first interpolating each of the HDMR

g(x1,x2,...,xn) ) f0 + ∑i)1

n

fi(xi) + ∑1eiejen

fi,j(xi,xj) + ... +

f1,2,3,...,n(x1,x2,...,xn) (1)

g(x1,x2,...,xn) ) f0 +∑i)1

n

fi(xi) + ∑1eiejen

fi,j(xi,xj) (2)

f0 ) g(xj)

fi(xi) ) g(xj i,xi) - f0

fi,j(xi,xj) ) g(xj i,j,xi,xj) - fi(xi) - fj(xj) - f0 (3)

fi,j,...,l(xi,xj,...,xl)|xp)xjp) 0 for p ∈ (i, j, ..., l) (4)

6688 Ind. Eng. Chem. Res., Vol. 41, No. 26, 2002

expansion terms in the look-up tables with respect tothe input values of x and then summing the interpolatedvalues of the HDMR terms from zero order to thehighest required order. An important property of HDMRis that each component function of cut HDMR iscomposed of an infinite subclass of the full multidimen-sional Taylor series. Therefore, any truncated cutHDMR expansion gives a better approximation of f(x)than any truncated Taylor series because the lattercontains only a finite number of terms of the Taylorseries.24

3. Process Design under Uncertainty

3.1. Feasibility Analysis. In any production system,many key parameters can have significant uncertaintyregarding their future values. This uncertainty can ariseeither from a lack of precise knowledge of its value, eventhough the parameter itself is deterministic, or fromrandom fluctuations in the parameter’s value. Thus, inthe choice of a particular process design, flexibility ofthe design is an important component, as it indicatesthe capability of a process to achieve feasible operationover a given range of uncertain conditions. To incorpo-rate flexibility into the design of chemical processes, itis important to analyze whether the given design isfeasible for operation over the range of variation of theuncertain parameter. Given a nominal value of anuncertain parameter, θN, and the expected deviation,∆θ+ and ∆θ-, the flexibility test problem22 for a givendesign d consists of determining whether the inequali-ties fj(d,z,θ) e 0, j ∈ J, holds for all θ ∈ T ) [θ|θL e θ eθU]. This problem is posed as a standard optimizationproblem by defining a scalar variable u such that

For the process to be feasible in the parameter range ofinterest, T ) [θ|θL e θ e θU], it must be establishedthat ψ(d,θ) e 0 for all θ ∈ T. To determine designfeasibility without relying on a process model, HDMRis applied considering problem 5 as a black-box model.Evaluation of HDMR component function is performedaccording to eq 3 satisfying the null-point criteria ofeq 4. In particular, the expansion for a fixed value of dis given by

The second-order HDMR expression is given by

The f0 term is determined with a single model run ofproblem 5 with all of the uncertain parameters (θ) attheir nominal values (θN). The first-order function,fi(θi), is calculated from ψ(θhi,θi) by setting all of the inputvariables except θi to their nominal values (θN) andperforming a series of model runs with the input value

of θi varied over its uncertain range (θL e θ e θU). Thef0 term is then subtracted from each model output toproduce the function fi(θi), as shown in eq 6. The pointssampling the function can be selected as appropriatefor the variation of ψ(θhi,θi). Different sampling tech-niques that can be employed include uniform sampling,random sampling, or sampling based on the probabilitydistribution of the uncertain parameters. For the presentstudy, the sampling points are taken at equal intervalscovering the entire range of uncertainty. If s values areused for each input variable, then (s - 1) model runsare needed for each first-order expansion function on awell-resolved grid. The model run at the nominal pointis not required owing to the null-point criteria of eq 3.Hence, for n uncertain parameters, there are n first-order expansion functions, which requires n(s - 1)model runs. A second-order function, fi,j(θi,θj), is calcu-lated by setting all of the input variables, except θi andθj, to their nominal values and performing a series ofruns with the values of θi and θj being varied to coverthe binary surface input space. Considering each vari-able to be represented with s points, the binary inputsurface will require n(s - 1)2 model runs. The pointsalong the two nominal cuts θi ) θhi and θj ) θhj of thesurface are zero and need not be recalculated. Thecomputational cost of generating the cut HDMR up toLth order is given by

where s is the number of sample points taken along eachaxis. If convergence of the cut HDMR expansion occursat L , n, then the sum in eq 8 is dominated by the Lthterm, and considering s . 1, full space resolution isobtained at the computational cost of ∼(ns)L/L!, ascompared to the conventional view of exponential scal-ing19 ∼sn. After the evaluation of the expansion func-tions, a look-up table is constructed and used to predictthe model output at query points by interpolation.

To evaluate the performance of this technique, theinterpolation results are compared with actual modelruns at the corresponding parameter value, and an errorvalue is computed as follows

where N is the number of interpolation points, úpredictedi

is the value obtained using HDMR prediction, andúactual

i is the solution from the optimization problem.Because feasibility problem 5 was developed to obtain

the range of process feasibility, i.e., range of θ for whichfj(d,z,θ) e 0, to evaluate the performance of the HDMR,the predicted feasible parameter range is compared withthe actual feasible region. The region that is actuallyfeasible but predicted as infeasible is measured andreported as the percentage (%) underprediction. On theother hand, the region that is predicted as feasible butis actually infeasible is reported as the percentage (%)overprediction.

3.2. Design Optimization. Once the feasible regionof process operation has been obtained, the next ques-tions that arise are: (1) How should the effects of

ψ(d,θ) ) minz,u

u

subject to fj(d,z,θ) e u j ∈ J (5)

f0 ) ψ(θh)

fi(θi) ) ψ(θhi,θi) - f0

fi,j(θi,θj) ) ψ(θhi,j,θi,θj) - fi(θi) - fj(θj) - f0 (6)

g(θ1,θ2,...,θn) ) f0 + ∑i)1

n

fi(θi) + ∑1ei<jen

fi,j(θi,θj) (7)

∑i)0

L n!

(n - i)!i!(s - 1)i (8)

ê )1

Nx∑i)1

N (úpredictedi - úactual

i

úactuali )2

(9)

Ind. Eng. Chem. Res., Vol. 41, No. 26, 2002 6689

uncertainty in the design’s economic performance bequantified? (2) How should parameter variability beincorporated into the decision-making stage to deter-mine an optimal but also flexible design?

Incorporating the uncertain nature of the designparameters within the design optimization frameworkresults in the following general optimization problem

where x and z denote vectors of state and controlvariables, respectively; θ is the set of uncertain param-eters; d is the vector of design parameters; h representsthe vector of equalities arising from mass and energybalances and equilibrium relations; and g is the vectorof inequalities corresponding to design specificationsand logical constraints.

To address the first question considering fixed design,it is proposed that the feasibility problem andproblem 9 be considered as black-box models used togenerate the input-output map following the basicideas of HDMR outlined in section 2. In the graphicalrepresentation of the overall proposed approach (Figure1), this step corresponds to the procedure within theshaded box. Addressing question 2 is a major challengebecause it involves the solution of a design optimizationunder uncertainty problem, for which a large numberof papers have been published, as briefly reviewed insection 1. To solve this problem, an additional layer isadded to the proposed approach, as shown in Figure 1.Design variables are treated as additional input vari-ables to derive a parametric expression of the optimaleconomic objective with respect to the design variables.Following this scheme, any model, steady-state ordynamic, can be used inside the optimization boxindependent of its complexity because the basic idea ofthe HDMR approach is to reduce the complexity ofexpensive models.

In detail, the proposed framework employs the fol-lowing steps: (1) The design variables are first dis-cretized following the ideas of HDMR presented insection 2. (2) For each set of design variables, thefeasibility problem is solved following the proceduredescribed in section 3.1. (3) For the feasible combinationof the uncertain parameters, design optimization isperformed, resulting in a parametric expression of the

economic objective with respect to uncertain param-eters. Note that, because the feasibility step is per-formed, the solution of infeasible problems is avoided,thus reducing the computational complexity of thedesign optimization step.

The outcome of the proposed approach is the completerepresentation of the optimal economic objective as aparametric expression of the design and uncertainparameters. For a given design, the above frameworkis reduced to the determination of the optimal objectivefunction with respect to uncertain parameters over thewhole feasible region.

4. Case Studies

In this section, a number of example problems arepresented to illustrate the applicability of the proposedapproach. The first six examples in the followingsubsections follow the feasibility analysis for a givendesign representing different problem characteristics,whereas the examples in subsection 4.2 illustrate theproposed approach for the solution of the design underuncertainty problem.

4.1. Feasibility Analysis. 4.1.1. Example 1: Fea-sibility Analysis for a Linear System. The firstexample considered is a linear problem with two un-certain parameters. The feasibility problem for thisexample takes the form

where z is the control variable and θ1 and θ2 are theuncertain parameters. The above problem is solved byfixing all parameters at their nominal values to obtainthe zeroth-order term. The first-order function is ob-tained by first fixing θ1 at its nominal value of 4 andsolving the problem at 10 values of θ2 over its uncertainrange and then fixing θ2 at its nominal value of 2 andsolving the problem at values of θ1 over its uncertainrange. Finally, the second-order function is obtained byvarying both θ1 and θ2 at 100 points over their rangesof uncertainty. These data points are used to estimatethe model output over the entire uncertain space by theHDMR technique.

The error between the actual solution and the pre-dicted value is found to be 0.1012 by first-order HDMR,whereas there is no error in the second-order estimation.The feasible region of operation as predicted by first-and second-order HDMR is plotted in Figure 2 andcompared with the actual feasible region. The first-orderHDMR is found to estimate the feasible region with0.59% overprediction and 7.4% underprediction and thesecond-order with zero overprediction or underpredic-tion. As clearly illustrated in Figure 2a, the first-orderHDMR results in considerable underprediction, whereasthe second-order HDMR (Figure 2b) estimates theactual region exactly.

4.1.2. Example 2: Feasibility Analysis for aNonlinear Convex System. The example considered

Figure 1. Design optimization under uncertainty.

min u

subject to z - θ1 + θ2/2 e u

z + θ1 - θ2 - 4 e u

-z - θ1/3 - θ2 + 4/3 e u

θ1 ∈ [0 8], θ1N ) 4

θ2 ∈ [0 4], θ2N ) 2 (11)

min C(d,x,z,θ)

subject to h(d,x,z,θ) ) 0

g(d,x,z,θ) e 0 (10)


here is the one presented by Ierapetritou.23 The exampleinvolves the following nonlinear, convex constraints

where θ1 and θ2 are the two uncertain parameters, withnominal values at (0,0). The feasible region describedby the above constraints is illustrated in Figure 3. Thesteps of the proposed feasibility analysis are applied toconstruct an approximation of the feasible region. Forthe first-order HDMR approximation, the data pointsrequired are the solutions of feasibility problem 5 firstat different values of θ1 with θ2 fixed at 0 and then atdifferent values of θ2 with θ1 fixed at 0. The f0 term isevaluated by fixing both parameters at 0. The first-orderapproximation is performed using 21 data points andestimates the feasible region with 4.733% underpredic-tion and 1.331% overprediction. The error value betweenthe actual and predicted values is found to be 6.83 ×10-2. The second-order approximation requires ad-ditional data points for the variation of both θ1 and θ2,resulting in 121 model runs. The error value for thesecond-order HDMR is calculated to be 7.72 × 10-3, with0.8875% underprediction and 0% overprediction. As canbe observed from Figure 3, the first-order approximation

exhibits considerable underprediction and a slight over-prediction, whereas the second-order estimation showsvery good agreement with the actual solution.

4.1.3. Example 3: Feasibility Analysis UsingThree Uncertain Parameters. The example consid-ered here includes three uncertain parameters, withconvex nonlinear constraints defining the feasible regionof design (d1, d2), as presented by Ierapetritou.23 Thefeasibility formulation for this problem has the followingform

where z is the control variable and θ1, θ2, and θ3 arethe uncertain parameters. All of the constraints arejointly convex on θ and z. The design examined herecorresponds to (d1, d2) ) (3, 1) with the feasible regionas depicted in Figure 4. Note that the region is furtherrestricted by the bounds on the uncertain parameters.For this example, first-, second-, and third-orderHDMRs are considered for the feasibility analysis tocompare the effect of increasing order on the error of

Figure 2. Comparison of actual feasible region with HDMRprediction for example 1.

θ12 - θ1 + θ2 - 40 e 0

θ12 + θ1 - θ2 - 2 e 0

-4θ1 + θ2 - 30 e 0

θ1 ∈ [-5 5], θ1N ) 0

θ2 ∈ [-50 50], θ2N ) 0 (12)


min u

subject to-z - θ1 + θ2

2/2 + 2θ33 + d1 - 3d2 - 8 e u

-z - θ1/3 - θ2 - θ3/3 + d2 + 8/3 e u

z + θ12 - θ2 - d1 + θ3 - 4 e u

θ1 ∈ [0 4], θ1N ) 2

θ2 ∈ [0 4], θ2N ) 2

θ3 ∈ [0 4], θ3N ) 2 (13)


predicting the feasible region of design. With threeuncertain parameters, the number of model runs re-quired for first-order estimation is 31, for second-orderestimation is 331, and for third-order estimation is 1331.The error value for the second-order estimation is 4.11× 10-2, and the feasible region is predicted with 0.76%underprediction and no overprediction. Figure 5 il-lustrates the feasible region with respect to θ2 and θ3,with θ1 fixed at a constant value of 2.56. Figure 6illustrates the performance of different orders of HDMR

approximation, showing that, as expected, the perfor-mance improves with increasing order of approximation.It should, however, be noted that the accuracy achievedby the second-order approximation is sufficient for mostproblems.

All of the examples presented above correspond toconvex feasible regions, where it is found that second-order HDMR provides excellent agreement with theactual solution. More importantly, note that the per-centage of overprediction using the proposed approachis zero in all of these examples. However, most chem-ical engineering applications involve nonconvex models,which are the focus of the next examples.

4.1.4. Example 4: One-Dimensional Quasi-Con-vex System. The example presented here is the one-dimensional quasi-convex problem discussed by Iera-petritou.23 Figure 7 illustrates the feasible region boundedby the following four constraints

where θ1 and θ2 are the uncertain parameters. Theproblem is solved following the proposed approach, with21 model runs to obtain the first-order approximation,which predicts the feasible region with 8.728% overpre-

Figure 4. Feasible region bounded by the three constraints inexample 3.

Figure 5. Comparison of actual feasible region with second-orderHDMR prediction for example 3.

Figure 6. Performance of first-, second-, and third-order HDMRfor example 3.


θ12 + 2θ1 - θ2 e 0

(θ1 + 6)θ2 - 350 e 0

θ12 + θ2 - 60 e 0

-10θ1 + θ2 - 50 e 0

θ1 ∈ [-5 5], θ1N ) 0

θ2 ∈ [0 60], θ2N ) 30 (14)


diction and 8.876% underprediction and an error valueof 0.1349. The second-order approximation is performedwith 121 model runs, and it predicts the feasible regionwith 2.219% underprediction and 0% overprediction andan error value of 1.478 × 10-2. As observed with theconvex examples, the second-order HDMR providesexcellent agreement with the actual solution.

4.1.5. Example 5: Feasibility Analysis for aNonconvex Model. The problem considered here ischaracterized by a nonconvex feasible region. Thefeasible region is bounded by four constraints of whichthe second one is nonconvex. The feasibility problem isdefined as follows

where θ1 and θ2 are the uncertain parameters. Com-parison of the actual feasible region with first- andsecond-order HDMR predictions is illustrated in Figure8. With the first-order HDMR, the error value is 0.1981,with 2.663% overprediction and 7.396% underpredictionof the feasible region. With the second-order HDMR, the

error value is calculated to be 4.76 × 10-3, with 0.5917%underprediction and 0% overprediction.

It can be seen from the above examples that theperformance of first-order HDMR deteriorates withnonlinearity and nonconvexity, but second-order HDMRestimation remains consistently accurate. This observa-tion is further substantiated by the following nonconvexexample.

4.1.6. Example 6: Feasibility Analysis for aNonconvex Model. In this example, another noncon-vex problem is considered. The first two constraintsdefine a convex region from which an ellipse is removed,as defined by third constraint, making the feasibleregion highly nonconvex. The feasibility problem isdefined as follows

Figure 9 compares first- and second-order HDMRpredictions with the actual solution. For the first-orderHDMR, the error value is 0.7543, and the feasible regionis predicted with 23.076% overprediction and 9.467%


min u

subject to θ12 + 2θ1 - θ2 e u

-θ12 - θ1 + θ2 - 40 e u

θ12 + θ2 - 60 e u

-10θ1 + θ2 - 50 e u

θ1 ∈ [-5 5], θ1N ) 0

θ2 ∈ [-25 75], θ2N ) 25

(15)


min u

subject to -2θ1 + θ2 - 15 e u

θ12

2+ 4θ1 - θ2 - 5 e u

-(θ1 - 4)2

5-

θ22

0.5+ 10 e u

θ1 ∈ [-10 5], θ1N ) -2.5

θ2 ∈ [-15 15], θ2N ) 0

(16)


underprediction. For the second-order HDMR, the errorvalue is calculated to be 7.95 × 10-2, with 0.8875%underprediction and 0.4438% overprediction. As men-tioned in example 5, the second-order HDMR providedan excellent approximation of the feasible region for thishighly nonconvex case.

4.2. Design Under Uncertainty. Whereas feasibil-ity analysis of a process determines the parameter rangeof feasible operation, optimum operation of the plantand selection of the optimum design under parametricuncertainty are also of interest. Identification of theoptimum requires knowledge of the propagation of inputuncertainty to model output. This section illustrates theperformance of the technique discussed in section 3.2in predicting the variation of the optimal solution withuncertainty in the input parameters and determiningthe optimum design under uncertainty.

4.2.1. Example 1: Uncertainty Analysis for aLinear System. The first example considered is thelinear problem with two uncertain parameters consid-ered in section 4.1.1. The aim is to minimize aneconomic objective represented here by the function C(z)) z subject to three linear constraints. The objective ofthis analysis is to determine the variation of the optimalsolution as a function of the design variable and theparametric uncertainty. The design optimization prob-lem is described by

where θ1 and θ2 are the uncertain parameters and d isthe design parameter. Solution of the above problem isapproached according to the scheme illustrated inFigure 1. The problem is first solved for a fixed design(d ) 5), which reduces it to a parametric programmingproblem of determining the variation of z with theuncertain parameters θ1 and θ2. To this end, thefeasibility problem is first solved to obtain the feasiblerange of θ1 and θ2 (section 4.1.1); then, the variation ofthe optimal solution within the feasible range of theuncertain parameters is analyzed. The second-orderHDMR is found to estimate the variation of the optimalsolution with great accuracy, with an error value of5.6592 × 10-3. To illustrate the performance of theHDMR estimation, two instances of the above analysis(d ) 5) are presented in Figure 10. In plot a, θ1 is fixedat 6, and θ2 is varied over its feasible range as obtainedfrom section 4.1.1, which is [0.8, 4.0]. In plot b, θ2 isfixed at 4, and θ1 is varied within the feasible range of[0, 8].

The next step is to address the design optimizationproblem under parametric uncertainty, which is ap-proached by solving the parametric programming prob-lem for different values of the design parameter in therange [1, 9] with the aim of determining P(d,θ), thevariation of z with θ1, θ2, and d. The feasible region ofoperation with respect to θ1, θ2, and d is illustrated in

Figure 11. Note that the feasible region is furtherrestricted by the bounds on d. Regarding the predictionof the feasible region, second-order HDMR is found toperform well, with 2.11% underprediction and 3.37%overprediction. Figure 12 compares the actual feasibleregion with the HDMR prediction for θ1 fixed at 5.44.Once the feasible range of operation has been deter-mined, the next step is the evaluation of P(d,θ) withinthe feasible range. The second-order HDMR approxi-mates the actual solution with an error value of 2.96 ×10-2. Figure 13 compares the HDMR prediction withthe actual solution for θ1 and θ2 fixed at 5.44 and 4,respectively.

4.2.2. Example 2: Uncertainty Analysis for aNonlinear Convex System. The example presentedhere is the convex, nonlinear problem with threeuncertain parameters considered in section 4.1.3. The

min z

subject to z - θ1 + 1/2θ2 e 0

z + θ1 - θ2 - d e 0

-z - θ1/3 - θ2 + d/3 e 0

θ1 ∈ [0 8], θ1N ) 4

θ2 ∈ [0 4], θ2N ) 2

d ∈[1 9], dN ) 5 (17)

Figure 10. Comparison of HDMR prediction with actual solutionfor example 1: (a) θ1 fixed at 6, θ2 varying within the feasible range[0.8, 4]; (b) θ2 fixed at 4, θ1 varying over entire range. Note that,in both cases, the predicted values match the actual values exactly.

Figure 11. Feasible region of operation, as bounded by linearconstraints of example 1.


objective here is also to minimize the cost function C(z)) z. The proposed framework described in Figure 1 isused to estimate the effect of parametric uncertaintyon the objective function value. The design optimizationproblem is described by

where z is the control variable; θ1, θ2, and θ3 are theuncertain parameters; and d1 and d2 are the designparameters. Following the procedure explained in sec-tion 4.2.1, the above problem is first solved for aparticular design with d1 ) 3 and d2 ) 1 to determine

the cost function with respect to the uncertain param-eters. Using 331 model solutions for second-order HDMRestimation, the error between the actual and predictedsolutions is found to be 3.73 × 10-2. Note that only thefeasible range of θ1, θ2, and θ3, as determined followingthe approach of section 4.1.3, is used, minimizing thecomputational requirement of solving infeasible prob-lems. Figure 14 shows the variation of the optimalsolution with θ2 and θ3 for θ1 fixed at 0.16 and comparesthe HDMR prediction with the actual solution. The errorbetween the actual solution and the HDMR predictionfor θ1 fixed near the mean is 3.3 × 10-2 and for θ1 fixedat 4 is 5.8 × 10-2. To obtain a better view of HDMRperformance, two instances of the above variation areillustrated in Figure 15 for θ2 fixed at 0.8 and 3.68,which are the extreme values of the feasible region.Figure 16 illustrates the variation of the optimal solu-tion with θ3 for θ1 and θ2 at their mean values. Note

Figure 12. Comparison of actual feasible region with HDMRprediction for example 1 with θ1 fixed at 5.44.

Figure 13. Comparison of HDMR prediction with actual solutionfor example 1 with θ1 fixed at 5.44 and θ2 fixed at 4.

min z

subject to-z - θ1 + θ2

2/2 + 2θ33 + d1 - 3d2 - 8 e 0

-z - θ1/3 - θ2 - θ3/3 + d2 + 8/3 e 0

z + θ12 - θ2 + θ3 - d1 - 4 e 0

θ1 ∈ [0 4], θ1N ) 2

θ2 ∈ [0 4], θ2N ) 2

θ3 ∈ [0 4], θ3N ) 2

d1 ∈ [1 5], d1N ) 3

d2 ∈ [1 5], d2N ) 3 (18)

Figure 14. Variation of optimal solution with θ2 and θ3 for a fixedvalue of θ1 ) 0.16.

Figure 15. Comparison of HDMR prediction with actual solutionfor example 2: θ1 fixed at 0.16, θ2 fixed at (a) 0.8 and (b) 3.68.


that the HDMR performs with great accuracy near themean values of the uncertain parameters because theapproximating functions yield exact information alongthe cut lines and surfaces. However, its performance candeteriorate toward the boundary, as seen in plot b ofFigure 14, where the information corresponds to anapproximation.

To address the design under parametric uncertaintyproblem, the above analysis is performed for differentvalues of the design variables in the range [1, 5] toobtain the variation of the optimal solution z as afunction of θ1, θ2, θ3, d1, and d2. Second-order HDMR isfound to predict the function with an error value of 7.39× 10-2. Figure 17 illustrates the variation of z with d1and d2 for θ1, θ2, and θ3 fixed at their nominal valuesin plot a and at 0.16 each in plot b and compares theHDMR estimation with the actual solution. Note thatall of the above figures present just a few instances ofthe obtained solution to illustrate the performance ofthe proposed approach. The information obtained fromthe application of the proposed technique is a completedescription of the optimal parametric solution withrespect to the uncertain parameters and design vari-

ables. No additional runs are required to evaluate theperformance of any prospective design at any point ofuncertain input.

5. Discussion and Future Direction

A new approach is introduced in this paper for theuncertainty analysis in design/planning problems. Theapproach involves the determination of the design’sfeasible region through the generation of an input-output mapping where the output corresponds to thedesign’s feasibility. Following the evaluation of thedesign’s feasibility, the economic optimization problemis solved within the feasible parameter range to capturethe variation of the design objective with respect touncertain parameters [P(θ)]. The procedure iterateswith design as a new set of variables to obtain thecomplete parametric information of value of the objec-tive with respect to the design and uncertain param-eters [P(d,θ)]. The high-dimensional model representa-tion (HDMR) approach is used to capture the input-output behavior. The HDMR approach treats the modelequations as a black box, and given a set of modeloutputs for input parameter values, it can estimate theoutput behavior for the entire input parameter space.The number of model runs required to determine theHDMR expansion depends on the number of uncertainparameters (n) and the number of sampling grid pointsof each parameter (s). Determination of the first-orderexpansion function requires n(s - 1) model runs, andthe second-order function requires n(n - 1)(s - 1)2/2model runs. In the present study, each uncertainparameter is sampled at 11 points, including its nominalvalue. This results in 21 model runs for first-order and121 runs for second-order HDMR estimation for the two-parameter case. This technique is applied in differentconvex and nonconvex problems in all of which second-order HDMR estimation is found to provide excellentagreement with the actual model solution. Although noreal case studies are considered, as this is out of thescope of this paper, an interested reader should realizethat (a) implementation of the presented approach isstraightforward as the only input required is a set ofmodel runs at different values of the uncertain param-eters and (b) the proposed approach overcomes theshortcomings of previous uncertainty analysis frame-works where the model should be known and shouldfollow certain convexity assumptions.

Work is in progress toward the extension of thepresented ideas to incorporate models involving binaryvariables, thus addressing the parametric synthesisproblem. Furthermore, the consideration of dynamicmodels will also be exploited in future publications.

Acknowledgment

The authors gratefully acknowledge financial supportfrom the National Science Foundation under the NSFCAREER program (CTS-9983406) and from the donorsof the Petroleum Research Fund administered by theACS.

Literature Cited

(1) Swaney, R. E.; Grossmann, I. E. An index of operationalflexibility in chemical process design. Part I. Formulation andtheory. AIChE J. 1985, 31, 621.

Figure 16. Comparison of HDMR prediction with actual solutionfor example 2 with θ1 and θ2 fixed at their mean values.

Figure 17. Variation of optimal solution with design parametersd1 and d2: θ1, θ2, and θ3 fixed at (a) their mean values and (b)0.16 each.


(2) Grossmann, I. E.; Sargent, R. W. H. Optimum design ofchemical plants with uncertain parameters. AIChE J. 1987, 24,1021.

(3) Grossmann, I. E.; Halemane, K. P. A decomposition strategyfor designing flexible chemical plants. AIChE J. 1982, 28, 686.

(4) Saboo, A. K.; Morari, M.; Woodcock, D. C. Design of resilientprocessing plants-VIII. A resilience index for heat exchangernetworks. Chem. Eng. Sci. 1983, 40, 1553.

(5) Kubic, W. L.; Stein, F. P. A theory of design reliability usingprobability and fuzzy sets. AIChE J. 1988, 34, 583.

(6) Pistikopoulos, E. N.; Mazzuchi, T. A. A novel flexibilityanalysis approach for processes with stochastic parameters. Com-put. Chem. Eng. 1990, 14, 991.

(7) Straub, D. A.; Grossmann, I. E. Design optimization ofstochastic flexibility. Comput. Chem. Eng. 1993, 17, 339.

(8) Goyal, V.; Ierapetritou, M. G. Determination of operabilitylimits using simplicial approximation. AIChE J. 2002, manuscriptaccepted.

(9) Takamatsu, T.; Hashimoto, I.; Ohno, H. Optimal design ofa large complex system from the viewpoint of sensitivity analysis.Ind. Eng. Chem. Process Des. Dev. 1970, 9, 368.

(10) Ganesh, N.; Biegler, L.T. A reduced Hessian strategy forsensitivity analysis of optimal flowsheets. AIChE J. 1987, 33, 282.

(11) Gal, T. Linear parametric programmingsA brief survey.Math. Prog. St. 1984, 21, 43.

(12) Jongen, H. T.; Weber, G. W. On parametric nonlinearprogramming. Ann. Op. Res. 1990, 27, 253.

(13) Sahinidis, N. V.; Grossmann, I. E.; Fornari, R. E.; Chath-rathi, M. Optimization model for long-range planning in chemicalindustry. Comput. Chem. Eng. 1989, 9, 1049.

(14) Paules, G. E.; Floudas, C. A. Stochastic programming inprocess synthesis: A two-stage model with MINLP recourse formultiperiod heat-integrated distillation sequences. Comput. Chem.Eng. 1992, 16, 189.

(15) Reinhart H. J.; Rippin, D. W. T. The design of flexible batchchemical plants. Presented at the AIChE Annual Meeting, NewOrleans, LA, Apr 7-8, 1986; Paper No. 50e.

(16) Ierapetritou, M. G.; Acevedo, J.; Pistikopoulos, E. N. Anoptimization approach for process engineering problems underuncertainty. Comput. Chem. Eng. 1996, 20, 703.

(17) Hene, T. S.; Dua, V.; Pistikopoulos, E. N. A hybridparametric/stochastic programming approach for mixed-integernonlinear problems under uncertainty. Ind. Eng. Chem. Res. 2002,41, 67.

(18) Rabitz, H.; Alis, O. General foundations of high dimen-sional model representations. J. Math. Chem. 1999, 25, 197.

(19) Rabitz, H.; Alis, O.; Shorter, J.; Shim, K. Efficient input-output model representations. Comput. Phys. Commun. 1999, 117,11.

(20) Shorter, J.; Ip, P. C.; Rabitz, H. An efficient chemicalkinetics solver using high dimensional model representations. J.Phys. Chem. A 1999, 103 (36), 7192.

(21) Wang, S. W.; Levy, H., II; Li, G.; Rabitz, H. Fullyequivalent operational models for atmospheric chemical kineticswithin global chemistry-transport models. J. Geophys. Res. 1999,104 (D23), 417.

(22) Halemane, K. P.; Grossmann, I. E. Optimal process designunder uncertainty. AIChE J. 1983, 29, 425.

(23) Ierapetritou, M. G. A new approach for quantifying processfeasibility: Convex and 1-D quasi-convex regions. AIChE J. 2001,47, 1407.

(24) Li, G.; Rosenthal, C.; Rabitz, H. High dimensional modelrepresentation. J. Phys. Chem. A 2001, 105, 7765.

Received for review April 11, 2002Revised manuscript received September 26, 2002

Accepted October 10, 2002

IE0202726


design optimization under parameter uncertainty for general black-box models

Documents