526 A GLOBAL OPTIMIZATION ALGORITHM

USING STOCHASTICDIFFERENTIAL EQURTIONS..(U) WISCONSIN UNIV-MADISONMATHEMATICS RESEARCH CENTER F ALUFFI-PENTINI ET AL.

UNcLAssIFIED FEB 85 MRC/TSR-2791 DAAG29-88-C-8041 F/G 9/2

EEEEEEllIEhlE~lllllEEEEIEEE.IIIII.

p..L1.

1 11111 .111111.

MICROCO)PY RESOLUIION TESI CHARI

NAHfONAL !HU!HAU 01 ,IANDARV> AN

0

MRC Technical Summary Report #2791

Ln A GLOBAL OPTIMIZATION ALGORITHM USING

Ln STOCHASTIC DIFFERENTIAL EQUATIONS

'~ Filippo Aluffi-Pentini, Valerio Parisi. and Francesco Zirilli .

I-

Mathematics Research CenterUniversity of Wisconsin-Madison610 Walnut StreetMadison, Wisconsin 53705

February 1985

(Received December 13, 1984)

Gum - Approved for public release

".W( i GOI~N Distribution unlimited fc

Sponsored byMAY 9 1985

U. S. Army Research Office DP. 0. Box 12211

Research Triangle ParkNorth Carolina 27709 D

. ".- .. .. - j :* . • .- - -.8 5

9L

UNIVERSITY OF WISCONSIN-MADISON

MATHEMATICS RESEARCH CENTER

A GLOBAL OPTIMIZATION ALGORITHM USING

STOCHASTIC DIFFERENTIAL EQUATIONS

Filippo Aluffi-Pentini , Valerio Parisi2 , and Francesco Zirilli3

Technical Summary Report #2791February 1985

ABSTRACT

SIGMA is a set of FORTRAN subprograms for solving the global optimization

problem, which implement a method founded on the numerical solution of a

Cauchy problem for stochastic differential equations inspired by quantum

physics.

This paper gives a detailed description of the method as implemented in

SIGMA, and reports on the numerical tests which have been performed while the

SIGMA package is described in the accompanying Algorithm.

The main conclusions are that SIGMA performs very well on several hard

test problems; unfortunately given the state of the mathematical software for

global optimization it has not been possible to make conclusive comparisons

with other packages.

AMS (MOS) Subject Classifications: 65K10, 60H10, 49D25

Key Words: Algorithms, Theory, Verification, Global Optimization,Stochastic Differential Equations

Work Unit Number 5 (Optimization and Large Scale Systems)

6Dipartimento di Matematica, Universita di Bari, 70125 Bari (Italy).2Istituto di Fisica, 2 UniversitA di Roma "Tor Vergata", Via Orazio Raimondo,

00173 (La Romanina) Roma (Italy).3Istituto di Matematica, UniversitA di Salerno, 84100 Salerno (Italy).

Sponsored by the United States Army under Contract No. DAAG29-80-C-0041 andthe U. S. Govein:.-ent through its European Research Office of the U. S. Armyunder Contract n. DAJA-37-81-C-0740 with the University of Camerino, Italy.

*. " , , ; , " .. . - K * ., . . .... .- " * "" ." < . .'' . " ". ". " "

SlGNI ICANCE AN FXPLA, ,L',

The paper reports about a new and very successful method for finding a"global" (or "absolute", o a :anctii of N reai variables, i.e. thepoint x in N-dimensional S; ,Ic .'"C Pnints, such that notonly the function increaies - )ic mcves iway tc. x in any direction,"local" or "relative" niun4 -. ), but ,Jsc such that no ot.'ei poi.,t exists

where f has a lower valuz.

The method, which was -r<t- -. sed by th Presen -uthors in a paperwLich is to appear in tho Jo!'rnal -f OptJmizat-.7,n Theory and Applications, ishasd on ideas from statistira! mechanirs, an looks f-: a point of globalrninimurvi by following the solution trajectorias of a stc-hastic differentialequation representing the motion of = particlc fin N-space) under the action

of a potential field and of a random pert,,:rbin3 foro.

The paper gives a detaila! description of the ;ompl-z:le algorithm based on

such a method, and summnnariTP- the results of extensive rumerical testing ofthe FORTRAN program implementing the alqorithm 'the FORTRAN program isdescribed in a companion pap-r of the same authors: Algorithm SIGMA. AStochastic-Integration Global Minirizticn Algorithm).

The tests have been performed by running the program on an extensive setaf iaefuily selected test problems of varying difficulty, and the performance

ra been remarkably successful. even on very hard problems (e.g. problems withA sngle point, of global minimum and up to about 1010 points of localminimum.)

The method is now being successfully tested on some real-world problemsin appiied chemistry, concerning the analysis of complex molecules, where one

k fc spatial patterns which are rot on],, stable (local minima of

potc-,,tiai energy), but have also an absolute minimum of the potential energy.

Mc e generally there are many prnb~ems in which the solution depends onthe values of several parameters, and the quality of the solution can bemeasurea by a single "performance figure" (which is therefore a function of

t;h .. r ar.eters), e.g. a cost, or a loss, or a cost/effectiveness ratio, whichbe low, or a gain, a utility, which should be high.

'r *u sitLations the method can be usefully applied if one is notsaticfied by finding a "sub-optimal" solution, i.e. a solution which is thebest amorg many other solutions, but one requires a truly optimal solution,i.e. the best among all possible solutions.

-t L fi-nally to be noted that thp majority of the optimization methods*,. ru'-.',, -vilable deal with the local ontimization problem, and that no

-.f comparable power seem to he available in the field of global

T.r, rcrsiblity for the wording and views expressed in this descriptive

Sles with MPC, and not with the aothors of this re.port.

....6.

- ,-"- .. ...

A CLOBAL OPTIMIZATION ALLDRITHM USINGSIOCASIC DIFFERENTIAL EQUATIONS

Filippo Aluffi-Pentini, Valerio Parisi2 , and Francesco Zirilli3

i. Introduction.

In (1] a method for solving the global optimization problem was

proposed. The method associates a stochastic differential equation with

the function whose global minimizer we are looking for.

The stochastic differential equation is a stochastic perturbation

of a "steepest descent" ordinary differential equation and is inspired

by quantum physics. In (i] the problem of the numerical integration of

the stochastic equations introduced was considered and a suitable "stochas-

tic" variation of the Euler method was suggested.

SIa'iA is a set of FORTRAN subprograms implementing the above

method.

In sect. 2 we describe the method as implemented in SiLi1; in sect.

3 we give a general description of the method and some details on the

implementation; in sect. 4 some numerical experience on test problems is

presented and in sect. 5 conclusions are given.

Unfortunately, given the state of the art of mathematical software

in global optimization, it has not been possible to make conclusive corn-

parisons with other packages.

The SIG4A, package and its usage are described in the accompanying

Algorithm.

lDipartininto di Matematica, Universita di Bari, 70125 Bar (Italy).Istituto di Fisica, 2a UniversitA di Roim '"Tor Vergata", Via Orazio

3Raimado, 00173 (La Rommina) Rom (Italy).Istituto di Matenmtica, Universita di Salerno, 84100 Salerno (Italy).

Soonsored by the United States Armiy under Contract No. DAAG29-80-C-0041 andthe U. S. Covernrmt through its European Research Office of the U. S. Aryunder Contract n. DAJA-37-81-C-0740 with the University of Camerino, Italy.

7 .* * * ** * .*.* . . o" -. - .

-o - - -' ." - - ,; r r r" r r'i ."-r- 7- - - . C . . - i - ' i . . - -' ,. T r J

2

The method.

Let iRN be the N-dimensional real euclidean space and let fNIR

'-,e a ieal valued function, regular enough to justify the following con-

3Sc, rat ions.

In t hi .a.r ,e consider the problem of 'finding a global minimizer

of , that is, h, c point x E (or possible one of the points) such

that

(2.1) f(x ) - f(x) Vx IR N

and we propose a method introduced in [1) inspired by quantum physics to

compute nUleri ily the global minimizers of f by following the paths

of a stochastic differential equation.

The interest of the global optimization problem both in mathematics

and in many applications is well known and will not be discussed here.

We want just to remark here that the root-finding problem for the

N N0 .system .E(x) = 0, where g R R can be formulated as a global optimi-

zatiorn problem considering the function F(x) yig(x)II here

is the euclidean norm in R

Despite its importance and the efforts of many researchers the

global optimization problem is still rather open and there is a need for

methnds .ith solid mathematical fotmdation and good numerical performance.

The present authors have considered this idea both from the mathematicalocint of vie,, (for a reviei,, see [21) and from the point of view of producing5 ood software (see [3], [4]). The method implemented in [3), [4) is in-* pirred by classical mechanics, uses ordinary differential equations, and canoe regarded as a method for global optimization.

. - .-- .-- -

3

-"ch more sat-sfactcrv is the situation for the problem of finding

the !cca3 minL.i',izers of f, ,hvrt a large body of theoretical and

., ... results txlsts; se- for instance [5], [6) and the references<iv,,--therein.

.- ra.'.irv differential equations have been used in the study of the

o.p-nization problem or of Lhe root finding problem by several

" atnj:s; for a review see [2].

.. .- ,.e methods usually obtain the local minimizers or roots by

. the traiectories of suitable ordinary differential equations.

Howe-er, sLnce the property (2.1) of being a global minimizer is a glo-

b o-, that is, depends on the behaviour of f at each point of RN

,no <e :xethods that follow a trajectory of an ordinary differential

equation are local, that is, they depend only on the behaviour of f

,, ~ the trajectory, there is no hope of building a completely satis-

1:.cory ~mthcd for global optimization hased on ordinary differential

-quat ions.

Ine situ-aticn i-s different if we consider a suitable stochastic

:u>;urbaticn of an ordinar U' differential equation as explained in the

Let us firit cnside r --he (I c) t hasti,_ differential equation

* .- ,-) ' , ': C.

. thc g.: iL c f 2::d ,' ) is a standard N-2 Lmensional

at- 2. a 111- .Z'n ste mchoki-,Kraners equation [7];

s qu-tlcn I-, s.,ngIor Iiji,, of T Langevin's equation when the

- S

4

The Smoluchowski-Knamers equation has been extensively used by

solid state physicists and chemists to study physical phenomena such

as atomic diffusion in crystals or chemical reactions.

In these applications (2.2) represents diffusion across potential

barriers under the stochastic forces cdw, where = /T , T is_ m- -

the absolute temperature, k the Boltzmann constant, m a suitable mass

coefficient, and f is the potential energy.

We assume that

(2.3) lim f(x) +lix 112

in such a way that:

(.)jr e- If(X) dx < oo (Je R\{0})"(2.4) ! e _ fVx

and that the minimizers of f are isolated and non degenerate.

It is well known that if _E(t) is the solution process of (2.2)

" starting from an initial point x0, the probability density function

p (t,x) of _ (t) approaches as t - o the limit density pc(x)

where

2

(2.5) p ( ) = A e f(X)

where A is a normalization constant. The way in which pE(t,x) forC

a class of one-di:ensional systems approaches pC(x) has been studied

in detail by considering the spectrun of the corresponding Fokker-Planck

operators in [8].

• . -, i - - ,. . . . .

We note that p is independent of x0 and that as c - 0 pc

becomes more concentrated at the global minimizers of f. That is,

(2.6) lir Ct) - in law

where c has a probability density given by (2.5) and

C1

(2.7) lirn c in lawC£40

where t is a random variable having as its probability density a

weighted sum of Dirac's deltas concentrated at the global minimizers of f.

For example if N = 1 and f has two global minimizers x1, xI , with

d2f"xr (xi ) = ci > 0, i = 1,2, we have (in distribution sense)

(2.8) lim p.(x) = y 6(x-x1 ) + (l-y) :(x-x 2 )

where y = (I +VCc/c 2 )-. In order to obtain the global minimizers of f

as as),mptotic values as t - of a sample trajectory of a suitable sys-

tem of stochastic differential equations it seems natural to try to perform

the limit t- (i.e. (2.6)) and the limit E 0 (i.e. (2.7)) together.

That is, we want to consider:

(2.9) d& = -Vf(&)dt + c(t)dw

with initial condition

(2.10) O) =x o

where

(2.11) lim C(t) 0.

- . ." - " - :- . " -" -- - - -. .?

6

In physical terms condition (2.11) means that the temperature T

zs decreased to 0 (absolute zero) when t =, that is, the system is

"frozen".

Since we want to end up in a global minimizer cf f, that is, a

global minimizer of the (potential) energy, the system has to be frozen

very slowly (adiabatically). The x-y in which E(t) must go to zero,

in order to have that when t - =, the solution C(t) of (2.9) becomes

concentrated at the global minimizers of f, depends on f. In par-

ticular, it depends on the highest barrier in f to be overcome to

reach the global minimizers.

This dependence has been studied using the adiabatic perturbation

theory in [1]. Similar ideas in the context of combinatorial optimization

have been introduced by Kirkpatrick, Gelatt, Vecchi in [9].

In this paper we restrict our attention to the numerical implemen-

tations of the previous ideas, that is, the computation of the global

uliin-izers of f by following the paths defined by (2.9), (2.10) . dis-

regarding mathematical problems such as the difference between the con-

verge-nce in law of _(t) to a random variable concentrated at the global

.:ninraers of f, and the convergence with probability one of the paths

f _(t) to the global minimizers of f.

We consider the problem of how to compute numerically these paths

keeping in mind that we are not really interested in the paths, but only

in their asymptotic values.

'e discretize (2.9), (2.10) using the Euler method, that is _tk)

j- approximated by the solution of the following finite difference

equations:

. .. .. .~~~ -1 MW 7 T . . . -- . .. ." VIN . 19 , i 7 ,

7

(2.12) - = -h f + E(tk k+1 " k k = 0,1,2,

(2.13) =

k-I,.here to = 0, tk = i hi, hk > 0, and wk w (tk), k = 0,1,2,....

i=0 k wt)

The coraputationally cheap Euler step seen5 a good choice here

since in order to obtain the global minimizers of f as asymptotic

values of the paths c(t) should go to zero very slowly when t -,

and therefore a large number of time integration steps mast be com-

puted.

On the right hand side of (2.12) we add the random term

(tk to the deterministic term -hk Vf( ), which is com-

putationally more expensive (e.g. N+l function evaluations ii a

forard-difference gradient is used), so that the effort spent in evaluat-

ing 7f(rk) is frequently lost.

In order to avoid this inconvenience we substitute the gradient

Vf(Q) with a "random gradient" as follows. Let r be an N-dimensional

random vector of length 1 uniformly distributed on the N-dimensional unit

sphere. Then for any given (non-random) vector v E IRN its projection

along r is such that:

2.I.') N.E(<r,v>r) = v

•..here E(.) is the expected value, and < .,. > is the euclidean inner

Or:duzt in ,

Sc that in order to save n,-nerical work (i.e. functions evaluations)

:r,12) we substitute 7f(-- with the "random gradient"

8

(2.15) -(_) = N < r, 7f(4) > r.

le note that since . _(_) is the directional derivative in the direc-

-ion r, it is co-putationally much cheaper (e.g. when fon,'ard differences

are used, only 2 function evaluations are needed to approximate y( )).

Therefore, the paths are computed approximating (tk) with the solution

of the following differences equations:

(2.16) E - = -\ -- + S(tk)(k - k) k = 0,1,2,

(2.17) =

.,here j(g) is a finite difference (forward or central) approximation

to

The complete algorithm is described in the next section.

9

3. The complete algorithm.

We give in sect. 3.1 a general description of the algorithm, while

irplementation details are given in sect. 3.2.

3.1. General description of the algorithm.

The basic time-integration step (eq. (2.16) and sect. 3.2.1) is

used to generate a fixed number NTRAJ of trajectories, which start at

time zero from the same initial conditions with possibly different values

of (O) (note that even if the starting values e(O) are equal the

trajectories evolve differently due to the stochastic nature of the inte-

gration steps).

The trajectories evolve (simultaneously but independently) during

an "observation period" having a given duration (sect. 3.2.5), and with-

in which the noise coefficient of each trajectory is Ike.pt at a constant

value e , while the values of the steplength hk and of the spatial

discretization increment Ax k for computing the random gradient (eq.

(2.15)' and sect. 3.2.2) are automatically adjusted for each trajectory

by the algorithm (sects. 3.2.3 and 3.2.4).

At the end of every observation period the corresponding trajec-

tories are compared, one of them is discarded (and will not be considered

any more), all other trajectories are naturally continued in the next

observation period, and one of the trajectories is selected for "branch-

ing" (sect. 3.2.6), that is for generating also a second continuation

trajectory differing from the first one only in the starting values for

£ and Lxk (sect. 3.2.7), and which is considered as having the same

"past history" of the first one.

23

Let X, be the largest eigenvalue of the (symmetric and non-negative

definite) matrix C.

We adopt the updating matrix

FA = X1 - C

where I is the NxN identity matrix, 8 > 1 (S = 1.3 in the present

implementation), and we obtain the updated value A' of A by means of

the formula

A =cFA FA

where a is a normalization factor such that the sum of the squares of

the elements of A' is equal to N (as in the identity matrix).

The matrix FA seems one of the possible reasonable choices,Ao

since it is positive definite for $ > 1, it has the same set of eigen-

vectors as C, its eigenvalue spectrun is obtained from the spectrum of

C by reflection around A = , and it therefore acts in the right

direction to counter the ill-conditioning of f.

The magnitude of the counter-effect depends on B: the adopted

value has been experimentally adjusted.

The updated bias vector b' is chosen in order that the scaling

at x does not alter R, i.e. in order that

A'x + b' = Ax + b.

3.2.13 Criteria for numerical equality.

The following two criteria are used in a nunber of places in the

algorithm to decide if two given numbers x and y are sufficiently

close to each other (within given tolerances) to be considered "nuneri-

cally equal".

rrr-:- w -7

22

,, c~m r (for each trajector.' the rescaled variable = Ax + b,

is the rescaline mat.x and b is a bias vector, and, instead cf

,.l:F) :e with respect to x the function fx) = f(R) = f(Ax + b)

. ° tc couin'r t:C 11-conditioning of f with resnect to x bv

1,U . :iO A . b is aduftted in order not to alter R).

:.,':c1ng of A is obt.ained by means of an updating matrix FA,

... .o at the end of an observation period if sufficient data

• :-u <see below), and if the number of elapsed observation periods

S "than a given nnber K and greater than 7N).: .,. c.._, nn agivn ~nbr pasca,

:: )da-ing matrix tA is computed as described below, keeping in

t. t ti he random giadients are the only simplyLusable data on the

,cisooCr ot t computed by the algorithm.

1,2, ... , N be the column vectors of the components

, he N finite-difference random gradients 7 (.F or _

Koat --d along the trajectory (also for rejected steps) from the last

L-Effcient data are available (i.e. if N > 2N2 ) we compute the

N

N (i)g i=1

,: .-tLtd covar-laice matrix

...... n ,-OCator, given the available data, of

: I conc o:, f i, as having the larger eigenvalues

.Ilatvd ", h the ,ir'ecticns alon which the second directional deriva-

~x~of is, on the average, 1aryer.

21

We note that each integration step can be rejected only a finite

number of times, each observation period lasts a finite number of accepted

integration steps, and there is a finite number of observation periods in

a trial; since a finite number of trials is allowed, the algorithm will

stop after a finite total number of steps and of function evaluations. -J

3.2.11 Admissible region for the x-values.

In order to help the user in trying to prevent computation failures

(e.g. overflow) the present implementation of the method gives the possi-

bility of defining (for any given problem and machine dynamic range, and

based on possible analytical or experimental evidence) an admissible region

for the x-values (hopefully containing the looked-for global minimizer)

within which the function values may be safely compu:ed. We use an

N-dimensional interval

RIN i = 1,2, ... , N,1 1 i '

where the interval boundaries must be given before trial start.

Outside the admissible region the function f(x) is replaced by

an exponentially increasing function, in such a way that the values of

f and of the external function are matched at the boundary of the region.

3.2.12 Scaling.

In order to make ill-conditioned problems more tractable, rescaling

is performed by the algorithm as follows.

. - . . " . . . .

- - -. , : . o- UmI

20

the preceding trial, according to the outcome (stopping condition) of the

preceding trial and to the number t of trials performed from algorithm

start, as compared to the given maximum number of trials NTRIAL

successful stop: a = 103

q

unsuccessful uniform stop:

= 10 if t < [[(2/5) NTRIAL]]

= 10' otherwise,

where [[x]] is the smallest integer not smaller than x

unsuccessful non-uniform stop: a = 10-4

The initial point x0 is selected as follows:

if t <[[(2/5) NTRIAL]] take the value of x0 at algorithm start

otherwise take x0 = OPT

where OPT is the current best minimizer found so far from algorithm

start.

All other initial values are those of the first trial, except the

initial values of h and Ac which are the values reached at the end of

the preceding trial.

3.2.10 Stopping criteria for the algorithm.

The complete algorithm is stopped, at the end of a trial, if a

given number NSUC has been reached of uniform trial stops all at the

current f OPT level, or in any case if a maximum given number NTRIAL

of trials has been reached.

Success is claimed by the algorithm if at least one uniform stop

occurred at the current fOPT level.

. . . . . . . ... .. . . . . . . . . .- -

19

and the best minimum function falue fOPT found so far from algorithm

start: if fTMMIN and fOPT satisfy at least one of the above criteria,

with the same tolerances, the trial is considered successful at the level

fOPT; otherwise the trial is again considered unsuccessful.

Checking of the stopping criteria is activated only if a minimum

given number NPMIN of observation periods has been reached.

3.2.9 Characteristics of the successive trials.

The operating conditions which are changed when another trial is

started are:

seed of the random number generator

- maximum duration of the trial

- policy for choosing c for the second continuation of a branchedp

traj ectory

- value of e at trial startp

- initial point x0 .

The maximum duration of a trial, i.e. the maximum number NpMx

of observation periods, is obtained as follows:

if the preceding trial had a uniform stop (sect. 3.2.8) take the

value of the preceding trial

otherwise take a value obtained by adding to the preceding value

a fixed given increment INPMAX .

The policy for selecting e for the second continuation of aP

branched trajectory was described in sect. 3.2.7.

The value of £ at the start of a new trial is obtained by means

pof a multiplicative updating factor c applied to the starting value of

18

The updating factor F for E is as follows:E p

for the first trial and for any trial following an

unsuccessful trial J

F -10 - where x is a random sample from a

standard normal distribution

for all other trials

F 2y - ,here y is a random sample from a

standard Cauchy distribution, i.e. with density

f (y) = l/(,T (l+y 2))

The updating factor for AXk is:

F = 1 0 3z 1,here z is a random sample from a standard

normal distribution.

3.2.8 Stopping criteria for a trial.

A trial is stopped, at the end of an observation period, and after

having discarded the worst trajectory, if all the final function values

Of the remaining trajectories (possibly at different points x) are

"nu.erically equal", i.e. if the maximum, f and the minimum,

fT IIN' among the trial final values satisfy at least one of the criteria

in sect. 3.2.13, the relative difference criterion with a given stopping

olerance R and/or the absolute difference criterion with given

stopping tolerance i,BS ("uniform stop at the level fTFMJN"

The trial is also anyway stopped, at the end of the observation

pcricd, if a maximum- given number N of observation periods has been' " PMAX-

c.-ched.

in the latter case the trial is considered unsuccessful, while in

the former case a comparison is made between the final value fT IIN

• " - . -. .. . . .-...-.. . .. .-

17

From the point of view of the noise coefficient- c a trajectoryp

*with larger c is considered better if the comparison is made in anp

early observation period (as long as < where k~ is the

number of elapsed observation periods, and I "b are defined below)

and worse otherwise.

A basic partial ordering of the trajectories is first obtained on

the basis of past funrction values, and a final total ordering is then ob-

tained, if needed, by suitably exploiting the noise-based ordering.

The discarded trajectory is always the worst in the ordering, while

the trajectory selected for branching is usually not the best one, to

avoid to be stuck in a non-global minimm.

Normal branching is performed on the trajectory which, in the order-

ing, occup~ies the place Ib (a given integer); exceptional branching,

where the best trajectory is selected, occurs for the first time at the

end of observation period kand then every M periods (k andpo PPOM pare given integers); i.e. exceptional observation periods are those

numbered

k p k po+ jM (j 0, 1,2,..)p po p

3.2.7 The second continuation of a branched trajectory.

Ihi~le the first (unperturbed) continuation of a trajectory thatundergoes branching starts with the current values Of E and 'xthe

second continuation starts with values obtained by means of mnultiplica-

tive random updating factors applied to the current values.

16

In phase 6a: y = 0.1

We finally remark that hk and LXk are bounded by suitable con-

stants to avoid computational failures.

3.2.5 Duration of the observation period.

The duration of observation period numbered k from trial start,

defined as the number Nhp of time integration steps in period k1 ,, is

computed as a function of kp by means of a formula which must be chosen

before algorithm start among the following three fornulas:

1) 1 + [1og 2 (k) ] ("short" duration)

2) Nhp = [kp] ("medium-size" duration)

3) Nhp = kp ("long" duration)

where k = 1,2, ... , and [x] is the largest integer not greater than x.P

3.2.6 Trajectory selection.

In order to decide, at the end of an observation period, which tra-

jectory is to be discarded, and which one should be selected for branch-

ing, we compare the trajectories on the basis of the values of their noise

coefficient in the observation period, and of the function values obtained

from trial start.

From the point of view of past function values a trajectory is con-

sidered better than another if it has attained a lower function value than

*I the other (excluding a possible initial part common to both trajectories).

.o 15

. We test f and f= f for nirnerical equality according

to the relative difference criterion (sect. 3.2.13) with tolerances

T = 10 and TR2= 10-, and take

=2 if fk and fk are "equal" within TRI

i f and f are not "equal" withink k R2

- =1 otherwise.

-11 5The interval (10- , i0) has been adopted since it contains both

the square root and the cubic root of the machine precision of most com-

puters in double precision (the square root is appropriate for forward

differences, while the cubic root is appropriate for central differences).

Updating factors y for hk

In phase 4a:

y = 1/1.05 for the first attempt to the first half-step

Y = I for the second attempt

-= 1/10 for all other attempts

In phase S the value of y depends on the current number a of accepted

time integration steps in the current observation period, and on the cur-

rent total number r of half-steps rejected so far in the current trial

(excluding those possible rejected while attempting the first step).

If r>0

y = 1 (if a 4 2r)

Y = 1.1 (if 2r < a 4 3r)

y 2 Z (if 3r < a)

If r = 0

y = 2 (if a 1)

= 10 (if a > 1)

°* *. . . * . .4 . . . . . . .

14

b, If the half-step is rejected: reject also the first half-step,

update (decrease) hk, and go back to 1.

irh'. Otherwise: accept the whole step and try the next one.

Note however that if the same half-step is rejected too many times

the h'alf-step is nevertheless accepted in order-not to stop the algorithm;

this is not too harmful since several trajectories are being computed,

a3,j a "bad" one will be eventually discarded (in the present inplementation

the bound is given explicitly for the first half-step (50 times), and im-

plicitiy for the second half-step (if hk becomes smaller than 10-)).

0k

" 3.2. The updating of hk and Ax k'

The time-integration steplength hk and the spatial discretization

increment for the trajectory under consideration are updated Ahile

performing the integration step, as described in the preceding section.

Updating is always performed by means of a multiplicative updating

factor which is applied to the old value to obtain the new one.

The magnitude of the updating factors, as used in the various

phases of the sequence in the preceding sect. 3.2.3, is as follows:

Updating factors for AXk

In phse 1b: 3 = 106

IT phase 2a: = 10

In phse 4b: E = 10 -

in phase S the value of B depends on the magnitude of the current esti-iated function increment Af =fkI AXk (where ior -Fk as

appropriate), and the function value fk -f(c- .

* 13

All attempts are with the current (i.e. updated) values of hk

* and Ak"

The sequence of attempts is as follows:

1. Pick up a random unit vector rk.

la. Compute the random increment sk (sect. 3.2.2).

lb. If s (and therefore Axk) is too small (i.e. if the square

of the euclidean norm of the difference between the computed

values of + lk and -k is zero, due to the finite arithme-

tics of the machine): update (increase) Axk and go back to la.~F

2. Compute k (eq. (3.2.2.2)).

2a. If the computed value of ( is zero (due to the finiteko

arithmetics): update (increase) 6xk and go back to la.

3. Compute the first half-step with .

Compute 6'fk (eq. (3.2.3.1)).

3a. If Afk < I 1 Lxk

accept the first half-step and jump to 5.

C4. Compute the first half-step with to check the appropriateness

Of hxk.

Compute A'fk (eq. (3.2.3.1)).

4a. If Afk > - j kXk

reject the half-step, update (decrease) hk, and go back to 1.

4b. Otherwise: accept the half-step, and update (decrease) Axk.

S. Update (increase) hk.

Update (decrease) Axk.

6. Compute the second half-step.

Compute -Ifk (eq. (3.2.3.2)).

' U : ,, " : : ", : - " , " , . - 4 :. . ." " _" " " . "" " " " "'''''" ''': L. ' " ",'

12

and the forward- and central-differences random gradients

_F -F -C

We use i or for f (o) in the first half-step as described

in the next section.

3.2.3 Accepting and rejecting the half-steps.

The computation of the first half-step can be attempted with the

-F -Cforward- or central-differences random gradient (1k or k eq. (3.2.2.3))

as described below.

In either case the half-step is accepted or rejected according to

the function increment

(3.2.3.1) A'fk - f( ) f(s)

Since A'fk should be non-positive for a sufficiently small valueof hk the half-step is rejected if A'fk is "numerically positive",

i.e. larger than a given positive small tolerance.

The second half-step is rejected if the corresponding function

increment

(3.2.3.2) Afk f (.k+l ) - f(_)

is positive and too large (greater than 100 02 in the present implemen-P

tation).

The sequence of attempts affects the updating of hk and Lxk as

described below; the amount of the updating is described in sect. 3.2.4.

-6 o .•. ," , .

The basic step (3.2.1.1) is act,3''" perfcrmed in two half-steps

(3.2.1.2) - hk -K) (first half-step)

and

(3.2.1.3) = C "IT Uk " (second half-step)

Both half-steps depend on hk while the first depends also on the

current value Axk of the spatial discretization increment used in com-

puting lnk).

Either half-step can be rejected if deemed not satisfactory, as6

described in sect. 3.2.3.

3.2.2 The finite-differences random gradient.

Given the current value Xk of the spatial discretization incre-

ment for the trajectory under consideration, we consider the random in-

crement vector

where rk is a random sample of a vector uniformly distributed on the

unit sphere in RN , the forward and central differences

(3.2.2.1)

fk = lf(& + 4) - f( -

* the forward- and central-differences directional derivatives

(3.2.2.2) F = Ffk/x = CfX,k k k k

10

The set of simultaneous trajectories is considered as a single trial,

which is stopped as described in sect. 3.2.8, and is repeated a nunber of

t_.rnes with different operating conditions (sect. 3.2.9).

" .The stopping criteria for the complete algorithm are described in

sect. 3.2.10.

The use of an admissible region for the x-values is described in

sect. 3.2.11, scaling is described in sect. 3.2.12, and criteria for

numerical equality in sect. 3.2.13.

.3.'. Implementation details.

3.2.1 The time-integration step.

The basic time-integration step (eq. (2.16)) is used, for the tra-

jectory under consideration, in the form

(3.2.1.1) hk1 = k - E = 0,1,2, ... )

where hk and c are the current values of the steplength and of thek' p

noise coefficient (the noise coefficient has a constant value e through-p

out the current observation period (sect 3.1)); uk is a random vector

- sample from an N-dimensional standard Gaussian distribution, and

due to the properties of the Wiener process.

The computation of the finite-differences random gradient (.)

is described in the next section.[.

* 24

a) Relative difference criterion

I I-yI (REL (jxj + jyj)/2

b) Absolute difference criterion

Ix-YI < -TAS

where TRL and T AS are given non-negative tolerances.

9 25

4. Numerical Testing.

SIL1 has been numerically tested on a number of test rpoblems run

on two computers. The test problems are described in sect. 4.1, the com-

puters in sect. 4.2 and some numerical results arereported in sect. 4.3.

4.1. Test problems.

The set of test problems is fully described in [10) together -with

the initial points; the test problems are:

1. A fourth order polynomial (N - 1)gI

2. Goldstein sixth order polynomial (N 1 1.)

3. One dimensional penalized Shubert function (N - 1)

4. A fourth order polynomial in two variables (N = 2)

S. A function with a single row of local minima (N 2)

6. Six hump camel function (N - 2)

7. Two dimensional penalized Shubert function = 0 (N = 2)

8. Two dimensional penalized Shubert function B = 0.5 (N = 2)

9. Two dimensional penalized Shubert function 6 = 1 (N = 2)

10. A function with three ill-conditioned minima a = 10 (N = 2)

10. A function with three ill-conditioned minima a 1 100 (N = 2)

12. A function with three ill-conditioned minima a =1 000 (N =2)

13. A function with three ill-conditioned minima a = 10000 (N = 2)

14. A function with three ill-conditioned minLma a = 10' (N = 2)

15. A function with three ill-conditioned minima a = 106 (N = 2)

16. Goldstein-Price function (N = 2)

17. Penalized Branin function (N = 2)

18. Penalized Shekel function M = S (N = 4)

64

26A'."

19. Penalized Shekel function M = 7 (N = 4)

0 Penalized Shekel function M = 10 (N 4)

21 ~Penalized three dimensional Hartman function (N = 3)

72. Penalized six dimensional Hartman function (N = 6)

23. Penalized Levy Montalvo function, type 1 (N = 2)

Penalized Levy Montalvo function, type 1 (N 3)

2S. Penalized Levy Montalvo function, type 1 (N - 4)

2. Penalized Levy Montalvo function, type 2 (N = 5)

2. Penalized Levy Montalvo function, type 2 (N = 8)

28. Penalized Levy Montalvo function, type 2, (N = 10)

29 Penalized Levy Montalvo function, type 3,ran 0 (N 29. Penalized Levy Montalvo function, type 3, range 10 (Ni 0)

30. Penalized Levy Montalvo function, type 3, range 10 (N 4 3)

K. Penalized Levy Montalvo function, type 3, range 10 (N 4)

3 . Penalized Levy Montalvo function, type 3, range 5 (N = 5)

33. Penalized Levy Montalvo ftu-ction, typ~e 3, range 5 (N = 6)I

34. Penalized Levy Montalvo function, type 3, range 5 (N = 7)

35. A function with a cusp shaped minima (N 5)

)6. A function with a global minimum having a small region

of attraction a = 100 (N = 2)

77. A function with a global minimum having a small region

of attraction a = 10 (N = 5)

We used the above functions, and the standard initial points as

K:. re coded in the subroutines GLWTF and GLQ(MIP, which are avilable

p. " 1

',S 40_

27

4.2. Test computers.

We considered two typical machines of "large" and "small" dynamic

range, that is, with 11 and 8 bits for the exponent (biased or signed)

of double precision numbers, and corresponding dynamic range of about+308 3 0

10 and 0 . The tests were actually performed on:

- UNIVAC 1100/82 with EXEC8 operating system and FORTRAN (ASCII)

computer (level 10RI) ("large" dynamic range)

- D.E.C. VAX 11/750 with VNIS operating system (vers. 3.0)

and FORTRAN compiler (vers. 3) ("small" dynamic range)

4.3. Numerical results.

Numerical results of running SIGAA on the above problems and on the

above machines are described below. All results were obtained under the

following operating conditions.

The easy-to-use driver subroutine SIGMAl (described in the accompany-

ing algorithm) was used, with N SUC ze 1,2,3,4,5. All numerical values used .

for the parameters are set in the driver SIGAl and in the other subroutines

which are described in the accompanying Algorithm.

All numerical results are reported on Tables 1, 2, and 3. Table 1

reports some performance data (i.e. output indicator IOUT and number of

functions evaluations) as obtained from SIG4A output for each of the 37

test problems and for the testing both on the "large" and "small" dynamic

range machines. In order to evaluate the performance of SICNA we consider

all the cases in which the program claimed a success (output indicator

IOUT > 0) or a failure (lOUT < 0) and - by comparing the final point

.1

i:::i,'- : :: ..:::: :: : ::. :: " : - .::. ..- • - . ' , •' - "

0e 28

with the knoirn solutions -we identify the cases in which such a claim

is clearly incorrect (i.e. success claim when the final point is not even

approximately close to the known solution, or failure claim when the final

point is practically coincident with the known solution). It is also

O meaningful to consider all the cases in which a computational failure

due to overflow actually occurrs at any point of the iteration.

Table 2 and Table 3 report for each problem and summarized for all

4 problems data concerning thereffectiveness, dependability and robustness

- in the form of total numbers of correctly claimed successes, correctly

claimed failures, incorrect success or failure claims and total number of

overflows - for the two machines.

* 29

*1D 0 00D 0 0D CD 0D 0D a 0D 0 C 0; 0D 0D 0: 0 0D al

-4 Ln r-I tr-I 0 UO '.C --q LA MO 3 LA O 3n44Ln 0%. C'j t' cN LJn Ln Ln f-4 0D Ln n Lo)N Ln N C '4 'LA Nc 0 . '0t 0'. CO O ( It LAi 0 nA nA a 'n A~t

-4N - T w ON C* 4 '0 '.0 r- t) N- CO LAO C-: N'0

A ' 0 0%0 00 00 CD 00 r) 0 0 0 0 0 0 = 0', 0 0 0

00 Q C) (D C) CD CN N CD C) CD D CD N D D QA C CO LA C) CD

r, r-4 00 00 t4 CO -c~ No LAO -4 ftA " cc 31 N- N C cc co3LA O - 0 rl CL N %i0 N- r- 0 tn 00 t-n Ln 00 " v..4 0 V-O C~-)

m Ln N- C) m~ 0) 0 m' N- 1-4 tAo N, CC C) 00 '.0 '.0vtA rA ULA '.0 0) LA LM LA -tr0~N AN'0 O NN' tn CO

N" rA~0 O t -4 0C) C14 Ln " N- -0 WOCO'0 Otn .0O C) 0 0 '.0 LAO - t " A Ol 00 C 0 N 0 * C 00 N- LA ,-4 N n .-q

tAO C7 0 C7 O oo CO N WtA C LA; '.0C7 0 .-4 .4 r1 NO 1U-iN C14 ton tn r-4 * -c~ N- ,- " ,--i4 N Ln CD tn %.0 f,4 tn

N l r -4 CO -4" tA '0% U*) LA -4 ' .0 CD C) 00 %.= CI O coO1.30 C) V, cN . LA tA N 0 tA 00 CLA tAn 3. N LA Ln cc N 00

N ' LA N CON 11 NO, '. AN .i . LAI \.0 LA LAO LA -

1-4 C) N- LA tn co VI N tn .0 cc 0- 0 C- '.C, \0 C- N0-4 N-- tn -RZ co F- -4 1- N 1 C) -4 ~t ,-

cc I! C D r O LA 00 .D0 r-4 C) LA -4 N r-4 V C)O LA 3o 0t'.4 CO L V) M~ WO MW CO MO M~ M ) ON c "O t- Ln LO -4 NWC* L '. L AI r.0 C.0 r-0 r-4 LA) r- N N N- C) Nco

z. .

W) tA cc C-3N LA '.0 tA tAn ,:I N- -o N .0 t"j ~ ' ~-4 tA 4

Z .- - -4 r-4 cN tN 3 N l N -, N. N li ~ ~ t .

Z ~~- -4 -- -4 r4 rN 4

* 30

1-1 00 00 00 0

Cn 1:, Lr0 r- Ln cc r,~~~~~~~L -- NJ r-4 ~ .

C L~ (-. '0 ~ ) C L) c)J C: N-

n -ti r-0 n r

L/) c tn - -4 Ll C) CD 0) "- r, \.o

C- C C) C:) C) D 0 C) 0) '

7.~ r- le) or 00 C N- f 4 r1 0 Lr n0 C LO ) r14 - . i' LI) 0 0~ r- N- 0 0

0: r 00000 00 0000cor4 14-

Lfl cc r- 00 L* 01 N- (J N O t.,) LI)LI 'I) NJ 0' - *-4 ri) LO N -0

N- r- 0? - L I) t N-) LM r00 0o 4 t,) q qctN

-n VI n 0 00 00 0: 00

r0 t4) N- N r- C: ~ L) LI) n N-4 -. 4

-4 '0 C) II) :') '0 (m o 0 c)(N

r 1, r n ,:I Li -4 00 )~ LI) . 0'j" -O 00 r - \O r) -4 ,-1 r4 C7 W -) t

* ('I) LI (NJ Ln -4 t O ~ L)n J C

c. -r t'n 'N 0 rf) tl '-4 0- t') I I .0

-- I) U') -D r-4 LI) L -4l tI)

Z ~ ~ r t-) -n "n) tn 0 J I) 'r I . - f N

0 31

m. I 3 O~C LA '3 Ln r-q 0 0.- Ln C) N NLn Ln N1 Pp l C ? 0 0 R~ N -4 m~ '3 N~ '0 . c~ N= r- 0on to rN tn O N " -, co LA 'D 00 r- t, 'n - co 'o t.0

C CN Q3 0 C) CD O> C (- CD C 0 CDC C CD CD ND CD 'D

c r4 \0 0 r M c n o "t- -I t.4 0. N" Nco 0

N -4 CD CA r- r- 'n 00 coC1 0 t 0 CD N cc LAT N 0r-) N O 00 ' O LA t N " \0 "A "O " " t-I O N cn \M 0. r- c .

-4 N N O O'~~ N O ,-4 Ln N O C N -04 O ~ CO N ~ L

C) N L Q Q CD) =A A N Q3 D N D N CDN CD, cA' 0 r CD 00 C>

r- L - t-n -7 N1 \0 c --I W n4

-o r.o) N, W '3 N- -4 L A t LA e4 -1 -4 -4 LA L O N ~ N

cu'~ N O A N C) L N Q3 0 ND C Q. CL 0D

~~~~L 0- COq '3f,4~,4 ~ O L N N ~ ~ ~ ' .4

pt Nm COz COt N- -d -c-4 w w- M C- Nn .-4 N4 -4 m

C LN N tn ) '3r-0 COr tO LW O) M C) C) IA le '3 NC 4 tN W3 tALi kl Ln -4 m 0 ~ OL A m ?') CO r-4 m LAn LD -4

'3DL a~ N NO tA C 0~ C) C) 0 CD CA c N C~ CD '3 a)

'3 (14 \ CI n qr Il '3 r LA 4 C) N -- f C -4 tO m '3 r -4T N C-- 4 z L -4 Ln o n C en r-4 C14 fl r- " u-4~ - t*, - r- te

- ~ ~ I vt ON 00 c tA R3'0 'r LA tL

32

Cl C) CDC Q 0 C 0 C

D 0000000000"CDOA0 00 c

Lni

0 Ni Ct N CNL L 9. NO 0 0 LnC> cl LM C') qr r- C- Or- 0o %C '0 %

c ~ " nr- r- tn M~ r-.4O m 00 o00 m,-4 L t - r-4 rn tnl tn N VI N~

C) ~ ~ ~ ~ ~ ~ ~ C C)C l0 CD)C 0 C

4.

u Liw00 Ln w0 LA 0 Ln Ln ti-4 - 0 - (n C 0

N ) "~ -4 m~ '-4 rn wA w- 0000 m0 a,4 'r- Ln 01 r- 0D 144 0a LA V)O 0D D le O Ln

Oa m '%0 CD .v -4 N1 LA)-r ~ C ~Lf 0 0)O0: Nq t" -4 Ln tAO C>Irc 00 (D0 NLn rN-

C) )

4J 49

C) I,- LAOa N le I-- tAO 0 - Ln Ln LM r-4:3 ~ ~ %0 w - m' Ln Ln N1 w0 '00w LA) t' Ln

z)0 - 14 0 n w0 w~ .- o m00 to N to) m z00 Ln %00 -4 C) r-4 v 00 LA r- r- N* N4

CD0 CD N D 00D ND '0D00 CD 0' CD Q

V ) EC) Ln C'q CD mA LA I' wA w0 w- N R m Nt

r-. Nn tAI v-Ln r4 .,. r- D 0, - t1

C) 4 14 . n C L)C o r )o n r4t*- 0 0

0 '0) C> C) C '0 ND Cl ON 0 0 N- CD

LD 'l O V 0 to -4 -4 N n tA . L Vt-4) 0) IA\o ., D = t) 4-r

1-4 Cf)

-44

0 V-4

N n \0 N- 00 m~ 0l N- " A r4 LA '0 N-N N N~~r Nn NtA el~At t~A t .. 0)

33

TABLE 2

UNIVAC

NSU C 2 3 4 5

NPROB N

1 1 1 1 1 1

2 1 1 1 1 1 13 1 1 1 1 1 1

4 2 1 1 1 1 15 2 1 1 1 1 1

6 2 1 1 1 1 1

7 2 1 1 1 1 1

8 2 3 1 1 1 1

9 2 3 3 1 ! 1

10 2 1 1 1 1 1

11 2 1 1 1 1 1

12 2 1 1 1 1 1

13 2 1 1 1 1 1

14 2 1 1 1 1 1

15 2 1 1 1 1 1

16 2 1 1 1 1 117 2 1 1 1 1 1

18 4 3 3 1 1 1

19 4 3 1 1 1 1

20 4 3 1 1 1 .

21 3 1 1 1 1

22- 6 1 1 1 1 1

23 2 1 1 1 1 1

24 3 1 1 1 1

25 4 1 1 1 1 1

26 5 1 1 1 1

27 8 3 3 3 1 1

28 10 1 1 1 1 1

34

Table 2 (continued)

LNIVAC (continued)

1 2 3 4 5

1.f 1 i 1 1

.,3 1 1 1 1

4 3 1 1 1 1

3 1 1 1 1- i1 1 1 1

.:73 1 1 1 1

1-! 1 1 1

S'3 3 3 3 3

-"3 3 3 3 3

S) - •. '- " "- . .. .- - - - - -S-- -

35

Table 2 (continued)

VAX

NSU C 1 2 3 4 S

N PROB N

1 1 1 1 1 1

2 1 1 1 1 1 13 1 1 1 1 1 1

4 2 1 1 1 1 1

s 2 1 1 1 1 1

6 2 1 1 1 1 1

7 2 1 1 1 1 1

8 2 3 3 3. 3 1

9 2 1 1 1 1 1

10 2 1 1 1 1 1II 2 1 1 1 1 1

12 2 1 1 1 1 1

13 2 1 1 1 1 1

14 2 1 1 1 1 1

15 2 1 1 1 1 1

16 2 1 1 1 1 1

17 2 1 1 1 1 1

18 4 3 1 1 1 1

19 4 1 1 1 1 1

20 4 1 1 1 1 1

21 3 3 11 1

22- 6 1 1 1 1 1

23 2 1 1 1 1 1

24 3 1 1 1 1254 1 1 1 1 1

26 5 1 1 1 1 1

8 1 1 1 1 1'8 10 1 1 1 1 1

36

Table 2 (continued)

VAX (continued)

NSUC 1 2 3 4 5

\PROB N

29 2 1 1 1 1 1

30 3 1 1 1 1 1

31 4 1 1 1 1 1

32 5 1 1 1 1 1

33 6 1 1 1 1 1

34 7 1 1 1 1 1

35 5 1 1 1 1 1

36 2 3 3 3 3 3

37 5 3 3 3 3 3

1 = success correctly claimed

Z failure correctly claimed

3 = incorrect claim

4 = overflow

.:~a~- .t ... .7. -. . . -" -- -r . -- - - -

7- W.

37

L' LM~0

-J-

0D -4 0> --4

-4 0

u4 u

.0 0 - $ -

- .. • .rr -v

38

- Conclusions.

The SIGMA package piesented here seems to perform quite well on

cup.,osC test problems.

.s it is shon in [101 some of the test problems are very hard;

-..: ., Problem 28 (N = 10) has a single global minimizer and a

.r ' ,,i 'al m nimiers of order 10"o in the region Ixil < 10

10. ..

"- 2 shows that from the point of view of the effectiveness as

. y the n'a-ber of correctly claimed successes the performance

.... i". s very satisfactory; moreover, it is remarkably machine inde-

eie.. ~(note that completely different pseudo-random numbers sequences

.91 V Z e La e by he algorithm on the two test machines). The results of

Table 2 also suggest that the performance of SIGAA is very satisfactory

"f ~ the point of view of dependability (only 2 incorrect claims on the

'-,... dynamic range machine when NS~ > 3 and on the "small" dynamic

c.::ge nachine when NSUc > 4) and robustness (no overflows on bothm:h n e s"

Unfortumately, given the state of the art on mathematical software

4loptimization, it has not been possible to make conclusive com-

si Jsons with other packages.

Finally, we note that a smaller value of NSUC gives a much cheaper

.._-tho, (less function evaluations) at the expense of a loss in effective-

, txa tcr numit,er of failures).

-. -- .9 . * . .. -- * . . . * . . .

r - ~~~- - 7-. . * * - **.

4 39 -

ACKNOWEDEMT: One of us (P.Z.) gratefully ackniowledges the hospitality

and the support of the Mathematics Research Center of the University of

Wisconsin and of the Department of Mathematical Sciences of Rice University

where part of this work was done, and Prof. A. Rinooy Kan for bringing toJour attention ref. [9).j

.4q

40.4

REFERENCES

_j

[1] Aluffi-Pentini F., Parisi V., Zirilli F.: "Global optimization

and stochastic differential equations," submitted to Journal Opti-

mization Theory and Applications.

(2] Zirilli F.: 'he use of ordinary differential equations in the

solution of nonlinear systems of equations," in "Nonlinear Optimi-

zation 1981," M.J.D. Powell, Editor, Academic Press, London, 1982,

39-47.

[3] Aluffi-Pentini F., Parisi V., Zirilli F.: "A differential

equations algorithm for nonlinear equations," ACM Transactions

on Mathematical Software, 10, (1984), 299-316.

[4] Aluffi-Pentini F., Parisi V., Zirilli F.: "Algorithm 617

DAFNE. A differential equations algorithm for nonlinear equa-

tions," AC Transactions on Mathematical Software, 10, (1984),

317-324.

[5] Powell M.J.D. (Editor): 'Nonlinear Optimization 1981," Academic

Press, London, 1982.

[6] Dennis J. E., Schnabel R. B.: 'Numerical methods for uncon-

strained optimization and nonlinear equations," Prentice Hall,

Inc., London, 1983.

- [7] Schuss Z.: "Theory and applications of stochastic cl 7 Cerential

equations," J. Wiley & Sons, New York, 1980, chapter 8.

[8] Angeletti A., Castagnari C., Zirilli F.: "Asymptotic eigen-

value degeneracy for a class of one dimensional Fokker-Planck

*: operators," to appear in J. of Math. Phys.

-. .,..-

* 41

- [9] Kirkpatric S., Gelatt J'. C.D., Vecchi M.P.: "Optimization by

simulated annealing," Science 220 , (1983), 671-680.

[10] Aluffi-Pentini F., Parisi V., Zirilli F.: 'Test problems for

global optimization software," submitted to A.C..1. Transactions

on Mathematical Software.

n

.j

S!

"

S '

S SECURITY LLASSIFICATION OF THIS PAGE (When Data Entered)

REPORT DOCUMENTATION PAGE READ INSTRUCTIONSBEFORE COMPLETING FORM

.4. REPORT NUMBER 2. GOVT ACCESSION NO. OARECI NT'S CATALOG NUMBER

2791-4. TITLE (and Subtitle) S. TYPE OF REPORT & PERIOD COVERED

Summary Report - no specificA GLOBAL OPTIMIZATION ALGORITHM USING reporting periodSTOCHASTIC DIFFERENTIAL EQUATIONS s. PERFORMING ORG. REPORT NUMBER

7. AUTHOR(@) S. CONTRACT OR GRANT NUMBER(e)

Filippo Aluffi-Pentini, Valerio Parisi DAAGZ9-80-C-0041and Francesco Zirilli DAJA-37-81-C-0740

S. PERFORMING ORGANIZATION NAME AND ADDRESS 10. PROGRAM ELEMENT, PROJECT, TASK

AREA & WORK UNIT NUMBERSMathematics Research Center, University of Work Unit Number 5 -

610 Walnut Street Wisconsin Optimization and Large

Madison, Wisconsin 53706 Scale Systems11. CONTROLLIMG OFFICE NAME AND ADDRESS 12. REPORT DATE

U. S. Army Research Office February 1985

P.O. Box 12211 13. NUMBER OF PAGES

Research Triangle Park, North Carolina 27709 4114. MONITORING AGENCY NAME & AODRESS(Ifdiffiait fro Controeind Office) IS. SECURITY CLASS. (of this Ppor)

UNCLASSIFIEDISa. DECL ASSIFICATION/DOWNGRADING

SCHEDULE

IS. DISTRIBUTION STATEMENT (of this Report)

Approved for public release; distribution unlimited. A Foi D:TIS GRA.il

; " i Unnnnounced/ [

17 DISTRIBUTION STATEMENT (of the abstract antored In Block 20, ii dilfoeant from Report)

_o By,

* Equations

20. ABSTRACT (Continue an reverse aide If necosearr and identify by block number)

SIGMA is a set of FORTRAN subprograms for solving the global optimization

problem, which implement a method founded on the numerical solution of a

Cauchy problem for stochastic differential equations inspired by quantum

physics.

(continued)

DD ,JAl 1473 EDITION OF I NOVSS IS OBSOLETE UNCLASSIFIED

- SECURITY CLASSIFICATION Of THIS PAGE (B'wen Date Entered)

H- ABSTRACT (cont.)

This paper gives a detailed description of the method as implemented in

SIGMA, and reports on the numerical tests whic." have been performed while the

SIGMA package is described in the accompanying Algorithm.

The main conclusions are that SIGMA performs very well on several hard

test problems; unfortunately given the state of the mathematical software for

cjlobal optimization it has not been possible to make conclusive comparisons

with other packaqes.

.q

FILMED

*. 6-85

* DTIC