Reprinted £ran KJNnll.Y WEATHER ~, vol. 96, tt>. 2, Feb. 1968, pp. 99-104.

CORRESPONDENCE

If the left-hand side of (1) or (3) is denoted r(CI), aresidual, RN, may be defined

BN= r(-")-Lu., (')

Comments on liThe Over.Rela~atlon Factor In theNumerical Solution of the Omega Equation"

JAMES J. O'BRIEN

Notional c."ter for Atmospheric Research, Boulder, Colo. where N is the number of 8C&D8 in the iteration procedure.This notation is misleading since B is dependent, inactual application, on Nand N + 1, &8 we shall see.

The relaxation technique consists of co~t.ing each (&I

using the relation

(5)--+I=~ +«BN J

which is assumed to be valid for every point (iJ,P);a is the over-relaxation factor. In practice, the initialgu~ ",° is zero for all Col. Note that (5) is linear in "'f/p'

At this point it is neceeaary to leave the discussionof the numerical solution of the omega equation and ~some theorems concerning the iterative solution of linearequations. Faddeeva [2) is an excellent source for thisdiscussion. Let us consider a set of n linear equations inthe forln

1. INTRODUCTIONThe recent paper by Stuart and O'Neill [4) is concerned

with determining the optimum over-relaxation coefficientin the numerical relaxation of the omega equation. Theseauthors do not consider some well-known theoreticalaspects of iteration procedures. At IMBt, from the peda-gogic point of view, it is instructive to reanalyze theirproblem.

They consider the omega equation in the form

~(II)B(p)vaJIII+~=G(zJ )'.p. (1)

X-A.X+', (6)The application of centered differenc. to (1) yieldsthe finite difference forDl: where A is the matrix of coefficients, , the column mam

or vector of constants, and X the column matrix of fa.unknowns. The Gau raooby or Ordinary Process ofIteration consists of writing (6) as

~,.J+l"+~+I.J.~+~I.J -I.,+(AlI-I.I.,-K~/' (AlII,

+ Ktn(Wt.I..~+~.J.~.]=LfJp, (2)where xa+u =u<» + 1', (7)..

K'h=mIB(p) (Ap)"l'

K',h=.+2K'h,

which in an explicit form we mean

(8)

The procees of iteration consists of choosing & certainvector XCO) and finding XCI), XCI), etc" until for & pre-

&88igned 'cThe horizontal grid size is a=4z=4Y; 4" is the verticalgrid intBval and m is map factor, taken to be unity.Equation (2) is t.he complete three-dimensional equation.Stuart and, O'Neill also consider the special two-dimen-sional case, K'J.=O, and the on&-dimensional equation,B(p) =0, whose finite dj:ff~ce form is

for all i.Izt+o -zt' 1<1,

..: 1.~~- 2...1. ~+ ~.I. ~.= (4.1' )J(J 'h. (3)

Note the minus sign is a correction of a misprint fromthe original paper. The subscripts i,j can be droppedfrom (3) without any 1068 of generality.

Faddeeva giVM us the fundamental theorem: For ~~~ oj 0e p~' oj itenJAon with anti i~itial ~X(O) aM with anti tGl1U oj the ,.tor r, u if ~ aMnJieVnt tAGt all ~tGl1U8 oj U&e mGb'iz A. be u" thanun"" in mod1dU8. This theorem is quite important and theproof is given by Faddeeva.

The Gauss.g&del iterative method is mTnilAt" to theordinary iterative pl'OOMS except that in computing the

100 MONTHLY WEATHER REVIEW

-2 0(k+l) approximation to %(1 one takes into considerationthe (k+ l)th approximations already computed to thecomponents %1, Zt, . . . %(-1. Equation (8) is written forthe GauS&.8eidel method

1

0 -2 1

E=. . ft. (9)i=1,2,

Let us write the equation (6) in the form

X=(B+C)x+r, (10)0

We can only prMUppose that (13) reprMents the p~used by Stuart and O'Neill under the additional empiricalrequirement that a=O when IRNI:Se, where e is theirprMent tolerance. This empirical alteration to the usualextrapolated Leibmann relaxation scheme leads to adifficulty in perfonning an exact theoretical analysis.Here we assume that the correction, RN, is applied eventhough IRNI<e for some "'«. As a principle of faith, we,intuitively, believe that the theoretical analysis is appli-cable to the approximation scheme. Stuart and O'~eillstate that there is no difference in their results for thecutoff and optimum value of the over-relaxation param-eter, a, &8 compared to other Mtimates, and thisstrengthens our supposition.

where the only non-zero elements of B are tbe elements aflbelow the main diagonal and C contains &8 non-zero ele-ments the diagonal elements aff and the elements aflabove the main diagonal. Using these definitions (or Band C, the Ga1188-Seidel process (9) becomM

X<&+I) = Brt.+u + CXU) +1', (11)

or, realTaDging,

X<t+l) =(1- B)-l[CX<t) + I']=MX<t) +G (12)

Hence the Gauss-Seidel method, (11), is equivalent toapplying the ordinary process of iteration to (12) whereM plays the role of the matrix J. in the fundamentaltheorem and G=(I-B)-IP'.

Using this background and the well-kno,,'D fact (e.g.Varga [5D that the speed of convergence is directlydependent on the spectral radius, i.e., the absolute valueof maximum eigenvalue of J. or M as applicable to aparticular case, we now return to the over-relaxation ofthe omega equation as considered by Stuart and O'Neill.First we shall consider the one-dimensional case, then thetwo-dimensional case and, finally, the three-dimensionalcase.

t. ONE-DIMENSIONAL CASE

The insertion of equation (3) in (4) yields

Wlk.+l) = Wlk.) +a[DW(}+l) + EW(k.) - L], (13)

where W"=[(IIl1 (&'I, Ca'al . . . (III) (read W" as transpose ofW). We are considering an arbitrary number of levels, "at which omega is to be determined. The matricM D andE are of order I and have the fonll

FrauD 1.-8peotral radius, p(M), .. a function of the over.reJua-tion coefficient, CI, for the I-D omep equation. The line from0.5 to 1 is common to aU Ourv8. The value of the number ofleveia, 1, for 1-2(1)20 Ipeoift. the appropriate curve.

James J. O'Brien 101February 1968

301-';;,,,

20

~

~

00.&0 0.75

a OPT

FIouu 2.-Tbe number of levels, l, 81 a function of the optimumover-relaxation ooefBoient, a.." with 1-1(1)20 for the 1-D 088e.As l-+m, G." mU8t approach 1.0.

1,00

Figure 2 shows the effect of the number of levels, I, ona..,j &8 l increues a.., approach. 1.0.

Note, since the convergence of (11) is independent of, and X(O), the convergence of the physical problem (13)is independent of L.= (~'P)* ()~ and the initial guees, W(O).Since the omega equation is linear, we should not besurprised that convergence is independent of the forcingfunction or the initial gu-. However, in general, we shallsee that the optimum omega will depend on the numberof grid points and, in the 3-d!mSlmonal case, on otherphysical parameters. Naturally, the number of iteraa()B8required should be dependent on our initial gu~, WCO),since if the p~ will converge with a particular choiceof a, then a "close guess" to the desired W will ~ult inrapid convergence. But for arbitrary a, the. speed of con-vergence should only be related to p(M).

3. TWO-DIMENSIONAL CASE

The omega equation becomes

V:I#=(}(~,lI,'P)IB(.). (1&)

The centered finite difference equation in matrix formfor ovS'-relaxation isWe now consider a to be specified and note that (13)

is equivalent to (11) ifWCt+I) = W(&) +a[Dwc-+l) +BWC&)-L], (16)

where W". = [~I' (IIg,. . . , ~.. 0'11, -,. . ., WI... . .,~.There are other orders which are a.llowable, e.g., seeFox [3] for a discussion of this aspect of the problem.The special case u=A1J=G ja considered and thm-e are~ X r total points on a particular rectangular prMsuresurface.

x=wB=aD

C=I+dJ'=-aL.

. Thus, the convergence of (13) depends on the spectral

radius of

M=(I- B)-lC=(I-aD)-l(l+d) (1.)

The cutoff a's, i.e., the allowable bounds on a for con-vergence, are determined by finding the valu. of awhich pennit the spectral radius of M, p(M), to be 1and -1. A routine &B&1ysis shows that, in this simplecase, 0< a< 1 rmtricta Ip(M) I ~ 1. The two-dimensionalcase and the three-dimensional case are not easily analyzedby hand. In this simple case, the allowable range of a isnot dependent on l, the number of levels and indeed, &8we shall see, this conclusion follows for the 2-D case also.

The determination of the optimum value of w requiresmore ingenuity. The suggested procedure is to specify0<w<1 and determine p(M) and, thU8, find the valueof w which minimiz. p(M). In brief, a was specifiedbetween 0 and 1 and l was alloWed to vary from 2 to 20.Figure 1 was constructed from the computer r.ulta.The case l=4: corrMponds to tJ1e six-level model ofStuart and O'Neill. The optimum a is 0.63. Figure 2shows the optimum a (a..,) 88 a function of l. Stuart andO'Neill oonclude that "the optimum « for tAle I-D equa-tion would probably show only slight variations from onegrid to another." This conclusion is vay miBleading.

I86-aI8 o. .

MONTHL Y WEATHER REVIEW102 Vol. 96, No. 2

FIG1JU 3.-8pectral radius, p(M) .. a function of the OVel'-relaDtion«»emcieDt, Go for tbe 2-D omep equa\ion. The lfDe from 0.25to 0.5 . ~D to all curvs. The value of the number of gridpointB, II, for ,,-1 (1) 10 ipecifiee the appropriate curve.

The rMulta for cz..1 are plotted in figUJoe 4. The curveis extrapolated toward n=20. These results illustratequite clearly that an und8l'8tanding of the theory ofiterative processes can provide considerable insight forund8l'8tanding the over-relaxation procedure. For example,the sharp cutoff a found by Stuart and D'N eill is easilyexplained. We find that a must be l~ than 0.5 if p(M)is to be less than unity for allsquGre grid8 and the size ofthe grid, 1&, is the only free parameter in the analysis.All the other physical parameters do not infiuence p(M).Therefore, if one plots the number of 8C&D8 versus a, onemust find a very sharp cutoff, because t.he number of8C&D8 must approach infinity as a-+O.5. On any scale, anextremely rapid increase in the number of scans will rMult.

The over-relaxation scheme for (16) is 8i~iJ~~ tA> theGauss-Seidel method given by (11) if we equate

X=WB=«DC=I+aEP=-crL.

Thus, as before, convergence of (16) is dependent onlyon the spectral radius of

M=(I-B)-1 C-(l-aD)-1 (I+d). (17)

~. THREE-DIMENSIONAL CASE

The entire problem is a great deal more specialized aswe shall 888. The combination of equations (2), (4), and(5) leads to the matrix equation

WU+l) = ww+«[pwu+I>+QW(&)-L], (18)

which is A1milA.p to (16) except that

Note, since D and E are independent of 4Z, 4Y, W andthe forcing function, convergence is independent of WIG),the initial guMS, the grid-spacing (when 4z=4Y), andthe forcing function.

Now, since we dMire to use an over-relaxation scheme,a> 7.t. Therefore, the cutoff a's are easily found by &1low-ing a to vary from 0 to 1 and by determining p(M),using an available matrix subroutine. The optimum a isfound by the above procedure, since as we allow a totake on valuM [0, 1], p(M) will be found .Iso. The mini-mum value of p(M) shall be taken to be related to theoPQmum Go

Following Stuart and O'Neill, let n=rj i.e., consideronly square regions. Look first at the trivial case, ~= 1where M=l-4a. Hence O<a<~ restricts p(M)<l andthe optimum a is 0.25. From the analysis of Cam [1] andthe I-D case, we expect, as n in~, the cutoffs toremain the same and the optimum a to approach 0.5.Figure 3 was constructed from the computer analysis forn=2(l)10. Note, for n=10, M is of order 100 and con-siderable computer storage is required.

James J. O'Brien 103February 1968

FIou.- '.-The number 01 grid point. in a square array, ", .. afunction 01 the optimum over-relaxation coefficient, a.- with,,=1(1)10 for the 2-D cue. As "-=, a.., muat approach 0.5.Note, tJ)e function is extI'apoIated to ,,=20 b88ed on Stuart andO'NeW'. work.

Hence, for this problem, M=(I-aP)-l (I+Q)i i.e.,convergence is ~ured if p(M)<l. Note, howev.., inthis problem that p(M) is a function of a, a, .6,p, B(P),and the number of grid points. It is not dependent on theinitial gu~ W<o" the boundary condition~ and the forcingfunction. For a particular inv.tigation, a, .6,p, B(p) andthe number of grid points would be specified from physicalor economical considerations and, in principle, the Opg-mum value of a can be found by allowing a to vary from0 to ~ and we can thus find the value of a which mini-mis. p(M).

In this special cue, we feel it is not practical to pa--form an analysis as performed for the 1-D and 2-D ~.The analysis can only be of intereet to a few invMQgatorswho might chooee (1) a square region, (2) the identicalprofile of G', (3) the same pressure intaoval, etc. HowevC',we can offer the following suggestions. First, the work ofCam [I] should be reviewed by all inVMtigators who areinterested in this problem. Cam outJinee a practicalprocedure to determine the optimum relaxation coefficient.as the numerical solution of an elliptic equation is beingaccomplished. His techniqUM are baaed on a sound theo-retical understanding of the iterative proc.-.

Secondly, we must concur with Stuart and O'N 6illthat the discrepancy between their results and thetheoretical results are easily explained by the distinctlydifferent physical problem analyzed by them. Finally,the strange rMult for tbe 1.60 grid must be suspect sinceif we consider the trivial case, n= I, O<a<2/(4+2K1J,).Now, since only a is varied, the maximum a must decreaseas a inCre88M. This was found for a= 1,2,3 degretW, butnot for the I.6-degree case.

where P and Q are of order nXrXl and D is exactly asthat for (16). at is a diagonal matrix

Kilt 0

Kta,0

q=l, 2, . . . toR,=

K~

In addition

at 0

5. CONQ.USlONIn brief, we have demonstra.ted the importance of

various physical parameters on the convergence of theomega equation in one, two, and three dimensions. It isconceptu&lly poesible to analyze the empirical rMUlts ofStuart and O'Neill using well-known theoretical aspectsof iterative methods for solving systems of linear equa-tions on a computer.

It is recognized that the empirical approach of Stuartand O'Neill may be the most feasible way to determinethe optimum over-relaxation coefficient. However, thispractical attack must be tempered with a knowledge ofthe theoretical basis of iterative p~. Such a blendis outlined by Carre [1].

E, Ra

where B.=B except that the diagonal elements of B'are - K' u. instead of -4. R. in Q are as in P. An el8D\entof L is L,w.

ACKNOWLEDGMEN1S

The work. 01 Stuart and O'NeiD waa brought to my attentionwhUe I wae a Visiting Lecturer at the Depart.meni of Meteoroloay,Florida State University, during the summer of 1967. The neoe8aryprograminc waa performed by John Llu on hl8 brief visit to theNCAR Computing Center in 1967.

Rearranging (18) we find

Wt&+l) =(I-aP)-l[(I+Q) WOO -L]. (19)

MONTHLY WEATHER REVIEW104 Vol. 96, No. i

.. D. W. Stuart and T. H. R. O'Neal, "The Over-ReluationFactor in the Numerical Solution 01 the Omega Equation,"MOIIlWv W~ R-w, vol. 95, No.6, May 1967, pp.303-307 .

5. R. B. Varga, MGtrU lWatWe Anal,.v, Prentice-Hall, EngI&-wood Clift's, N.J., 1962, 322 pp.

REFERENCES

1. B. A. C~ "The Determination of the Optimum AcceleratingFactor for SuOOtBive Over--Re1uation," COIfIpWer JhnIGl,vol. 4, No.1, Apr. 1961, pp. 73-78.

2. V. N. Faddeeva, COIfIJ'1'talional MecAodI 01 L..r Algebra,Dover Publications, New York, 1959,252 pp.

3. L. Fox, Numerical Soluti~ of Ordinory and Partial DifferentialBquGtt:.., Perpmon PJ'eE, Oxford, 1962, 509 pp.

[~ &ptembIr 11 J 1967]

ReplyDAVID W. STUART

Florida State University, Tallahassee, Fla.

We are indebted to Dr. O'Brien [I] for calling the workof Carr~ [2] to the attention of U8 and perhaps of otheriny~tigators. Our interest in the omega equation stemsfrom a. desire to use this equation to obtain diagnosticvertical motions for carehtl analysis of the thre&-dimen-sional stnlcture of atmospheric systems. Ha\ing startedsuch studi~ late in 1960 prior to the \,'ork of Carre andprior to kno\"ledge of Miyakoda's [3] \vork, I ''-as forcedinto the more empirical approach for finding the optimumover-relaxation factor, a.~I' 88 mentioned in our paper(Stuart and O'Neill [4]). Since some kno,cledge of theover-relaxation factor W88 needed for our work, ,ve decidedto search a bit for aO~I' We tested the I-D and 2-D casesmore for completeness and to show that our relaxationtechnique fitted the knO\vn theoretical results. Suffice it tosay that .our paper reports on:the r~ults of an aql studyfor a particular grid and stability profile and we hope otherinvestigators ,vill benefit by Carr~'s paper and our results.Indeed we have followed the approach suggested byO'Brien at the end of his section 4, but ,vithout the aid ofO~'s work.

Recently I have extended the model to yield omega atnine interior levels for the ~ grid (i.e., N.=N,= 18,.¥.= 11, 4p= 10 cb., and Q' for the standard atmosphere).a..,=0.15 was found with the sharp cutoff near 0.20.Again a.., ,vas folmd r.t a much lower value than the 0.33given by Miyakoda's [3]. It is hoped that the ~ults fora.., obtained for our grid model will be of aid to othersusing very similar models for obt~g omega.

REFERENCES

1. J. J. O'Brien, "On the Over-Relaxation Faetor In the NumericalSolution of the Omega Equation," MOtItAlr WeGtA.!' ReIIW,vol. 96, No. 2, Feb. 1968, pp. 99-104.

2. B. A. Carri, "The Determination of the Optimum AcceleratingFactor for Succe.ive Over-Relaxation," COII&f"Illf' Journal,~1. 4, No.1, Apr. 1961, pp. 73-78.

3. K. Miyakoda, "Tcat of Converaence Speed of Iterative Methodsfor Solving 2 And 3 Dimensional Elliptio-Type DifferentialEquationa," Journal 01 tA. M~ BociC, 01 J4J'G""Ser. 2, vol. 38, No.2, Apr. 1960, pp. 107-124.

4. D. W. Stuart and T. H. R. O'Neill. "The Ova--ReJaxaUonFactor in the Numerical Solution of the 0Ine«a Equation,"MonlAlr WealAer R6I/w, voi. 95, No.5, May 1967, pp.303-307 .

[ ~ (kto6er 17, 1967)