inverse–based multilevel ilu preconditioners theory, applications...

Inverse–Based Multilevel ILU PreconditionersTheory, Applications and Software

Matthias Bollh ofer Yousef SaadTU Berlin / TU Braunschweig Univ. of Minnesota

8-TH COPPERMOUNTAIN CONFERENCE ONITERATIVE METHODS

MARCH 28 – APRIL 02, 2004

April 02, 2004

OUTLINE

1. inverse–based ILUs, theoretical background

2. Multilevel algorithms

3. ILUPACK

4. Numerical experiments

5. Conclusions

supported by the DFG research center “Mathematics for key technologies” in Berlin 0

0

Inverse–Based ILUs

Ax = b

INCOMPLETE LU DECOMPOSITION

A =

(B F

E C

)= LDU + E ≈

(@

@@

@@

)(@

@@

@@

)(@

@@

@@

), where L, U are unit diagonal.

INVERSE–BASED DROPPING

At elimination step k, drop lik, ukj if

|lik| ∗ ‖e>k L−1‖ 6 τ |dkk|, |ukj| ∗ ‖U−1ek‖ 6 τ |dkk|,

where τ is a drop tolerance [B. ’01, ’03], [Li, Saad, Chow ’03].

=⇒ L−1 and U−1 serve as approximate inverses

M. Bollh ofer /Y. Saad Inverse–Based Multilevel ILUs — Theory, Applications and Software 1

1

Inverse–Based ILU — Approximate Inverse Relation

FACTORED APPROXIMATE INVERSE (AINV) , [Benzi, Tuma ’98]

W>AZ =

(@

@@

@@

)( )(@

@@

@@

)= D + E

W ←→ L−>, Z ←→ U−1

THEOREM [B., Saad ’01]. Suppose that the AINV process and the inverse–based ILU are computedusing the same approximate Schur complement.Let τ be the drop tolerance. Then

|(I −WL>)ij| 6 2(j− i)τ, |(I − ZU)ij| 6 2(j− i)τ ∀i < j.


2

Impact on the Inverse Error

GROWTH OF THE INVERSE TRIANGULAR FACTORS

Problem. ‖L−1‖ and ‖U−1‖ may become very large.

⇒ pivoting recommended.

BUT WHAT KIND OF PIVOTING IS APPROPRIATE?

Consider the approximate factorization

A =

(B F

E C

)=

(LB 0

LE I

)︸︷︷︸

Lk

(DB 0

0 SC

)︸︷︷︸

Dk

(UB UF

0 I

)︸︷︷︸

Uk

+Ek,

where Lk and U>k are unit lower triangular matrices and the leading k × k part of Dk is diagonal.

INVERSE ERROR

Fk := L−1k EkU

−1k


3

Inverse Error Bound

LEMMA . Denote by vk and wk the entries in column k of LkDk (resp. row k of DkUk) that are droppedat step k of the incomplete LU decomposition.

1. Suppose that the approximate Schur complement SC is defined via

SC = C − LEDBUF (S–version)

⇒ Fk =∑l6k

(L−1

k vle>l U−1

k + L−1k elw

>l U−1

k

)2. Suppose that the approximate Schur complement SC is defined via

SC =[−LEL−1

B I]A

[−U−1

B UF

I

](T–version)

⇒ Fk =∑l6k

(L−1

k vle>l + elw

>l U−1

k

)


4

Application: Inverse-Based Pivoting

CONSEQUENCES

• ‖L−1k ‖ and ‖U−1

k ‖ contribute to the inverse error

• For the S–version even their PRODUCT ‖L−1k ‖ · ‖U

−1k ‖!

INVERSE–BASED PIVOTING

Control ‖L−1‖, ‖U−1‖ 6 κ by a factor or skip strategy

At step k :

@@

@@

@@

@@

∗

↗

↘

@@

@@

@@

@@

∗

@@

@@

@@

@@

∗

factor

skip

Rows/columns that exceed the prescribed bound are pushed to the end


5

Inverse–Based Multilevel ILUs

SKETCH OF THE TEMPLATES

1. Compute a static partial reordering

A→ P>AQ =

(B F

E C

)

2. Factor P>AQ and control ‖L−1k ‖, ‖U−1‖ 6 κ.

P>AQ −→ P>AQ =

(B F

E C

)=

(LB 0

LE I

)(DB 0

0 SC

)(UB UF

0 I

)

3. Apply strategy recursively to the approximate Schur complement SC (multilevel scheme)

A1.→ P>AQ

2.→ P>AQ3.→

(L ·D · U F

E S

)

S1.→ P>S SQS

2.→ P>S SQS3.→(

LS ·DS · US FS

ES T

)· · ·


6

Inverse-Based Multilevel ILUs — Templates

TEMPLATE 1. STATIC REORDERINGS

Examples.

• RCM (Reverse Cuthill–McKee), MMD (Multiple Minimum Degree), AMF (Approximate MinimumFill), INDSET (independent set, ARMS), MC64 (from HSL).

• ddPQ [Saad ’03]. Permutations P, Q as a compromise between diagonal dominance and fill

1. Define numerical weights

ρi =max

j|aij|∑

j |aij|, γj =

maxi|aij|∑

i |aij|2. define fill weights, e.g.

fi = |{j : aij 6= 0}, gj = |{i : aij 6= 0}, hij = (fi − 1)(gj − 1) + 1, . . .

3. order ratios in decreasing order, e.g. order

{ ρi

gki

∀i}, {γj

flj

∀j}, {ρiγki

hi,ki

∀i,ρljγj

hlj ,j∀j}, . . .

4. use associated indices (i, j) and skip duplicate entries

Further selection criteria, e.g. enforce submatrix to be close to lower triangular


7

Inverse–Based Multilevel ILUs — Templates

TEMPLATE 2. PILUC — INVERSE-BASED ILU THAT CONTROLS‖L−1k ‖,‖U

−1k ‖

• ILU (Crout–version)??

−→

-- ↓

• ‖L−1k ‖, ‖U

−1k ‖ 6 κ controlled by diagonal pivoting

@@

@@

@@

@@

∗

↗

↘

@@

@@

@@

@@

∗

@@

@@

@@

@@

∗

factor

skip

• ‖L−1k ‖, ‖U

−1k ‖ estimated [Cline, Moler, Stewart, Wilkinson ’77], [B. ’03]


8


TEMPLATE 3. MULTILEVEL SCHEME

(P>AQ)−1 =

(B F

E C

)−1

≈{(

LB 0

LE I

)(DB 0

0 S

)(UB UF

0 I

)}−1

≈(

B−1 0

0 0

)+

(−B−1F

I

)S−1

C

(−EB−1 I

), where B = LBDBUB

Recursive application to SC −→ (additive) algebraic multigrid scheme (or block ILU)

• use concept of ARMS [Saad, Suchomel ’99], i.e. skip LE, UF .

• After AMG is set up, SC can be discarded.

• Without LE, UF , forward/backward substitution twice as expensive, but more accurate and lessmemory needed.


9


APPROXIMATE SCHUR COMPLEMENTS

1. SC = C − LEDBUF

use analogous approximation scheme for B ≈ LBDBUB (S–version)

2. SC =[−LEL−1

B I]A

[−U−1

B UF

I

]use analogous approximation scheme for B ≈ LBDBUB (T–version)

3. Use S–version for the computation of B ≈ LBDBUB (M–version)Use T–version for the computation of SC

APPROXIMATION ORDER

Denote by τ the drop tolerance and let B = LBDBUB + EB

• If we use 1., then ‖EB‖ = O(τ ), ‖L−1B EBU−1

B ‖ = O(τκ2)

• if we use 2., then ‖EB‖ = O(τ 2), ‖L−1B EBU−1

B ‖ = O(τ 2κ)

• Similar arguments apply to SC


10

Preconditioning Package: ILUPACK

Preconditioning software package (C/FORTRAN 77) based on inverse–based multilevel ILUs

� � � � � � � � � � � � � ��

� � � � � � � � � � � � � ��

S P A R S K I T RA M S BLASMC64SPARSPAK

Appr. Min. Fill

Interfaces

ILUPACK


11

ILUPACK — General

NAMES AND FORMATS

1. ILUPACK currently supports real single/double precision and complex single/double precision.

2. Apart from the arithmetic ILUPACK distinguishes between two classes of matrices, namelygeneral matrices and symmetric (Hermitian) positive definite matrices.

3. matrices are stored in SPARSKIT format (compressed sparse row format).

SPARSKIT, ARMS, . . .

• Several routines from SPARSKIT incorporated (GMRES, FGMRES, ILUT, ILUTP,. . . )

• Reorderings (INDSET, ddPQ) and several tools from ARMS adapted

• Most common reorderings supported or at least interfaces are provided


12

ILUPACK — Templates

TEMPLATE 1

• Preconditioner allows three arbitrarily chosen permutation/scaling functions

1. Initial preprocessing (applied only once)2. Regular reordering (applied to any subsequent system)3. Final pivoting (if the regular reordering fails)

• Custom reorderings possible

TEMPLATE 2

Partial ILUC (Crout version) with static permutation and diagonal pivoting (F77 core routine)

TEMPLATE 3

• Three different versions of approximate Schur complements are supported (S–, M– and T–version). M–version is default.

• After the computation of the approximate Schur complement the parts LE and UF are discarded.

• if the fill exceeds a certain amount, the algorithm switches to full-matrix processing.


13

ILUPACK — Experiments

PRECONDITIONERS

1. inverse–based multilevel ILU (referred as “ILUPACK”)

2. ILUTP

3. ILUC (single–level, inverse–based)

ORDERINGS

PQ, RCM, . . . + row/column scaling

ITERATIVE SOLVERS

• GMRES, PCG. Iteration stopped if ‖b− Ax(k)‖ 6√

eps ‖b− Ax(0)‖

• Iterative solver treated as failure, if a break down occurred or more than 500 steps were needed.


14

ILUPACK — Application to Unstructured Systems

EXAMPLE . Matrix ATT/TWOTONE

10−6

10−4

10−2

0

50

100

150

200

250

300

350

400 ILUC, MMD+MC64ILUTP, MMD+MC64ILUPACK−M, PQILUPACK−S, PQ

10−6

10−4

10−2

0

2

4

6

8

10

12ILUC, MMD+MC64ILUTP, MMD+MC64ILUPACK−M, PQILUPACK−S, PQ

Computation time Memory requirement

drop tol. drop tol.


15

ILUPACK — Application to Unstructured Systems

EXAMPLE . Matrix VAVASIS/AV41092

10−6

10−4

10−2

0

100

200

300

400

500

600ILUTP, RCMILUPACK−M, PQILUPACK−S, PQ

10−6

10−4

10−2

0

2

4

6

8

10

12ILUTP, RCMILUPACK−M, PQILUPACK−S, PQ


drop tol. drop tol.


16

ILUPACK — dependence on inverse bound κ

Almost no difference between κ = 10 , κ = 25 , κ = 50 , κ = 100

64 LARGE TEST MATRICES, NUMBER OF SUCCESSFUL COMPUTATIONS

1 2 3 4 50

10

20

30

40

50

60

70

1 2 3 4 50

10

20

30

40

50

60

70

ILUPACK M–version ILUPACK S–version

drop tol. [10−∗] drop tol. [10−∗]


17




1 2 3 4 50

10

20

30

40

50

60

70

1 2 3 4 50

10

20

30

40

50

60

70

ILUPACK M–version ILUTP + RCM



18




1 2 3 4 50

10

20

30

40

50

60

70

1 2 3 4 50

10

20

30

40

50

60

70

ILUPACK M–version ILUTP + RCM + MC64



19

ILUPACK — Abused as AMG

EXAMPLE . cylindric shell problem

Tight bounds on ‖U−1‖ 6 κ implicitly yield FINE GRID/COARSE GRID selection

10−2

10−1

4

6

8

10

12

14

16κ= 2κ= 5κ= 10κ= 25κ= 50κ=100

10−2

10−1

0.5

1

1.5

2

2.5

3κ= 2κ= 5κ= 10κ= 25κ= 50κ=100


drop tol. drop tol.


20

ILUPACK — Abused as AMG

EXAMPLE . cylindric shell problem

Number of levels. κ = 2, κ = 5, κ = 10, κ = 25, κ = 50, κ = 100.

1 2 3 4 50

0.5

1

1.5

2

2.5

3

3.5

4Number of levels depending on κ

drop tol. 0.1 0.05 0.025 0.01 0.005


21

Conclusions

1. Multilevel preconditioners using inverse–based ILUs

2. numerical experiments indicate higher robustness with respect to parameter tuning

(time dependent problems, nonlinear equations)

3. Main objective: inverse triangular factors are kept bounded

4. AMG-like preconditioners realized in the software package ILUPACK that offers inverse–basedmultilevel ILUs for a broad class of matrices

5. Still to do: reordering strategies for PDE-type problems

ILUPACK BEING RELEASED BYAPRIL/MAY, 2004

http://www.math.tu-berlin.de/ilupack/


22

inverse–based multilevel ilu preconditioners theory, applications...

Documents