model reduction of large-scale dynamical systemsaca/aca_monopoli_iii.pdf · model reduction of...

38
Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation Thanos Antoulas Rice University and Jacobs University email: [email protected] URL: www.ece.rice.edu/ ˜ aca International School, Monopoli, 7 - 12 September 2008 Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 1 / 38

Upload: others

Post on 17-Jul-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Model reduction of large-scale dynamical systemsLecture III: Krylov approximation and rational interpolation

Thanos Antoulas

Rice University and Jacobs University

email: [email protected]: www.ece.rice.edu/ aca

International School, Monopoli, 7 - 12 September 2008

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 1 / 38

Page 2: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 2 / 38

Page 3: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov approximation methods

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 3 / 38

Page 4: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov approximation methods

Krylov approximation methods

Given Σ =

(A BC D

), expand the transfer function around s0:

H(s) = η0 + η1(s − s0) + η2(s − s0)2 + η3(s − s0)

3 + · · ·

Moments at s0: ηj , j ≥ 0. Find Σ =

(A BC D

), with

H(s) = η0 + η1(s − s0) + η2(s − s0)2 + η3(s − s0)

3 + · · ·

such that for appropriate k :

ηj = ηj , j = 1, 2, · · · , k

Moment matching methods can be implemented in a numerically stableand efficient way.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 4 / 38

Page 5: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov approximation methods

Krylov approximation methods: Special cases

• s0 =∞ Moments: Markov parametersProblem: (partial) realizationSolution computed through: Lanczos and Arnoldi procedures• s0 = 0Problem: Pade approximationSolution computed through: Lanczos and Arnoldi procedures• In general: arbirtary s0 ∈ CProblem: Rational interpolationSolution computed through: Rational Lanczos

• Computation of moments: numerically problematic

• Key fact for numerical reliability: If (A, B, C, D) given

• moment matching without moment computation⇒ iterative implementation.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 5 / 38

Page 6: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 6 / 38

Page 7: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Arnoldi procedure

The Arnoldi procedure

Given is A ∈ Rn×n, and b ∈ Rn. Let Rk (A, b) ∈ Rn×k be the reachability orKrylov matrix. It is assumed that Rk has full column rank equal to k .

Devise a process which is iterative and at the k th step we have

AVk = Vk Hk + Rk , Vk , Rk ∈ Rn×k , Hk ∈ Rk×k , k = 1, 2, · · · , n

=V VA R+

H

These quantities have to satisfy the following conditions at each step.

The columns of Vk are orthonormal: V∗k Vk = Ik , k = 1, 2, · · · , n.

span col Vk = span colRk (A, b), k = 1, 2, · · · , n

The residual Rk satisfies the Galerkin condition: V∗k Rk = 0,k = 1, 2, · · · , n.

This problem can be solved by the Arnoldi procedure.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 7 / 38

Page 8: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Arnoldi procedure

Arnoldi: recursive implementation

Given: A ∈ Rn×n, b ∈ Rn

Find: V ∈ Rn×k , f ∈ Rn, and H ∈ Rk×k , such that

AV = VH + fe∗k where H = V∗AV, V∗V = Ik , V∗f = 0,

with H in upper Hessenberg form.

1 v1 = b‖b‖ , w = Av1; α1 = v∗1w

f1 = w− v1α1; V1 = (v1); H1 = (α1)

2 For j = 1, 2, · · · , k − 1

1 βj =‖ fj ‖, vj+1 =fjβj

2 Vj+1 =(Vj vj+1

), Hj =

(Hj

βje∗j

)3 w = Avj+1, h = V∗j+1w, fj+1 = w− Vj+1h

4 Hj+1 =(

Hj h)

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 8 / 38

Page 9: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Arnoldi procedure

Properties of Arnoldi

Hk is obtained by projecting A onto the span of the columns of Vk : Hk = V∗k AVk .

The remainder Rk has rank one and can be written as Rk = rk e∗k , where ek is the k th unit vector; thus rk ⊥ Rk .

This further implies that vk+1 =rk‖rk‖

, where vk+1 is the (k + 1)st column of V. Consequently, Hk is an upper

Hessenberg matrix.

Hk =

h1,1 h1,2 h1,3 · · · h1,k−1 h1,kh2,1 h2,2 h2,3 · · · h2,k−1 h2,k

h3,2 h3,3 h3,k−1 h3,k

. . ....

.

.

.

hk−1,k−1 hk−1,khk,k−1 hk,k

Let pk (λ) = det(λIk − Hk ), be the characteristic polynomial of Hk . This monic polynomial is the solution of thefollowing minimization problem

pk = arg min ‖p(A)b‖2

where the minimum is taken over all monic polynomials p of degree k . Since pk (A)b = Ak b +Rk · p, where pi+1

is

the coefficient of λi of the polynomial pk , it also follows that the coefficients of pk provide the least squares fit betweenAk b and the columns ofRk .

There holds

rk =1

‖pk−1(A)b‖pk (A)b, Hk,k−1 =

‖pk (A)b‖‖pk−1(A)b‖

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 9 / 38

Page 10: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Arnoldi procedure

An alternative way of looking at Arnoldi

Consider a matrix A ∈ Rn×n, a starting vector b ∈ Rn, and the correspondingreachability matrix Rn = [b Ab · · · An−1b]. The following relationship holdstrue:

ARn = RnF where F =

0 0 · · · 0 −α01 0 · · · 0 −α10 1 · · · 0 −α2

...0 0 · · · 1 −αn−1

and χA(s) = sn + αn−1sn−1 + · · ·+ α1s + α0, is the characteristic polynomialof A. Compute the QR factorization of Rn:

Rn = VU, V∗V = In, U upper triangular

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 10 / 38

Page 11: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Arnoldi procedure

It follows that

AVU = VUF ⇒ AV = V UFU−1︸ ︷︷ ︸A

⇒ AV = VA

Since U is upper triangular, so is U−1; furthermore F is upper Hessenberg.Therefore A being the product of an upper triangular times an upperHessenberg times an upper triangular matrix is upper Hessenberg.

The k-step Arnoldi factorization can now be obtained by considering the firstk columns of the above relationship, to wit:

[AV]k =[VA]

k ⇒ A[V]k = [V]k Akk + fe∗k

where f is a multiple of the (k +1)-st column of V. Notice that Akk is still upperHessenberg, while the columns of [V]k provide an orthonormal basis for thespace spanned by the first k columns of the reachability matrix Rn.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 11 / 38

Page 12: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Lanczos procedure

The symmetric Lanczos procedure

If A = A∗ then the Arnoldi procedure is the same as the symmetric Lanczosprocedure. In this case Hk is tridiagonal:

Hk =

α1 β2β2 α2 β3

β3 α3

. . .. . .

. . .

αk−1 βkβk αk

This matrix shows that the vectors in the Lanczos procedure satisfy a threeterm recurrence relationship

Avi = βi+1vi+1 + αivi + βivi−1, i = 1, 2, · · · , k − 1

Remark. If the remainder rk = 0, the procedure has terminated, in whichcase if (λ, x) is an eigenpair of Hk , (λ, Vk x) is an eigenpair of A (since Hk x =λx implies AVk x = Vk Hk x = λVk x).

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 12 / 38

Page 13: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Lanczos procedure

Two-sided Lanczos

The two-sided Lanczos procedure. Given A ∈ Rn×n which is notsymmetric, and two vectors b, c∗ ∈ Rn, devise a process which is iterativeand the k th step there holds:

AVk = Vk Hk + Rk , A∗Wk = Wk Hk + Sk , k = 1, 2, · · · , n.

Biorthogonality: W∗k Vk = Ik ,

span col Vk = span colRk (A, b), span col Wk = span colRk (A∗, c∗),

Galerkin conditions: V∗k Sk = 0, W∗k Rk = 0, k = 1, 2, · · · , n.

Remarks.

• The second condition of the second item above can also be expressed as span rows W∗k = span rowsOk (c, A), whereOk isthe observability matrix of the pair (c, A).

• The assumption for the solvability of this problem is detOk (c, A)Rk (A, b) 6= 0, k = 1, 2, · · · , n.

• The associated Lanczos polynomials are defined as pk (λ) = det(λIk − Hk ), and the induced inner product is defined as〈p(λ), q(λ)〉 = 〈p(A∗)c∗, q(A)b〉 = c p(A) · q(A) b.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 13 / 38

Page 14: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Lanczos procedure

Two-sided Lanczos: recursive implementation

Given: the triple A ∈ Rn×n, b, c∗ ∈ Rn

Find: V, W ∈ Rn×k , , g ∈ Rn, and H ∈ Rk×k , such that

AV = VH + fe∗k , A∗W = WH∗ + ge∗k where

H = V∗AW, V∗W = Ik , W∗f = 0, V∗g = 0. The projections πL and πU above,are given by V∗, W, respectively.

1 β1 :=√|b∗c∗|, γ1 := sgn (b∗c∗)β1

v1 = b/β1, w1 := c∗/γ1

2 For j = 1, · · · , k , set

1 αj = w∗j Avj

2 rj = Avj − αjvj − γjvj−1, qj = A∗wj − αjwj − βjwj−1

3 βj+1 =√|r∗j qj |, γj+1 = sgn (r∗j qj)βj+1

4 vj+1 = rj/βj+1, wj+1 = qj/γj+1

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 14 / 38

Page 15: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures The Lanczos procedure

Properties of two-sided Lanczos

Hk is obtained by projecting A as follows: Hk = W∗k AVk .

The remainders Rk , Sk have rank one and can be written as Rk = rk e∗k ,Sk = qk e∗k .

This further implies that vk+1, wk+1 are scaled versions of rk , qkrespectively Consequently, Hk is a tridiagonal matrix.

The generalized Lanczos polynomials pk (λ) = det(λIk − Hk ),k = 0, 1, · · · , n−1, p0 = 1, are orthogonal: 〈pi , pj〉 = 0, for i 6= j .

The columns of Vk , Wk and the Lanczos polynomials satisfy thefollowing three-term recurrences

γk vk+1 = (A− αk )vk − βk−1vk−1βk wk+1 = (A∗ − αk )wk − γk−1wk−1γk pk+1(λ) = (λ− αk )pk (λ) − βk−1pk−1(λ)βk qk+1(λ) = (λ− αk )qk (λ) − γk−1qk−1(λ)

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 15 / 38

Page 16: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures An example

Example: symmetric Lanczos

Consider the following symmetric matrix:

A =

2 1 2 11 2 0 12 0 2 11 1 1 0

With the starting vector b = [1 0 0 0]∗, we obtain

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 16 / 38

Page 17: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

The Arnoldi and the Lanczos procedures An example

V1 =

[ 1000

], H1 = [2], R1 =

[ 0121

]

V2 =

1 00 1√

60 2√

60 1√

6

, H2 =[

2√

6√6 8

3

], R2 =

0 00 1√

540 −1√

540 1√

54

V3 =

1 0 00 1√

61√3

0 2√6

−1√3

0 1√6

1√3

, H3 =

[2

√6 0√

6 83

1√18

0 1√18

43

], R3 =

[0 0 0

0 0√

32

0 0 0

0 0 −√

32

]

V4 =

1 0 0 00 1√

61√3

1√2

0 2√6

−1√3

0

0 1√6

1√3

−1√2

, H4 =

2√

6 0 0√6 8

31√18

0

0 1√18

43

√3√2

0 0√

3√2

0

, R4 = 04×4

where

AVk = Vk Hk + Rk , V∗k Rk = 0 ⇒ Hk = V∗k AVk , k = 1, 2, 3, 4.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 17 / 38

Page 18: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov methods and moment matching

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 18 / 38

Page 19: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov methods and moment matching

Arnoldi and moment matching

The Arnoldi factorization can be used for model reduction as follows. Recallthe QR factorization of the reachability matrix Rk ∈ Rn×k ; a projection VV∗can then be attached to this factorization:

Rk = VU ⇒ V = Rk U−1

where V ∈ Rn×k , V∗V = Ik , and U is upper triangular. The reduced ordersystem is:

Σ =

(A BC

)where A = V∗AV , B = V∗B , C = CV

Theorem. Σ as defined above satisfies the equality of the Markov parametersηi = ηi , i = 1, · · · , k . Furthermore, A is in Hessenberg form, and B is amultiple of the unit vector e1.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 19 / 38

Page 20: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov methods and moment matching

Proof. First notice that since U is upper triangular, v1 = B‖B‖ , and since

V∗Rk = U it follows that B = u1 =‖ B ‖ e1; therefore B = V∗B.VV∗B = VB = B, hence AB = V∗AVV∗B = V∗AB; in general, since VV∗ is aprojection along the columns of Rk , we have VV∗Rk = Rk ; moreover:Rk = V∗Rk ; hence

(η1 · · · ηk ) = CRk = CVV∗Rk = CRk = (η1 · · · ηk )

Finally, the upper triangularity of U implies that A is in Hessenberg form.

Remark.Similarly, one can show that reduction by means the two-sided Lanczosprocedure preserves 2k Markov parameters.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 20 / 38

Page 21: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Krylov methods and moment matching Remarks

Remarks

The number of operations is O(k2n) vs. O(n3), which implies efficiency.The requirement for memory is large if k is relatively large.

Only matrix-vector multiplications are required. No matrix factorizationsand/or inversions. There is no need to compute the transformed n-thorder model and then truncate. This eliminates ill-conditioning.

Drawbacks: • Numerical issue: Arnoldi/Lanczos methods looseorthogonality. This comes from the instability of the classicalGram-Schmidt procedure. Remedy: re-orthogonalization.• no global error bound.• Σ tends to approximate the high frequency poles of Σ. Remedy: matchexpansions around other frequencies⇒ rational Lanczos.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 21 / 38

Page 22: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 22 / 38

Page 23: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection Realization by projection

Partial realization by projection

Given a system Σ = (A, B, C), where A ∈ Rn×n and B, C∗ ∈ Rn, We seek alower dimensional model Σ = (A, B, C), where A ∈ Rk×k , B, C∗ ∈ Rk , k < n,such that Σ preserves some properties of the original system, throughappropriate projection methods. In other words, we seek V ∈ Rn×k andW ∈ Rn×k such that W∗V = Ik , and the reduced system is given by:A = W∗AV, B = W∗B, C = CV.

Lemma

With V = [B, AB, · · · , Ak−1B] = Rk (A, B) and W any left inverse of V , Σ isa partial realization of Σ and matches k Markov parameters.

From a numerical point of view, one would not use V as defined above since usually the columns of V are almost linearlydependent. As it turns out any matrix whose column span is the same as that of V can be used.

Proof.

We have CB = CVW∗B = CRk (A, B)e1 = CB; furthermore

CAj B = CRk (A, B)W∗AjRk (A, B)e1 = CRk (A, B)W∗Aj B = CRk (A, B)ej+1 = CAj B, j = 1, · · · , k − 1.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 23 / 38

Page 24: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection Interpolation by projection

Rational interpolation by projection

Suppose now that we are given k distinct points sj ∈ C. V is defined as thegeneralized reachability matrix

V =[(s1In − A)−1B, · · · , (sk In − A)−1B

],

and as before, let W be any left inverse of V. Then

Lemma

Σ defined above, interpolates the transfer function of Σ at the sj , that is

H(sj) = C(sj In − A)−1B = C(sj Ik − A)−1B = H(sj), j = 1, · · · , k .

Proof.

The following string of equalities leads to the desired result:

C(sj Ik − A)−1B = CV(sj Ik − W∗AV)−1W∗B

= C[(s1In − A)−1B, · · · , (sk In − A)−1B

] (W∗(sj In − A)V

)−1W∗B

=[C(s1In − A)−1B, · · · , C(sk In − A)−1B

] ([· · · W∗B · · · ]

)−1 W∗B

=[C(s1In − A)−1B, · · · , C(sk In − A)−1B

]ej

= C(sj In − A)−1B.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 24 / 38

Page 25: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection Interpolation by projection

Matching points with multiplicity

We now wish to match the value of the transfer function at a given points0 ∈ C, together with k − 1 derivatives. For this we define the generalizedreachability matrix

V =[(s0In − A)−1B, (s0In − A)−2B, · · · , (s0In − A)−k B

],

together with any left inverse W thereof.

Lemma

Σ interpolates the transfer function of Σ at s0, together with k − 1 derivativesat the same point, j = 0, 1, · · · , k − 1:

(−1)j

j!

d j

dsjH(s)

∣∣∣∣∣s=s0

= C(s0In − A)−(j+1)B = C(s0Ik − A)−(j+1)B =(−1)j

j!

d j

dsjH(s)

∣∣∣∣∣s=s0

Proof.

Let V be as defined as above, and W be such that W∗V = Ik . It readily follows that the projected matrix s0Ir − A is in companionform (expression on the left) and therefore its powers are obtained by shifting its columns to the right:

s0Ik − A = W∗(s0In − A)V = [W∗B, e1, · · · , ek−1] ⇒ (s0Ik − A)` = [ ∗ · · · ∗︸ ︷︷ ︸`−1

, W∗B, e1, · · · , ek−`].

Consequently [W∗(s0In − A)V]−`W∗B = e` , which finally impliesC(s0Ik − A)−`B = CV [W∗(s0I− A)V]−` W∗B = CVe` = C(s0In − A)−`B, ` = 1, 2, · · · , k.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 25 / 38

Page 26: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection Interpolation by projection

General result: rational Krylov

A projector which is composed of any combination of the above three casesachieves matching of an appropriate number of Markov parameters andmoments. Let the partial reachability matrix be

Rk (A, B) =[B AB · · · Ak−1B

],

and partial generalized reachability matrix be:Rk (A, B;σ) =

[(σIn − A)−1B (σIn − A)−2B · · · (σIn − A)−k B

].

Rational Krylov

(a) If V as defined in the above three cases is replaced by V = VR, R ∈ Rk×k ,det R 6= 0, and W by W = R−1W, the same matching results hold true.(b) Let V be such that

span col V = span col [Rk (A, B) Rm1(A, B;σ1) · · · Rm`(A, B;σ`)] ,

and W any left inverse of V. The reduced system matches k Markovparameters and mi moments at σi ∈ C, i = 1, · · · , `.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 26 / 38

Page 27: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection Interpolation by projection

Two-sided projections: the choice of W

Let Ok (C, A) ∈ Rk×n, be the partial observability matrix consisting of the firstk rows of On(C, A) ∈ Rn×n. The first case is

V = Rk (A, B), W = (Ok (C, A)Rk (A, B)︸ ︷︷ ︸Hk

)−1Ok (C, A).

Lemma

Assuming that detHk 6= 0, Σ is a partial realization of Σ and matches 2kMarkov parameters.

Given 2k distinct points s1, · · · , s2k , we will make use of the followinggeneralized reachability and observability matrices:

V =[(s1In − A)−1B · · · (sk In − A)−1B

], W =

[(sk+1In − A∗)−1C∗ · · · (s2k In − A∗)−1C∗

].

Lemma

Assuming that det W∗V 6= 0, the projected system Σ where V = V andW = W(V∗W)−1 interpolates the transfer function of Σ at the 2k points si .

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 27 / 38

Page 28: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Rational interpolation by Krylov projection Interpolation by projection

Remarks(a)The same procedure as above can be used to approximate implicitsystems, i.e., systems that are given in a generalized formEx(t) = Ax(t) + Bu(t), y(t) = Cx(t), where E may be singular. The reducedsystem is given by

E = W∗EV, A = W∗AV, B = W∗B, C = CV,

where

W∗ =

C(sk+1E− A)−1

...C(s2k E− A)−1

, V =[(s1E− A)−1B · · · (sk E− A)−1B

](b) Sylvester equations and projectors. The solution of an appropriateSylvester equation AX + XH + BG = 0, provides a projector that interpolatesthe original system C, A, B at minus the eigenvalues of H. Therefore theprojectors above can be obtained by solving Sylvester equations.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 28 / 38

Page 29: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 29 / 38

Page 30: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

Choice of Krylov projection points:Optimal H2 model reduction

Recall: the H2 norm of a stable system is:

‖Σ‖H2 =

(∫ +∞

−∞h2(t)dt

)1/2

where h(t) = CeAtB, t ≥ 0, is the impulse response of Σ.

Goal: construct a Krylov projector such that

Σk = arg mindeg(Σ) = rΣ : stable

∥∥∥Σ− Σ∥∥∥H2

=

(∫ +∞

−∞(h− h)2(t)dt

)1/2

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 30 / 38

Page 31: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

First-order necessary optimality conditions

Let (A, B, C) solve the optimal H2 problem and let λi denote the eigenvaluesof A. The necessary conditions are

H(−λ∗i ) = H(−λ∗i ) anddds

H(s)

∣∣∣∣s=−λ∗i

=dds

H(s)

∣∣∣∣s=−λ∗i

Thus the reduced system has to match the first two moments of the originalsystem at the mirror images of the eigenvalues of A.

The H2 norm: if H(s) =∑n

k=1φk

s−λk⇒ ‖H‖2

H2=

n∑k=1

ck H(−λ∗i )

Corollary. With H(s) =∑r

k=1φk

s−λk, the H2 norm of the error system, is

J =∥∥∥H− H

∥∥∥2

H2=

n∑i=1

φi

[H(−λi )− H(−λi )

]+

r∑j=1

φj

[H(−λj )− H(−λj )

]

Conclusion. The H2 error is due to the mismatch of the transfer functionsH− H at the mirror images of the full-order and reduced system poles λi , λi .

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 31 / 38

Page 32: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

An iterative algorithm

Let the system obtained after the (j − 1)st step be (Cj−1, Aj−1, Bj−1), whereAj−1 ∈ Rk×k , Bj−1, C∗j−1 ∈ Rk . At the j th step the system is obtained as follows

Aj = (W∗j Vj)

−1W∗j AVj , Bj = (W∗

j Vj)−1W∗

j B, Cj = CVj ,

whereVj =

[(λ1I− A)−1B, · · · , (λk I− A)−1B

],

W∗j =

[C(λ1I− A)−1, · · · , C(λk I− A)−1

],

and: −λ1, · · · ,−λk ∈ σ(Aj−1),i.e., −λi are the eigenvalues of the (j − 1)st iterate Aj−1.

The Newton step: can be computed explicitly

λ(k)1

λ(k)2...

←−

λ(k)1

λ(k)2...

− J−1

λ

(k−1)1

λ(k−1)2

.

.

.

⇒ local convergence guaranteed.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 32 / 38

Page 33: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

An iterative rational Krylov algorithm (IRKA)

The proposed algorithm produces a reduced order model H(s) that satisfiesthe interpolation-based conditions, i.e.

H(−λ∗i ) = H(−λ

∗i ) and

d

dsH(s)

∣∣∣∣s=−λ∗i

=d

dsH(s)

∣∣∣∣s=−λ∗i

1 Make an initial selection of σi , for i = 1, . . . , k

2 W = [(σ1I− A∗)−1C∗, · · · , (σk I− A∗)−1C∗]

3 V = [(σ1I− A)−1B, · · · , (σk I− A)−1B]

4 while (not converged)

A = (W∗V)−1W∗AV,σi ←− −λi(A) + Newton correction, i = 1, . . . , kW = [(σ1I− A∗)−1C∗, · · · , (σk I− A∗)−1C∗]V = [(σ1I− A)−1B, · · · , (σk I− A)−1B]

5 A = (W∗V)−1W∗AV, B = (W∗V)−1W∗B, C = CV

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 33 / 38

Page 34: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

Moderate-dimensional example

total system variables n = 902, independent variables dim = 599, reduceddimension k = 21reduced model captures dominant modes

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 3−40

−38

−36

−34

−32

−30

−28

−26

−24

−22

−20

Frequency x 108(rad/s)

Sin

gu

lar

valu

es (

db

)

Frequency responseSpectral zero method with SADPA

n=902 dim=599 k=21

Original

Reduced(SZM)

−0.0136 −0.0135 −0.0135 −0.0134 −0.0134−3

−2

−1

0

1

2

3

Dominant spectral zerosTheoretical and found with SADPA

Real

Imag

Spz: original

Spz: dominant

Spz: SADPA computed

R

C RC

L

C RC

RL

RCC

RLL-

?

- -

??? ?

-

?

. . .

?

u

y

1

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 34 / 38

Page 35: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Choice of Krylov projection points: OptimalH2 model reduction

H∞ and H2 error norms

Relative norms of the error systems

Reduction Methodn = 902, dim = 599, k = 21 H∞ H2

PRIMA 1.4775 -Spectral Zero Method with SADPA 0.9628 0.841

Optimal H2 0.5943 0.4621Balanced truncation (BT) 0.9393 0.6466

Riccati Balanced Truncation (PRBT) 0.9617 0.8164

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 35 / 38

Page 36: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Summary: Lectures II and III

Outline

1 Krylov approximation methods

2 The Arnoldi and the Lanczos proceduresThe Arnoldi procedureThe Lanczos procedureAn example

3 Krylov methods and moment matchingRemarks

4 Rational interpolation by Krylov projectionRealization by projectionInterpolation by projection

5 Choice of Krylov projection points: Optimal H2 model reduction

6 Summary: Lectures II and III

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 36 / 38

Page 37: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Summary: Lectures II and III

Approximation methods: SummaryPPPPPPPPq

������

Krylov

• Realization• Interpolation• Lanczos• Arnoldi

SVD

@@@R

��

Nonlinear systems Linear systems• POD methods • Balanced truncation• Empirical Gramians • Hankel approximation@

@@

@R�

��

Krylov/SVD Methods

��

r@

@R

rProperties

• numerical efficiency

• n� 103

• choice of matching moments

Properties

• Stability

• Error bound

• n ≈ 103

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 37 / 38

Page 38: Model reduction of large-scale dynamical systemsaca/aca_monopoli_III.pdf · Model reduction of large-scale dynamical systems Lecture III: Krylov approximation and rational interpolation

Summary: Lectures II and III

Complexity considerations

• Dense problemsMajor cost Balanced Truncation:Compute gramians≈ 30N3 (eigenvalue decomp.)

Perform balancing≈ 25N3 (sing. value decomp.)

Rational Krylov approximation:

Decompose (A− σi E) for k points≈ 23 kN3

Remark : Iterations (Sign, Smith) can accelerate the computation of gramians (esp. on parallel machines)

• Approximate and/or sparse decompositionsMajor cost Balanced Truncation:Compute gramians≈ c1αkN

Perform balancing O(n3)

Rational Krylov approximation:

Iterative solves for (A− σi E)x = b≈ c2kαN, where k = number of expansion points; α = average number of non-zero

elements per row in A, E.

Thanos Antoulas (Rice U. & Jacobs U.) Reduction of large-scale systems 38 / 38