happy birthday, volker!happy birthday, volker! david s. watkins department of mathematics washington...

Happy Birthday, Volker!

David S. Watkins

Department of MathematicsWashington State University

Berlin, May, 2015

David S. Watkins Happy Birthday, Volker!

Contribution from Michael Overton

Volker Mehrmann

Author or coauthor of more than 160 scientific articles

Author or coauthor of 5 monographs/textbooks

Coeditor of 5 books

Co-editor-in-chief of Linear Algebra and its Applications

Member of the German Academy of Engineering

Recent President of GAMM

Recent Director of MATHEON

An outstanding scientist with broad interests and knowledge

A superlatively nice person who always has time for everyone!

A good person to be with in a jeep . . .when an elephant is charging it!!

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

Volker Mehrmann

Coeditor of 5 books

A good person to be with in a jeep . . .

when an elephant is charging it!!

Volker Mehrmann

Coeditor of 5 books

Kaziranga National Park (Thanks to Shreemayee Bora)

and now moving back in time . . .

A question

An engineer, an algebraist, and Volker . . .

. . . walk into a bar.

Engineer: I have this interesting problem where I need to findthe roots of polynomials of high degree.

Algebraist: What a coincidence! I have a problem just likethat.

Question: What does Volker say?

A question

An engineer,

an algebraist, and Volker . . .

A question

An engineer, an algebraist,

and Volker . . .

A question

Engineer:

I have this interesting problem where I need to findthe roots of polynomials of high degree.

A question

Algebraist:

What a coincidence! I have a problem just likethat.

A question

Question:

What does Volker say?

A question

Show me the eigenvalue problem!

Nevertheless, there is a demand for the product.

MATLAB roots (companion matrix)

Chebfun roots (colleague matrix)

I never thought I would get caught up in this racket,

. . . but somehow I got sucked in.

... and we’ve done some good stuff.

Nevertheless,

there is a demand for the product.

MATLAB roots

(companion matrix)

Chebfun roots

(colleague matrix)

Our International Research Group

This is joint work with

Jared Aurentz (Oxford)

Thomas Mach (KU Leuven)

Raf Vandebril (KU Leuven)

MATLAB

p(x) = xn + an−1xn−1 + an−2x

n−2 + · · ·+ a0 = 0

monic polynomial

companion matrix

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

. . . 0 −an−2

1 −an−1

balance, then . . .

. . . get the zeros of p by computing the eigenvalues.

This is not always the best thing to do.

MATLAB

p(x) = xn + an−1xn−1 + an−2x

n−2 + · · ·+ a0 = 0

monic polynomial

companion matrix

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

. . . 0 −an−2

1 −an−1

balance, then . . .

MATLAB

p(x) = xn + an−1xn−1 + an−2x

n−2 + · · ·+ a0 = 0

monic polynomial

companion matrix

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

. . . 0 −an−2

1 −an−1

balance, then . . .

MATLAB

p(x) = xn + an−1xn−1 + an−2x

n−2 + · · ·+ a0 = 0

monic polynomial

companion matrix

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

. . . 0 −an−2

1 −an−1

balance, then . . .

Chebfun

p(x) = Tn(x) + bn−1Tn−1(x) + · · · b0T0(x)

Chebyshev polynomials

colleague matrix

This is sometimes better.

Chebfun

p(x) = Tn(x) + bn−1Tn−1(x) + · · · b0T0(x)

colleague matrix

Chebfun

p(x) = Tn(x) + bn−1Tn−1(x) + · · · b0T0(x)

colleague matrix

What we’ve been doing

companion matrix or

companion pencil

p(x) = anxn + an−1x

n−1 + an−2xn−2 + · · ·+ a0

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

0 −an−2

1 −an−1

− λ

1 · · · 0 0

1 · · · 0 0. . .

......

. . . 1 0an

. . . and variants.

Today we restrict attention to the companion matrix.

companion matrix or

companion pencil

n−1 + an−2xn−2 + · · ·+ a0

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

0 −an−2

1 −an−1

− λ

1 · · · 0 0

1 · · · 0 0. . .

......

. . . 1 0an

. . . and variants.

companion matrix or

companion pencil

n−1 + an−2xn−2 + · · ·+ a0

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

0 −an−2

1 −an−1

− λ

1 · · · 0 0

1 · · · 0 0. . .

......

. . . 1 0an

. . . and variants.

companion matrix or

companion pencil

n−1 + an−2xn−2 + · · ·+ a0

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

0 −an−2

1 −an−1

− λ

1 · · · 0 0

1 · · · 0 0. . .

......

. . . 1 0an

. . . and variants.

companion matrix or

companion pencil

n−1 + an−2xn−2 + · · ·+ a0

0 · · · 0 −a01 0 · · · 0 −a1

1. . .

......

0 −an−2

1 −an−1

− λ

1 · · · 0 0

1 · · · 0 0. . .

......

. . . 1 0an

. . . and variants.

Cost of solving companion eigenvalue problem

If structure not exploited:

O(n2) storage, O(n3) flopsFrancis’s implicitly-shifted QR algorithm

If structure exploited:

O(n) storage, O(n2) flopsdata-sparse representation + Francis’s algorithmSeveral methods proposed

O(n) storage, O(n2) flopsdata-sparse representation + Francis’s algorithm

Several methods proposed

Some of the Competitors

Chandrasekaran, Gu, Xia, Zhu (2007)

Bini, Boito, Eidelman, Gemignani, Gohberg (2010)

Boito, Eidelman, Gemignani, Gohberg (2012)

Fortran codes available

evidence of backward stability

quasiseparable generator representation

We will do something else.

Our Contribution

We present

Yet another O(n) representation

Francis algorithm in O(n) flops/iteration

Fortran codes (we’re faster)

normwise backward stable (We can prove it.)

Our Contribution

We present

Our Contribution

We present

Our Contribution

We present

Structure

Companion matrix is unitary-plus-rank-one0 · · · 0 e iθ

1 0. . .

...1 0

0 · · · 0 −e iθ − a00 0 −a1...

......

0 · · · 0 −an−1

preserved by unitary similarities

Companion matrix is also upper Hessenberg.

preserved by Francis algorithm

We exploit this structure.

Structure

1 0. . .

...1 0

0 · · · 0 −e iθ − a00 0 −a1...

......

0 · · · 0 −an−1

Structure

1 0. . .

...1 0

0 · · · 0 −e iθ − a00 0 −a1...

......

0 · · · 0 −an−1

Structure

1 0. . .

...1 0

0 · · · 0 −e iθ − a00 0 −a1...

......

0 · · · 0 −an−1

Structure

1 0. . .

...1 0

0 · · · 0 −e iθ − a00 0 −a1...

......

0 · · · 0 −an−1

Structure

A = QR

Q is upper Hessenberg and unitary.

R is upper triangular and unitary-plus-rank-one.

We do this too.

Structure

A = QR

We do this too.

Structure

A = QR

We do this too.

The Unitary Part

x x x xx x x x

x x xx x

x xx x

1x xx x

x xx x

Q =��

��

O(n) storage

The Unitary Part

x x x xx x x x

x x xx x

x xx x

1x xx x

x xx x

Q =��

��

O(n) storage

The Unitary Part

x x x xx x x x

x x xx x

x xx x

1x xx x

x xx x

Q =��

��

O(n) storage

The Upper Triangular Part

R = U + xyT unitary-plus-rank-one, so

R has quasiseparable rank 2.

x · · · x x · · · x. . .

......

...x x · · · x

x · · · x. . .

quasiseparable generator representation (O(n) storage)

Chandrasekaran et. al. exploit this structure.

We do it differently.

x · · · x x · · · x. . .

......

...x x · · · x

x · · · x. . .

x · · · x x · · · x. . .

......

...x x · · · x

x · · · x. . .

x · · · x x · · · x. . .

......

...x x · · · x

x · · · x. . .

x · · · x x · · · x. . .

......

...x x · · · x

x · · · x. . .

Our Representation

Add a row/column for extra wiggle room

0 −a0 11 −a1 0

. . ....

...1 −an−1 0

Extra zero root can be deflated immediately.

A = QR, where

0 ±1 01 0 0

. . ....

...1 0 0

1 −a1 0

. . ....

...1 −an−1 0±a0 ∓1

Our Representation

Add a row/column for extra wiggle room

0 −a0 11 −a1 0

. . ....

...1 −an−1 0

Extra zero root can be deflated immediately.

A = QR, where

0 ±1 01 0 0

. . ....

...1 0 0

1 −a1 0

. . ....

...1 −an−1 0±a0 ∓1

Our Representation

0 ±1 01 0 0

. . ....

...1 0 0

Q is stored in factored form

Q =��

��

Q = Q1Q2 · · ·Qn−1

Our Representation

0 ±1 01 0 0

. . ....

...1 0 0

Q is stored in factored form

Q =��

��

Q = Q1Q2 · · ·Qn−1

Our Representation

1 −a1 0

. . ....

...1 −an−1 0±a0 ∓1

R is unitary-plus-rank-one:

1 0 0. . .

......

1 0 00 ∓1

0 −a1 0

. . ....

...0 −an−1 0±a0 0

∓1 0

Representation of R

R = U + xyT , where

...−an−1

±a0∓1

0 · · · 0 1 0]

Next step: Roll up x .

Representation of R

R = U + xyT , where

...−an−1

±a0∓1

0 · · · 0 1 0]

Representation of R

R = U + xyT , where

...−an−1

±a0∓1

0 · · · 0 1 0]

Representation of R

C1 · · ·Cn−1Cnx = αe1 (w.l.g. α = 1)

Representation of R

��

C1 · · ·Cn−1Cnx = αe1 (w.l.g. α = 1)

Representation of R

��

C1 · · ·Cn−1Cnx = αe1 (w.l.g. α = 1)

Representation of R

��

C1 · · ·Cn−1Cnx = αe1 (w.l.g. α = 1)

Representation of R

��

C1 · · ·Cn−1Cnx = αe1 (w.l.g. α = 1)

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = U + xyT = U + C ∗e1yT = C ∗(CU + e1y

R = C ∗(B + e1yT )

B is upper Hessenberg (and unitary) so B = B1 · · ·Bn.

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Bonus: Redundancy! No need to keep track of y .

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

B is upper Hessenberg (and unitary)

so B = B1 · · ·Bn.

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of R

C1 · · ·Cn−1Cnx = e1

Cx = e1

C ∗e1 = x

R = C ∗(B + e1yT )

R = C ∗(B + e1yT ) = C ∗

n · · ·C ∗1 (B1 · · ·Bn + e1y

O(n) storage

Representation of A

Altogether we have

A = QR = Q C ∗ (B + e1yT )

A = Q1 · · ·Qn−1 C∗n · · ·C ∗

1 (B1 · · ·Bn + e1yT )

��

+ · · ·

Francis Iterations

We have complex single-shift code . . .

real double-shift code.

We describe single-shift case for simplicity.

ignoring rank-one part . . .

��

Francis Iterations

��

Francis Iterations

��

Two Basic Operations

Two basic operations:

Fusion� �� ⇒ ��

Turnover (aka shift through, Givens swap, . . . )

� ��

�� ⇔

��

� ��

�� ⇔

��

� ��

�� ⇔

��

The Bulge Chase

��

The Bulge Chase

� ��

��

� ��

��

The Bulge Chase

� ��

��

� ��

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

� ��

��

The Bulge Chase

� ��

��

The Bulge Chase

� ��

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

The Bulge Chase

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

� ��

��

The Bulge Chase

��

The Bulge Chase

��

iteration complete!

Cost: 3n turnovers/iteration, so O(n) flops/iteration

Double-shift iteration is similar.

(Chase two core transformations instead of one.)

iteration complete!

See our papers for . . .

Paper to appear in SIMAX has

. . . timings,

. . . accuracy comparisons,

. . . backward error analysis.

Paper on companion pencils is in progress.

. . . timings,

Summary

We have a new fast method for companion eigenvalueproblems

and unitary-plus-rank-one matrices (or pencils) in general.

Method is normwise backward stable, accurate,

and faster than other fast methods.

Thank you for your attention.

Summary

happy birthday, volker!happy birthday, volker! david s. watkins department of mathematics washington...

Documents