Conversion methods for improving structuralanalysis of differential-algebraic equation systems

Guangning Tan

PhD defense, 3 August 2016

Simple pendulum

ODE formulation

θ′′ = −G

`sin θ

I Need initial values (IVs) θ, θ′

I 2 degrees of freedom (DOF)

(x,y)

L

x

y

G

θθ

l

DAE formulation

0 = f1 = x ′′ + xλ

0 = f2 = y ′′ + yλ− G

0 = f3 = x2 + y2 − `2 0 = f ′3 = 2(xx ′ + yy ′)

I IVs (x , y) should satisfy f3; then (x ′, y ′) satisfy f ′3PhD defense c©2016 Guangning Tan tgn3000.com 2

Structural analysis (SA) of DAEs

The Σ-method by John Pryce

I determinesI index

I DOF

I constraints

I variables/derivatives that need consistent IVs

I equivalent to Pantelides’s algorithm

I succeeds on many practical DAE problems of interest

PhD defense c©2016 Guangning Tan tgn3000.com 3

0 = f1 = x ′′ + xλ

0 = f2 = y ′′ + yλ− G

0 = f3 = x2 + y2 − `2

Structural analysis says

I solve f3 for x , y ,

I solve f ′3 for x ′, y ′, and

I solve with nonsingularJacobian linear systems(f

(k)1 , f

(k)2 , f

(2+k)3

)for

(x (2+k), y (2+k), λ(k)

)counting up from k = 0

x y λ ci[ ]f 1 2 0• 0

Σ = f 2 2• 0 0

f 3 0• 0 2

dj 2 2 0

x ′′ y ′′ λ[ ]f 1 1 x

J = f 2 1 yf ′′3 2x 2y

PhD defense c©2016 Guangning Tan tgn3000.com 4

Limitations of structural analysis

I Structural analysis can fail on simple, solvable DAEs withsingular Jacobian matrices

“In such cases the algorithm terminates without detecting allthe equation subsets which ought to be differentiated.”[Pantelides, 1988]

I Even in success case, it may overestimate index at arbitrarilyhigh number

“Pantelides’ algorithm applied to DAEs of index 1 mayperform an arbitrarily high number of iterations anddifferentiations.” [Reissig, Martinson, Barton, 2000]

PhD defense c©2016 Guangning Tan tgn3000.com 5

Motivation

I I want to dissolve DAE theorists’ concerns and doubt towardstructural analysis

I Make this useful tool more reliable

This is not a trivial task

I “Wrong structural analysis. ... They pose another interestingand difficult task, more in computer science than in numericalanalysis. ... What are the principles and methods involved,and can they be at least partially automated?”

[Nedialkov & Pryce, 2007]

PhD defense c©2016 Guangning Tan tgn3000.com 6

Contributions

“A DAE should not be formulated to exhibit more degrees offreedom than the underlying problem has.”

I Give insight into structural analysis’s failures that were notwell understood before

I Develop two conversion methods that systematicallyreformulate DAEs on which structural analysis fails intoequivalent success cases

I Show how to improve efficiency by exploiting block triangularform of a DAE

I Automate a conversion process in a computer algebra system

PhD defense c©2016 Guangning Tan tgn3000.com 7

Modified pendulum I

I Multiply pendulum by nonsingular

[1 1

1 11 1

]

0 = f1 = (x ′′ + xλ) + (y ′′ + yλ− G )

0 = f2 = (y ′′ + yλ− G ) + (x2 + y2 − `2)

0 = f3 = (x ′′ + xλ) + (x2 + y2 − `2)

x y λ ci f 1 2• 2 0 0

Σ = f 2 0 2• 0 0

f 3 2 0 0• 0

dj 2 2 0

dj − ci > σij ≥ 0

x ′′ y ′′ λ[ ]f 1 1 1 x + y

J = f 2 1 yf 3 1 x

det J ≡ 0

I Val Σ = 4 > 2 = DOF of pendulum

I SA says “give arbitrary x , x ′, y , y ′ to start integration”, wrong!

PhD defense c©2016 Guangning Tan tgn3000.com 8

Conversion method I: linear combination (LC)

Replace an fi by a combination of equations

J =∂(f1, f2, f3)

∂(x ′′, y ′′, λ)=

x ′′ y ′′ λ[ ]f 1 1 1 x + yf 2 1 yf 3 1 x

I Find u = [−1, 1, 1]T ∈ coker J

I u does not depend on x ′′, y ′′, λ, hence nor does

f ← u1f1 + u2f2 + u3f3 = 2(x2 + y2 − `2)

I ∂f /∂(x ′′, y ′′, λ) = uTJ = 0T

I Replace an fi by f ⇒ decrease Val Σ

PhD defense c©2016 Guangning Tan tgn3000.com 9

If we replace f1 = (x ′′ + xλ) + (y ′′ + yλ− G )

by f = 2(x2 + y2 − `2)

x y λ ci f 0• 0 2

Σ = f 2 0 2• 0 0

f 3 2 0 0• 0

dj 2 2 0

x ′′ y ′′ λ[ ]f′′ 4x 4y

J = f 2 1 yf 3 1 x

I Val Σ = 2, det J = 4`2 6= 0⇒ success

I u1 = −1⇒ we can recover f1 =1

u1

(f − u2f2 − u3f3

)I Now structural analysis says

“to start integration, give x , x ′, y , y ′ subject to f , f′

= 0”

PhD defense c©2016 Guangning Tan tgn3000.com 10

Ideas behind conversion methods

I In a success case, structural analysis indicates how to

“determine N derivatives that satisfy M constraints”

where DOF =N −M of a DAE

I Failing on a solvable DAE, structural analysis gives wrongstructural information, obviously

I Provided conditions satisfied for a conversion method, we canreduce Val Σ

I If the Σ-method succeeds on a converted DAE, then DOF ofthe original problem must have been overestimated—itappears freer than it should

PhD defense c©2016 Guangning Tan tgn3000.com 11

Modified pendulum II

Perform a linear transformation on x , y , λ xyλ

=

1 11 1

1 1

z1

z2

z3

0 = f1 = (z1 + z2)′′ + (z1 + z2)(z3 + z1)

0 = f2 = (z2 + z3)′′ + (z2 + z3)(z3 + z1)− G

0 = f3 = (z1 + z2)2 + (z2 + z3)2 − `2

z1 z2 z3 ci f 1 2• 2 0 0

Σ = f 2 0 2• 2 0

f 3 0 0 0• 2

dj 2 2 2

z ′′1 z ′′2 z ′′3[ ]f 1 1 1

J = f 2 1 1f ′′3 2x 2(x + y) 2y

det(J) ≡ 0

I Val Σ = 4 > 2 = DOF of pendulum

PhD defense c©2016 Guangning Tan tgn3000.com 12

Conversion method II: expression substitution (ES)

Substitute new variables for common subexpressions

z ′′1 z ′′2 z ′′3[ ]f 1 1 1

J = f 2 1 1f ′′3 2(z1 + z2) 2(z1 + 2z2 + z3) 2(z2 + z3)

I Find v = [1,−1, 1]T ∈ ker J

I Common subexpressions are replaced, according to

0 = g1 = −w1 +

(z1 −

v1

v3z3

)= −w1 + (z1 − z3)

0 = g2 = −w2 +

(z2 −

v2

v3z3

)= −w2 + (z2 + z3)

I z1 = w1 + z3, z2 = w2 − z3

PhD defense c©2016 Guangning Tan tgn3000.com 13

I Replace z1 by w1 + z3, replace z2 by w2 − z3

0 = f 1 = (w1 + w2)′′ + (w1 + w2)(w1 + 2z3)

0 = f 2 = w ′′2 + w2(w1 + 2z3)− G

0 = f 3 = (w1 + w2)2 + w22 − `2

0 = g1 = −w1 + (z1 − z3)

0 = g2 = −w2 + (z2 + z3)

Σ =

z1 z2 z3 w1 w2 ci

f 1 0 2• 2 0

f 2 0• 0 2 0

f 3 0 0• 2

g1 0• 0 0 0

g2 0• 0 0 0

dj 0 0 0 2 2

J =

z1 z2 z3 w ′′1 w ′′2

f 1 2α 1 1

f 2 2w2 1

f′′3 2α 2(α + w2)g1 1 −1g2 1 1

α = w1 + w2

I Val Σ = 2 = DOF of pendulum, det(J) = −4`2 6= 0⇒ success

PhD defense c©2016 Guangning Tan tgn3000.com 14

Index overestimation problem on Reissig’s DAEs

I Despite nonsingular J, structural analysis can overestimate index

I Mass matrix has nonzero linearly dependent rows

0 = f1 = x1 + x ′2 + x ′3 + q1(t)

0 = f2 = x2 + x ′2 + x ′3 + q2(t)

0 = f3 = x3 + x ′4 + x ′5 + q3(t)

0 = f4 = x4 + x ′4 + x ′5 + q4(t)

0 = f5 = x5 + q5(t)

Σ =

x1 x2 x3 x4 x5 ci

f1 0• 1 1 0

f2 1• 1 0

f3 0• 1 1 1

f4 1• 1 1

f5 0• 2

dj 0 1 1 2 2

J =

x1 x ′2 x ′3 x ′′4 x ′′5

f1 1 1 1f2 1 1f ′3 1 1 1f ′4 1 1f ′′5 1

det(J) = 1

I Size n = 2k − 1, differentiation index 1, structural index kPhD defense c©2016 Guangning Tan tgn3000.com 15

I Replace f1 by f 1 = f1 − f2, replace f3 by f 3 = f3 − f4

0 = f 1 = x1 − x2 + q1(t)− q2(t)

0 = f2 = x2 + x ′2 + x ′3 + q2(t)

0 = f 3 = x3 − x4 + q3(t)− q4(t)

0 = f4 = x4 + x ′4 + x ′5 + q4(t)

0 = f5 = x5 + q5(t)

Σ =

x1 x2 x3 x4 x5 ci

f 1 0• 0 1

f2 1• 1 0

f 3 0• 0 1

f4 1• 1 0

f5 0• 1

dj 1 1 1 1 1

J =

x ′1 x ′2 x ′3 x ′4 x ′5

f′1 1 −1

f2 1 1

f′3 1 −1

f4 1 1f ′5 1

det J = 1

I correct structural index 1

PhD defense c©2016 Guangning Tan tgn3000.com 16

Wrong structural analysis remedied so far

I DAEs arbitrarily constructed to be failure cases

I DAEs in literature

DAE Reference Method Result

Robot arm Campbell et al. (1995) LC, ES correct index &

nonsingular J

Transistor amplifier}Mazzia et al. (2003)

LC, ES

Ring modulator LC, ES

Coupled DAE Scholz et al. (2013) LC

Reissig’s DAEs1 Reissig et al. (1999) LC, ES

1Structural analysis succeeds but overestimates index

PhD defense c©2016 Guangning Tan tgn3000.com 17

Conclusions

I Provide insight into structural analysis’s failure

I Identically (but not structurally) singular Jacobian

I Overestimation of DOF of the underlying problem

I Convert to an equivalent problem by reducing Val Σ

I Linear combination of equations

I Expression substitutions by new variables

I Improve efficiency by exploiting a block triangular form

I Can automate conversions in a computer algebra system

PhD defense c©2016 Guangning Tan tgn3000.com 18

References

G. Tan†, N. Nedialkov, J. Pryce

I †Conversion methods for improving structural analysis of DAEs.Submitted to BIT Numerical Mathematics arXiv:1608.06691

I †Conversion methods, block triangularization, and structuralanalysis of DAEs. Submitted to BIT Numerical MathematicsarXiv:1608.06693

I †Symbolic-numeric methods for improving structural analysis ofDAEs. The 2015 AMMCS conference proceedings DOI10.1007/978-3-319-30379-6 68

I DAESA—a Matlab tool for structural analysis of DAEs: Theory &Algorithm. ACM TOMS, 41 (2015)

PhD defense c©2016 Guangning Tan tgn3000.com 19

Robot arm Ring modulator Coupled system Transistor amplifier

More examples

Robot arm

Ring modulator

Coupled system

Transistor amplifier

PhD defense c©2016 Guangning Tan tgn3000.com 20

Robot arm Ring modulator Coupled system Transistor amplifier

Campbell & Griepentrog’s Robot Arm

0 = f3 = − x ′′3 − 2c(x3)(x ′1 + x ′3)2 − d(x3)x ′21

+ (2x3 − x2)(a(x3) − 9b(x3)

)− 2x ′21 c(x3) − d(x3)

(x ′1 + x ′3

)2

−(a(x3) + b(x3)

)(x4 − x5)

0 = f1 = − x ′′1 + 2c(x3)(x ′1 + x ′3)2 + d(x3)x ′21

+ (2x3 − x2)(a(x3) + 2b(x3)

)+a(x3)(x4 − x5)

2•

0

0

0

0•

0

0

0

0•

1

1

2

0•

0

1

2

1

0

0•

f2

f3

f1

f4

f5

x2 x4 x5 x1 x3 ci

0

0

0

2

2

dj 2 0 0 2 2 ValΣ = 2

I det J ≡ 0⇒ failure

I Singular J22 =1

2− cos2 x3

[−2− cos x3 2 + cos x3

2 −2

]I Block 2: equations f3, f1 in variables x4, x5

PhD defense c©2016 Guangning Tan tgn3000.com 21

Robot arm Ring modulator Coupled system Transistor amplifier

Block LC method on robot arm

2•

0

0

0

0•

0

0

0

0•

1

1

2

0•

0

1

2

1

0

0•

f2

f3

f1

f4

f5

x2 x4 x5 x1 x3 ci

0

0

0

2

2

dj 2 0 0 2 2 ValΣ = 2

0•

0

0

0•

2

0

0•

1

2

2

0•

0

1

1

2

0

0•

f2

f1

f 3

f4

f5

x4 x5 x2 x1 x3 ci

0

0

2

4

4

dj 0 0 2 4 4 ValΣ = 0

I u = (2, 2 + cos x3)T ∈ coker J22

I x4, x5 not in u

I f 3 ← 2f3 + (2 + cos x3)f1

I Val Σ = 0 < 2 = Val Σ, nonsingular J⇒ success

PhD defense c©2016 Guangning Tan tgn3000.com 22

Robot arm Ring modulator Coupled system Transistor amplifier

Block ES method on robot arm

2•

0

0

0

0•

0

0

0

0•

1

1

2

0•

0

1

2

1

0

0•

f2

f3

f1

f4

f5

x2 x4 x5 x1 x3 ci

0

0

0

2

2

dj 2 0 0 2 2 ValΣ = 2

0•

0

0

0• 2

0•

0

0

0

0•

1

1

2

0•

0

1

2

1

0

0•

f6

f2

f 3

f 1

f4

f5

x4 x5 x2 x6 x1 x3 ci

0

0

2

2

4

4

dj 0 0 2 2 4 4 ValΣ = 0

I Constant v = (1, 1)T ∈ ker J22

I Introduce x6 and 0 = f6 = −x6 + (x4 − x5)

I Replace x4 by x6 + x5 in f3, f1

I Val Σ = 0 < 2 = Val Σ, nonsingular J⇒ success

PhD defense c©2016 Guangning Tan tgn3000.com 23

Robot arm Ring modulator Coupled system Transistor amplifier

Ring modulator DAE

1•

0

0

0

0

1•

0

0

0

0

1•

0

0

0

0

0

1•

0

0

1•

0

0

1•

0

0

1•

0

0

0

0•

0

0

0

0

0•

0

0

0

0

0

0

0•

0

0

0

0

0•

0

1•

0

1•

0

1•

0

1•

f1

f2

f7

f13

f11

f12

f10

f3

f4

f5

f6

f8

f9

f14

f15

x1 x2 x7 x13 x11 x12 x10 x3 x4 x5 x6 x8 x9 x14 x15 ci

0

0

0

0

0

0

0

0

0

0

0

0

0

0

0

dj 1 1 1 1 1 1 1 0 0 0 0 1 1 1 1 ValΣ = 11

I Val Σ = 11, det J = 0

I 4× 4 singular block

PhD defense c©2016 Guangning Tan tgn3000.com 24

Robot arm Ring modulator Coupled system Transistor amplifier

Block LC method on ring modulator

1•

0

0

0

0

1•

0

0

0

0

1•

0

0

0

0

0•

0

0

0

0•

0

0

0

0

0

0•

0

0

0

0•

0

0

1

0•

0

0

0

1•

0

0

0

1•

0

0

0

1•

0

1•

0

1•

0

1•

0

1•

f1

f2

f7

f10

f5

f4

f6

f 3

f11

f12

f13

f8

f9

f14

f15

x1 x2 x7 x3 x4 x5 x6 x10 x11 x12 x13 x8 x9 x14 x15 ci

0

0

0

0

0

0

0

1

0

0

0

0

0

0

0

dj 1 1 1 0 0 0 0 1 1 1 1 1 1 1 1 ValΣ = 10

I (1,−1, 1,−1)T ∈ coker J88

I f 3 ← f3 − f4 + f5 − f6

I Val Σ = 10 < 11 = Val Σ

I det J 6= 0⇒ success

I 4× 4 singular block mergeswith four 1× 1 blocks

PhD defense c©2016 Guangning Tan tgn3000.com 25

Robot arm Ring modulator Coupled system Transistor amplifier

Block ES method on ring modulator

1•

0

0

0

0

1•

0

0

0

0

0•

0

0

0

0

0•

0

0

0

0•

0

0

0•

0

1

0•

0

0

0

0

1•

0

0

1•

0

0

1•

0

0

0

1•

0

0

0•

0

0

0

0•

0

0

0

0

0•

0

1•

0

1•

0

1•

0

1•

f1

f2

f17

f16

f7

f18

f 6

f10

f11

f12

f13

f 5

f 4

f 3

f8

f9

f14

f15

x1 x2 x3 x4 x5 x6 x7 x10 x11 x12 x13 x16 x17 x18 x8 x9 x14 x15 ci

0

0

0

0

0

0

1

0

0

0

0

1

1

1

0

0

0

0

dj 1 1 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 ValΣ = 10

I (−1, 1,−1, 1)T ∈ ker J88

I 0 = f16 = −x16 + (x4 + x3)

0 = f17 = −x17 + (x5 − x3)

0 = f18 = −x18 + (x6 + x3)

I Val Σ = 10 < 11 = Val Σ

I det J 6= 0⇒ success

I One 12× 12 block andsix 1× 1 blocks

PhD defense c©2016 Guangning Tan tgn3000.com 26

Robot arm Ring modulator Coupled system Transistor amplifier

Scholz and Steinbrecher’s coupled system

I Linear constant coefficient DAE of d-index 3 (S&S, 2013)

0 = f1 = −x ′1 + x3 − b1(t)

0 = f2 = −x ′2 + x4 − b2(t)

0 = f3 = x2 + x3 + x4 − b3(t)

0 = f4 = −x1 + x3 + x4 − b4(t)

I SA reports νS = 1, obviously wrong

I Apply block LC method

1•

0

1•

0

0

0•

0

0

0

0•

f1

f2

f3

f4

x1 x2 x3 x4 ci

0

0

0

0

dj 1 1 0 0 ValΣ0 = 2

1•

0

0

0•

1

0

0• 0

0•

f1

f 3

f4

f2

x1 x2 x3 x4 ci

0

1

0

0

dj 1 1 0 0 ValΣ1 = 1

f 3 = f4 − f3

0• 0

0• 1

0•

0

0

0•

f4

f2

f 3

f 1

x3 x4 x2 x1 ci

0

0

1

1

dj 0 0 1 1 ValΣ2 = 0

f 1 = −f1−f2+f′3+f4

I Val Σ0 = 2 > Val Σ1 = 1 > Val Σ2 = 0

I det J = 1⇒ success

PhD defense c©2016 Guangning Tan tgn3000.com 27

Robot arm Ring modulator Coupled system Transistor amplifier

Transistor amplifier DAE

1•

1

1

1•

1•

1

0

1

1•

0

0

0

1•

1•

1

0

1

1•

0

0

0

1•

f7

f8

f4

f5

f6

f1

f2

f3

x7 x8 x4 x5 x6 x1 x2 x3 ci

0

0

0

0

0

0

0

0

dj 1 1 1 1 1 1 1 1 ValΣ = 8

I Val Σ = 8, det J = 0

I Three 2× 2 singular blocks

(1, 1)T ∈ coker J11,

coker J22,

coker J44

PhD defense c©2016 Guangning Tan tgn3000.com 28

Robot arm Ring modulator Coupled system Transistor amplifier

Block LC method on transistor amplifier

0•

1

0

1•

0•

1

0

0

1•

0

0

0

0

1•

0•

1

0

0

1•

0

0

0

0

1•

f 7

f8

f 4

f5

f6

f 1

f2

f3

x7 x8 x4 x5 x6 x1 x2 x3 ci

1

0

1

0

0

1

0

0

dj 1 1 1 1 1 1 1 1 ValΣ = 5

I f 7 ← f7 + f8

f 4 ← f4 + f5

f 1 ← f1 + f2

I Val Σ = 5 < 8 = Val Σ

I det J 6= 0⇒ success

PhD defense c©2016 Guangning Tan tgn3000.com 29

Robot arm Ring modulator Coupled system Transistor amplifier

Block ES method on transistor amplifier

1•

0

1

0•

0

0

0•

1•

0

0

1

0•

0

0

0

0•

0

0

1•

1•

0

0

1

0•

0

0

0

0•

0

0

1•

f9

f 8

f 7

f10

f 5

f 4

f6

f11

f 2

f 1

f3

x7 x8 x9 x4 x5 x10 x6 x1 x2 x11 x3 ci

0

1

1

0

1

1

0

0

1

1

0

dj 1 1 1 1 1 1 1 1 1 1 1 ValΣ = 5

I (1, 1)T ∈ ker J11,ker J22,ker J44

I 0 = f9 = − x9 + (x7 − x8)

0 = f10 = −x10 + (x4 − x5)

0 = f11 = −x11 + (x1 − x2)

I Val Σ = 5 < 8 = Val Σ

I det J 6= 0⇒ success

PhD defense c©2016 Guangning Tan tgn3000.com 30