Empirical Gramians and FriendsChristian Himpe

2016-10-11

CSC

Outline

1 My Research Interests

2 A Tour Through my Scientific Deeds

3 Lift-Off for MathEnergy

4 Scientific Sideshows

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 242

Outline

1 My Research Interests

2 A Tour Through my Scientific Deeds

3 Lift-Off for MathEnergy

4 Scientific Sideshows

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 242

Research Interests

in a Nutshell

Combined State and Parameter Reduction

for Nonlinear Input-Output Systems

Modeling (Complex) Networks

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 342

Notation

Nonlinear Input-Output System

x(t) = f (t x(t) u(t) θ)

y(t) = g(t x(t) u(t) θ)

x(0) = x0

Input Function u Rge0 rarr RM

State Trajectory x Rge0 rarr RN

Output Trajectory y Rge0 rarr RQ

Parameter θ isin RP

Vector-Field f Rge0 times RN times RM times RP rarr RN

Output Functional g Rge0 times RN times RM times RP rarr RQ

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 442

Combined Reduction

Reduced Order Model

xr (t) = fr (t xr (t) u(t) θr )

yr (t) = gr (t xr (t) u(t) θr )

xr (0) = xr 0

State Trajectory xr Rge0 rarr Rn

Output Trajectory yr Rge0 rarr RQ

Parameter θr isin Rp

Vector-Field fr Rge0 times Rn times RM times Rp rarr Rn

Output Functional gr Rge0 times Rn times RM times Rp rarr RQ

Reduced State Dimension n NReduced Parameter Dimension p P

Accuracy y(θ)minus yr (θr ) 1

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 542

Projection-Based Reduction

Projection-Based Reduced Order Model

xr (t) = Vf (tUxr (t) u(t)Πθr )

yr (t) = g(tUxr (t) u(t)Πθr )

xr (0) = Vx0

θr = Λθ

Reducing Truncated State Projection V isin RntimesN

Reconstructing Truncated State Projection U isin RNtimesn

Bi-Orthogonality VU = 1

Reducing Truncated Parameter Projection Λ isin RptimesP

Reconstructing Truncated Parameter Projection Π isin RPtimesp

Bi-Orthogonality ΛΠ = 1

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 642

Previously

Dynamic Causal Modelling

Combined State and Parameter Reduction

Empirical-Cross-Gramian-BasedGreedy-Optimization-Based

Hierarchical Approximate POD

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 742

Dynamic Causal Modelling (DCM)

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 842

Dynamic Causal Modelling [Friston et alrsquo03]

AimReconstruction of brain connectivity

from functional neuroimaging data (fMRIfNIRS EEGMEG)

ConceptHierarchical Model

1 Network Submodel (Dynamic Submodel)2 Observation Submodel (Forward Submodel)

Connectivity Parametrization

Effective Connectivity (amp Lateral Connectivity)

SIMO Models (amp MIMO Models)

Bayesian Inference

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 942

fMRI amp fNIRS Model

Dynamic Submodel (Taylor)

x(t) = f (x u θ)

asymp f (0 0 θ) +partf

partxx(t) +

partf

partuu(t)

= A(θ)x(t) + Bu(t)

Aij(θ) = θik+j rarr x isin Rkθ isin Rk2

Forward Submodel (Balloon)

si (t) = xi (t) minus κsi (t) minus γ(fi (t) minus 1)

fi (t) = si (t)

vi (t) =1

τ(fi (t) minus vi (t)

1α )

qi (t) =1

τ(

1

ρfi (t)(1 minus ((1 minus ρ))

1fi (t) ) minus vi (t)

1αminus1

qi (t))

yi (t) = V0(k1(1 minus qi (t)) + k2(1 minus vi (t)))

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1042

EEG amp MEG Model [Moran et alrsquo07]

Neural Mass Model(v(t)x(t)

)=

(0 1minusT minusT 2

)(v(t)x(t)

)+

(0

A(θ)

)ς(Kx(t)) +

(0B

)u(t)

y(t) = Lx(t)

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1142

Efficient DCM Implemention

Bayesian Inference using

EM AlgorithmE xpectation

WeightedTikhonov regularizedLeast-squaresTemporal Correlations

M aximization

Various additional modifications

See my diploma thesis doi106084m9figshare1027354v1

Now what if one wants to infer on large networks (N gt 6rarr P gt 36)

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1242

Combined State and Parameter Reduction

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1342

Combined Reduction(Together with M Ohlberger)

1 Empirical-Cross-Gramian-Based

2 Greedy-Optimization-Based

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1442

Cross Gramian [Fernando amp Nicholsonrsquo83]

Linear time-invariant system

x(t) = Ax(t) + Bu(t)

y(t) = Cx(t)

Hankel operator

H = OC

Cross Gramian (assume a square system M = Q)

WX = CO =

int infin0

eAt BC eAt dt isin RNtimesN

rArr AWX + WXA = minusBC

Cross Gramianrsquos central property

CAminus1B = (CAminus1B)ᵀ rArr σi (H) = |λi (WX )|

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1542

Cross-Gramian-Based Reduction

Balancing Transformation (for a symmetric system) [Aldhaherirsquo91]

WXEVD= TΛTminus1

Approximate Balancing Transformation [Sorensen amp Antoulasrsquo02]

WXSVD= UDV

Direct Truncation [Himpe amp Ohlbergerrsquo14]

WXSVD= UDV rarr U =

(U1 U2

)rarr V1 = Uᵀ

1

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1642

Empirical Cross Gramian [Streif et alrsquo06]

Empirical linear cross Gramian

WX =

int infin0

eAt BC eAt dt =

int infin0

(eAt B

)(eA

ᵀt Cᵀ)ᵀ dt

Empirical cross Gramian

WX =1

KLM

Ksumk=1

Lsuml=1

Msumm=1

1

ckdl

int infin0

Ψklm(t) dt isin RNtimesN

Ψklmij (t) = 〈xkmi (t)minus xi y

ljm(t)minus ym〉

State trajectory xkm(t) for the impulse input u(t) = ckemδ(t)

Output trajectory y lj(t) for the initial state x0 = dlej

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1742

Joint Gramian [Himpe amp Ohlbergerrsquo14]

Augmented system (treat parameters as constant states)(x(t)

θ(t)

)=

(f (t x(t) u(t) θ(t))

0

)y(t) = g(t x(t) u(t) θ(t))(

x(0)θ(0)

)=

(x0

θ0

)

Joint Gramian (empirical cross Gramian of the augmented system)

WJ =

(WX Wm

0 0

)

Lower blocks are zero as parameter-states are uncontrollable

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1842

Cross-Identifiability Gramian

Cross-Identifiability Gramian (Schur complement of symmetric part of WJ)

WI = minus1

2W ᵀ

m(WX + W ᵀX )minus1Wm

WI encodes the observability of parameters

Schur-complement can be approximated efficiently

Direct Truncation

WISVD= Π∆Λrarr Π =

(Π1 Π2

)rarr Λ1 = Πᵀ

1

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 1942

Cross-Gramian-Based Combined Reduction

1 Compute Empirical Joint Gramian WJ

2 State-Space Direct Truncation

WXSVD= UDV rarr U =

(U1 U2

)rarr V1 = Uᵀ

1

3 Parameter-Space Direct Truncation

WISVD= Π∆Λrarr Π =

(Π1 Π2

)rarr Λ1 = Πᵀ

1

Empirical-Cross-Gramian-Based Reduced Order Model

xr (t) = V1f (tU1xr (t) u(t)Π1θr )

yr (t) = g(tU1xr (t) u(t)Π1θr )

xr (0) = V1x0

θr = Λ1θ

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2042

Non-Symmetric Cross Gramian [Himpe amp Ohlbergerrsquo16]

A cross Gramian for non-square and non-symmetric systems

Cross Gramian Superposition (B =(b1 bM

) C =

(c1 cQ

)ᵀ)

WX =M=Qsumi=1

int infin0

eAt bici eAt dt

Non-Symmetric Cross Gramian

WZ =Msumi=1

Qsumj=1

int infin0

eAt bicj eAt dt

=

int infin0

eAt( Msumi=1

bi)( Qsum

j=1

cj)

eAt dt

Interestingly there exists a connection to tangential interpolation

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2142

emgr ndash Empirical Gramian Framework (Version 40)

Empirical Gramians

Empirical Controllability Gramian

Empirical Observability Gramian

Empirical Linear Cross Gramian

Empirical Cross Gramian

Empirical Sensitivity Gramian

Empirical Identifiability Gramian

Empirical Joint Gramian

Features

Interfaces for Solver Kernel Distributed Memory

Non-Symmetric option for all cross Gramians

Compatible with OCTAVE and MATLAB

Vectorized and parallelizable

Open-source licensed

More info at httpgramiande

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2242

Combined Reduction(Together with M Ohlberger)

1 Empirical-Cross-Gramian-Based

2 Greedy-Optimization-Based

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2342

Iterative Projection Assembly [Lieberman et alrsquo10]

Idea Alternatingly extend parameter and state projection

1 Select parameter base vector θI+1 for parameter projectionΠI+1 = ΠI cup (θI+1 minus ΠIΠ

ᵀI θI+1)

2 Select state base vector x(θI+1) based on parameter θI+1UI+1 = UI cup (x(θI+1)minus UIU

TI x(θI+1))

3 Go to 1

Parameter selection Greedy method

State selection Energy-based

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2442

Parameter Reduction [Bui-Thanh et alrsquo08]

Greedy Sampling Strategy

Locate currently worst approximated parameter

Formulation and solution as optimization problem

θI+1 = arg maxθisinΘ y(θ)minus y(ΠIΠᵀI θ)2

L2minus β2θ2

2

subject to

x(t) = f (t x(t) u(t) θ)

y(t) = g(t x(t) u(t) θ)

x(0) = x0

Adaptive sampling (no pre-defined grid)

Tikhonov regularization

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2542

State Reduction

In each iteration after selection of θI+1

1 Simulate a state trajectory x(θI+1)

2 Select principal component x(θI+1)

Based on input-to-state mapping u 7rarr x

U1 = arg minUisinRNtimesnUᵀU=1 x(θI+1)minus Uxr (θI+1)2L2

POD model reduction

proven method for nonlinear systems

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2642

Data-Driven Regularization [Himpe amp Ohlbergerrsquo15]

An inverse problem provides data yd

Extended cost function with data mismatch as regularization

Jd = y(θ)minus yr (θr )2L2minus β2θ2

2 minus βdy(θ)minus yd2L2

Reduced order model is specific to data

Partial inversion during reduction

Accelerates projection assembly

Determining weighting coefficient more complicated

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2742

Optimization-Based Combined Reduction

Truncated parameter-space projection

ΠI+1 rarr ΛI+1 = ΠᵀI+1

Truncated state-space projection

UI+1 rarr VI+1 = UᵀI+1

Greedy-optimization-based reduced order model

xr (t) = VI+1f (tUI+1xr (t) u(t)ΠI+1θr )

yr (t) = g(tUI+1xr (t) u(t)ΠI+1θr )

xr (0) = VI+1x0

θr = ΛI+1θ

Software Implementation httpgithubcomgramianoptmor

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2842

Combined Reduction INonlinear Resistor-Capacitor Ladder Gramian- vs Optimization-Based

2040

6080

100

2040

6080

100

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

2040

6080

100

2040

6080

100

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

L2otimes L2-output error for varying reduced state and parameter dimensions

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 2942

Combined Reduction IIHyperbolic Network Model Gramian- vs Optimization-Based

2040

6080

100

2040

6080

100

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

2040

6080

100

2040

6080

100

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

L2otimes L2-output error for varying reduced state and parameter dimensions

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3042

Combined Reduction IIIfMRI amp fNIRS Dynamic Causal Model Gramian- vs Optimization-Based

51102

153204

255

1632

4864

80

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

51102

153204

255

1632

4864

80

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

L2otimes L2-output error for varying reduced state and parameter dimensions

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3142

Combined Reduction IIIIEEG amp MEG Dynamic Causal Model Gramian- vs Optimization-Based

51102

153204

255

3264

96128

160

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

51102

153204

255

3264

96128

160

10-16

10-14

10-12

10-10

10-8

10-6

10-4

10-2

100

Parame

ter

State

L2otimes L2-output error for varying reduced state and parameter dimensions

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3242

Hierarchical Approximate POD

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3342

Hierarchical Approximate POD(Together with T Leibner amp S Rave)

Why HAPOD Memory-bound or compute-bound POD applications

Properties

Column-wise slicing of snapshotsMean L2 projection error boundMaximum mode boundRooted tree organizationModular POD backend

Special Cases

Distributed HAPOD (Minimum communication parallel)Rolling HAPOD (Minimum memory footprint)

See our preprint httparxivorgabs160705210

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3442

HAPODDefinitionLet S sub V be a finite multiset of snapshot vectors in a Hilbert space V Given arooted tree T and mappings

D S rarr LT εT NT rarr Rge0

define recursively for each α isin NT

HAPOD(S T D εT )(α) = POD(Iα εT (α))

where the local input data multiset Iα is given by

Iα =

Dminus1(α) α isin LT⋃

βisinCT (α)σn middot ϕn|(σn ϕn) isin HAPOD(S T D εT )(β) otherwise

We call HAPOD(S T D εT ) the hierarchical approximate POD (HAPOD) ofS for the rooted tree T the snapshot mapping D and the local tolerances εT

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3542

Faster Empirical Cross Gramians

Simplified Description of the Empirical Cross Gramian

WX =Msum

m=1

int infin0

Ψm(t) dt isin RNtimesN

Ψmij (t) = 〈xmi (t) y jm(t)〉

Column-Wise Compution of the Empirical Cross Gramian (k-th column)

rArr WX lowastk =Msum

m=1

int infin0

ψmk(t) dt isin RNtimes1

ψmki (t) = 〈xmi (t) ykm(t)〉

Low-Rank Empircal Cross Gramian via HAPOD

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3642

MathEnergy Project(Together with P Benner and S Grundel)

MathEnergy ndash Mathematical Key Technologies for Evolving Energy Grids

Subproject Model Order Reduction

Model Reduction for Gas Networks

Software implementation

Power-to-Gas integration

BMWi funded

Federal Ministryfor Economic Affairsand Energy

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3742

Model [Grundel et alrsquo14]

Hierarchical Model

1 Network

2 Pipes (Euler equation)

partp(x t)

partt= minusc2

A

partq(x t)

partxpartq(x t)

partt= minusApartp(x t)

partx+λc2

2DA

q(x t)|q(x t)|p(x t)

Challenges

Semi-discretized rarr DAE System

Nonlinear

Hyperbolic

Perspective Extensions Compressibility Elevation Temperature

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3842

Reduction

Empirical Gramians

Modifications for

Hyperbolic systemsDAE systems

Alternative methods

Efficient simulation

FOMROM

Implementation

Comparison of

ModelsReduction approaches

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 3942

Scientific Sideshows

Replicability Reproducibility amp Reusability (doi103934Math20163261)

MORwiki (httpmodelreductionorg)

MOR Benchmark Framework

Stability Analysis on ROMs (with A Petersen)

RKHS Method for Empirical Gramians (with B Hamzi)

Scientific Computing on Single-Board Computers

FlexiBLAS (httpwwwmpi-magdeburgmpgdeprojectsflexiblas)

Explicit Runge-Kutta Methods

Efficient Octave MATLAB Code (httpgitiomtips)

ldquoGoodrdquo Colormaps

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 4042

tldl

Combined State and Parameter Reduction

for Nonlinear Network Systems

using Empirical Cross Gramians

httphimpescience

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 4142

ReferencesC Himpe and T Leibner and S Rave Hierarchical Approximate Proper OrthogonalDecomposition Preprint arXiv mathNA 1ndash24 2016

J Fehr J Heiland C Himpe and J Saak Best Practices for ReplicabilityReproducibility and Reusability of Computer-Based Experiments Exemplified by ModelReduction Software AIMS Mathematics 1(3) 262ndash281 2016

C Himpe and M Ohlberger A Note on the Non-Symmetric Cross Gramian SystemScience and Control Engineering 4(1) 199ndash208 2016

U Baur P Benner B Haasdonk C Himpe I Martini and M Ohlberger Comparison ofmethods for parametric model order reduction of instationary problems To appear inldquoModel Reduction and Approximation Theory and Algorithmsrdquo 2016

C Himpe and M Ohlberger Data-driven combined state and parameter reduction forinverse problems Advances in Computational Mathematics (MoRePaS Special Issue)41(5)1343ndash1364 2015

C Himpe and M Ohlberger The Empirical Cross Gramian for Parametrized NonlinearSystems In MATHMOD 2015 vol 48(1) pages 727ndash728 2015

C Himpe and M Ohlberger Cross-Gramian Based Combined State and ParameterReduction for Large-Scale Control Systems Mathematical Problems in Engineering20141ndash13 2014

C Himpe and M Ohlberger Model Reduction for Complex Hyperbolic Networks InProceedings of the ECC pages 2739ndash2743 2014

C Himpe and M Ohlberger A Unified Software Framework for Empirical GramiansJournal of Mathematics 20131ndash6 2013

C Himpe himpempi-magdeburgmpgde Empirical Gramians and Friends 4242