presented by: david dov

PRESENTED BY: DAVID DOV

Problem setting▪Consider a partial differential equation (PDE) of the form:

▪Where:

▪ is the latent solution

▪ is a nonlinear parametric operator

▪ is a subset of

Problem setting – cont’d▪Problem 1: data driven solution of PDE▪ Given a fixed find

▪Problem 2: system identification/ data driven discovery of PDE▪ Find that best describes the observed data

▪Consider the continuous and discrete time settings

Data driven solution of PDE (problem 1)Continuous time setting

▪Assuming is given, we have:

▪Let be a neural network

▪Let be a physics informed neural network defined by:

▪The key idea of this work:▪ Calculate with automatic differentiation

Training▪Proposed loss function:

▪Where:

▪Here is the training data on the boundaries

▪And:

▪Where are collocation points

Example - Schrodinger equationThe equation:

Initial and boundary conditions:

Goal: find (complex valued solution)

Example - Schrodinger equation – cont’dDefine:

Loss function:

Example - Schrodinger equation – cont’d

Understanding the loss function and the simulation

Initial conditions:

randomly sampledinitial colocations

Outputs of the NN



Boundary conditions:

randomly sampledboundary colocations Outputs of the NN Obtained with automatic

differentiation



Imposing the structure of the differential equation:

randomly sampledcolocations

Outputs of the physics-informed NN, (calculated with automatic differentiation)

Example - Schrodinger equation – cont’dResults

Ground truth: numerical (exact) solution for each x and t simulated at a fine grid using Runge–Kutta method


Limitations

▪Absence of theoretical error/convergence estimates

▪The interplay between the neural architecture/training procedure and the differential equation is not understood

▪Requires dense sampling –a bottle neck in high dimensional problems

Data driven solution of PDE (problem 1)

Related work:1. DEQGAN [Randle et al 20’] (presented last week)

▪ Limited to only problem 1 in the continuous time setting

▪ New loss function based on GAN instead of L2.

2. Learning data-driven discretizations for partial differential equations [Bar-Sinai et al 19’]▪ Use a parametric model to approximate spatial derivatives in

▪ NN is used to estimate the parameters

▪ A classical approach is used to integrate the equation over time

3. FiniteNet [Stevens et al 20’]▪ Extends 2 by using LSTM to model evolution of the solution over time

4. PDENet2.0 [Long et al 19’]▪ Solves problem 2 (PDE/system identification)

▪ Reveals the PDE rather than only its parameters as in this work

▪ Assumes measurements over multiple time steps (on a grid)

Discrete time setting

Background: Runge-Kutta (RK) method for numerical solution of PDE via integration

▪Recall the PDE:

▪Discretization: let , where

▪Goal: calculate given

▪RK1 – the simplest form (Euler method):

Discrete time settingBackground: Runge-Kutta (RK) method for numerical solution of PDE via integration

▪RK4 – a very common variant:

Image credit: https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#Implicit_Runge%E2%80%93Kutta_methods

Discrete time settingBackground: Runge-Kutta (RK) method for numerical solution of PDE via integration

▪RK the most general (implicit) form with q stages:

▪Where

▪The classical approach: solve a system of equations▪ The unknowns are and

▪ Guarantees on the temporal accumulated error:

▪ Computational bottleneck Image credit: https://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods#Implicit_Runge%E2%80%93Kutta_methods


Proposed solution:

▪Rewrite the equations:

▪Define a neural network with the multiple outputs:

▪Define the physics informed neural network:

Example – Allen-Cahn equation

Example – Allen-Cahn equation-cont’d▪Loss function:

Example – Allen-Cahn equation- cont’d▪Observation: the method suggests using (training) a neural network for a single step

▪Consider a large step:

▪Large number of Runge-Kutta stages

▪Results:


Notes

▪The highest order of ever used in Runge–Kutta methods

▪Huge time step of vs in standard simulations

▪No guaranties

▪Every step requires to retrain the network

Data driven discovery of PDE (problem 2)

Continuous time

▪Example – Navier-Stokes equation

▪Constraint - divergence-free functions

▪Assume:

where is a latent function (potential)

▪Goal: given the measurements

find the parameters and the pressure

Data driven discovery of PDE (problem 2)

Propose approach

▪Define:

▪Neural network with two outputs:

▪ are learnable parameters

▪Physics-informed neural network:

▪Loss function

Data driven discovery of PDE (problem 2)Example simulation: incompressible flow past a circular cylinder

▪Training and testing data is sampled from the rectangle

Data driven discovery of PDE (problem 2)Results

Data driven discovery of PDE (problem 2)Observation

▪The continuous time setting assumes the availability of measurements throughout the entirespatio-temporal domain

▪In many cases – measurements are available only at distinct time instnats

▪Next: discrete time setting

▪Assume: only two data snapshots

▪Solution by utilizing Runge-Kutta method as in problem 1

Data driven discovery of PDE (problem 2)Discrete time setting

▪ Recall the general form of Runge-Kutta method

▪Define:

▪Note the difference from problem 1


▪ Define the multi-output neural network:

▪Define the physics informed neural networks:


▪Given the measurements:

▪ Loss function is defined by:

Data driven discovery of PDE (problem 2)Example: Korteweg-de Vries (KdV) equation

▪Initial condition:

▪Periodic boundary condition

Data driven discovery of PDE (problem 2)Data:

▪ points

▪At

Data driven discovery of PDE (problem 2)Results

Conclusions▪Very nice application of NN for PDE

▪More questions than answers:▪ Architecture selection – why some architecture works and fail

▪ How would it work in high dimensions

▪ How to quantify uncertainty of predictions

presented by: david dov

Documents