an introduction to generalized sobolev spacesmeshfree-methods-seminar/presentations/...2014/07/21...

Mathematics and

Computer Science

An Introduction to Generalized Sobolev Spaces

Michael McCourt

Department of Mathematical and Statistical SciencesUniversity of Colorado Denver

Meshfree SeminarIllinois Institute of Technology

July 21, 2014

[email protected] (UCDenver) Generalized Sobolev Spaces July 21, 2014 1 / 21

http://math.ucdenver.edu/~mmccourt

Mathematics and

Computer Science

Disclaimer

Today’s talk was thrown together very quickly and stolen liberally fromold talks and chapter 7 of the as-yet unpublished book “Kernel-basedApproximation Methods in MATLAB”.

As a result, I am inviting comments and questions to better understandhow we can present this content effectively.

We are also skipping some material for the sake of time and simplicity,but if you do not understand something please stop me so that we canfix it.



Mathematics and

Computer Science

Fundamental Application (Scattered Data Fitting)

Given data (x j , yj), j = 1, . . . ,N, with x j ∈ Rd , yj ∈ R, find a(continuous) function sf such that sf (x j) = yj , j = 1, . . . ,N.

Consider here multivariate kernel-based interpolation using adata-dependent linear function space (basis)

s(x) =N∑

j=1

cjK (x ,x j) = k(x)T c, x ∈ Ω ⊆ Rd

with K : Ω× Ω→ R a positive definite kernel.

To find cj solve the interpolation equations

s(x i) = f (x i) = yi , i = 1, . . . ,N

which gives a linear system Kc = y ; K is symmetric positive definite.



Mathematics and

Computer Science

Kernel Flexibility: Goodness and Badness

The shape parameter ε associated with some RBFs plays an importantrole in the quality of associated approximations.

Example Gaussian support vector machine decision contourε = 10, Too Erratic ε = 1, Nice ε = .1, Insensitive

This freedom to choose your kernel basis is a challenge when noguidance is available (e.g., MLE, CV).

Is it possible to “design” the right kernel?



Mathematics and

Computer Science

Reproducing Kernel Hilbert Spaces

Each positive definite kernel K has a unique Hilbert space associatedwith it called its Native Space, HK . For some domain Ω,

HK =

f

∣∣∣∣∣∣f =

NK∑k=1

ckK (·,xk ), xk ∈ Ω

,

where, in most cases, NK =∞.

It can be very difficult to classify functions in this space, but all suchfunctions satisfy the reproducing property:

f (z) = (f ,K (·, z))HK , z ∈ Ω,

for the native space inner product (f ,g)HK .

Of course, as Dr. Erickson recently discussed, the Native space innerproduct is itself difficult to understand. But some of its properties arevery useful.



Mathematics and

Computer Science

Reproducing Kernel Optimality

One important consequence of the reproducing property is theinterpolant error bound

|s(x)− f (x)| ≤ ‖f‖HK

√K (x ,x)− k(x)T K−1k(x).

Another significant result is the orthogonality property

‖f‖2HK= ‖s − f‖2HK

+ ‖s‖2HK,

which is proved in Fasshauer’s book. This has the importantconsequence of implying the optimality of the interpolant s:

‖s‖HK ≤ ‖v‖HK , v ∈ HK , v(x i) = yi



Mathematics and

Computer Science

Interpolation through Optimization with Regularization

Infinitely many interpolants exist for any given scattered data input.The RKHS interpolant s = k(·)T c is the interpolant that solves

minc

∥∥∥∥∥∥∥k(x1)T c

...k(xN)T c

−y1

...yN

∥∥∥∥∥∥∥+ µcT Kc.

because ‖s‖2HK= cT Kc.

GoalDesign an inner product which measures a function’s“crumminess”Find the kernel whose RKHS involves that inner productInterpolate with that kernel to minimize “crumminess”



Mathematics and

Computer Science

What is a “Good” Interpolant?

The norm minimization we achieve for free through RKHS willhopefully steer us towards a good interpolant.

The $100 question is: how does one define good?

We believe that the answer to this question is application-dependent.Positive (Pressure, density, etc)Monotone (Increasing field)Integrates to 1 (Cumulative distribution function)

Often times, the real metric to be minimized is “Dollars/Science”, butsuch a metric is often ill-defined so we will suffice to work with someproxy.

We will focus on the relationship between function values andderivatives to define our inner product, and only work on [0,1].



Mathematics and

Computer Science

Generalized Sobolev Spaces

Drs. Ye and Fasshauer have developed Hilbert spaces whosereproducing kernels are of the form

K (x , z) = G(x , z) + R(x , z), R(x , z) =na∑

k=1

akψk (x)ψk (z), ak ≥ 0

where

LG(x , z) = δ(x − z), x , z ∈ Ω,

BG(x , z) = 0, x ∈ ∂Ω,

andLR(x , z) = 0 (implying ψ ∈ null(L)),

for differential operator L = P∗P and boundary condition operator B.



Mathematics and

Computer Science

A Specific Generalized Sobolev SpaceSo that things fit on slides, we are going to consider only the case Dr.Erickson spoke on last week: L = −D2 and P = D with B = I.

The standard Sobolev space inner product, without boundaryconditions, would involve only the P operator:

(f ,g)P =

∫ 1

0(Pf )(x)(Pg)(x) dx =

∫ 1

0f ′(x)g′(x) dx .

Our GSS inner product will also include contributions from theboundary condition operator:

(f ,g)B = f (0)g(0) + f (1)g(1).

Functions in this generalized Sobolev space include

HRP,B = f | (f , f )P , (f , f )B <∞,

although other restrictions may apply depending on the design of [email protected] (UCDenver) Generalized Sobolev Spaces July 21, 2014 10 / 21


Mathematics and

Computer Science

The Components of a Generalized Sobolev Space KernelThe kernel G(x , z) = min(x , z)− xz is called the “Brownian-Bridge”kernel and serves as half of our GSS kernel

K (x , z) = G(x , z) + R(x , z).

To determine R(x , z) =∑na

k akψk (x)ψk (z) we must findψk ∈ null(L) = spanx ,1− x,

ψ1(x) = c1x + c2(1− x),

ψ2(x) = c3x + c4(1− x),

which are (·, ·)B orthonormal (recall (f ,g)B = f (0)g(0) + f (1)g(1)),

c21 + c2

2 = 1,

c23 + c2

4 = 1,c1c3 + c2c4 = 0.



Mathematics and

Computer Science

The Components of a Generalized Sobolev Space KernelPlugging in these ψ ∈ null(L) functions, and using the B-ON to simplifythings, gives us

K (x , z) = min(x , z) + xz(−1 + a1 + a2 − 2c1c2(a1 − a2))

+ (x + z)(c1c2(a1 − a2)− a1 + c21(a1 − a2))

+ a1 − c21(a1 − a2),

which is the reproducing kernel for HRP,B with inner product

(f ,g)HRP,B

= (f ,g)P

+ f (0)g(0)[(1− c2

1)/a1 + c21/a2 − 1

]+ (f (0)g(1)+f (1)g(0))

[c1c2(1/a1−1/a2)−1+8c2

1 − 8c41

]+ f (1)g(1)

[c2

1/a1 + (1− c21)/a2 − 1

].

We’ve assumed a1,a2 > 0; other formulas exist as [email protected] (UCDenver) Generalized Sobolev Spaces July 21, 2014 12 / 21


Mathematics and

Computer Science

The Components of a Generalized Sobolev Space Kernel

These results suggest that, because of the choice of P,B and thedesign of the ∂Ω inner product, the Native space inner product must beof the form

(f ,g)HRP,B

= (f ,g)P + p1f (0)g(0) + p2(f (0)g(1)+f (1)g(0)) + p3f (1)g(1).

At this point, we may choose p1,p2,p3 and solve

(1− c21)/a1 + c2

1/a2 − 1 = p1

c1c2(1/a1 − 1/a2)− 1 + 8c21 − 8c4

1 = p2

c21/a1 + (1− c2

1)/a2 − 1 = p3

c21 + c2

2 = 1

Note: Not all possible pi values are acceptable.



Mathematics and

Computer Science

Samples of Generalized Sobolev Space Kernel

(f ,g)H

R1P,B

=(f ,g)P + 2.875f0g0 − .0471(f0g1+f1g0) + 1.125f1g1,

K1(x , z) = min(x , z)− .129xz − .3261(x + z) + .2656.

(f ,g)H

R2P,B

=(f ,g)P + 7f0g0 + 1(f0g1+f1g0) + 2f1g1,

K2(x , z) = min(x , z)− .3598xz − .2055(x + z) + .1376.



Mathematics and

Computer Science

Samples of Generalized Sobolev Space Kernel

(f ,g)H

R3P,B

=(f ,g)P + 100f0g0 + 101(f0g1+f1g0) + 102f1g1,

K3(x , z) = min(x , z) + 3.3171xz − 2.1692(x + z) + 1.1006.

(f ,g)H

R4P,B

=(f ,g)P + .001f0g0 + .002(f0g1+f1g0) + .0005f1g1,

K4(x , z) = min(x , z) + .998xz − .9988(x + z) + .9999.



Mathematics and

Computer Science

Numerical Example of Interpolation

For this simple example, there is not enough freedom to see anydifference between interpolants when boundary values are provided:

N = 10 evenly spaced points, f (x) = sin(2πx) + x .



Mathematics and

Computer Science

Reference of GSS Kernels in Use

(f ,g)H

R1P,B

=(f ,g)P + 2.875f0g0 − .0471(f0g1+f1g0) + 1.125f1g1,

K1(x , z) = min(x , z)− .129xz − .3261(x + z) + .2656.

(f ,g)H

R2P,B

=(f ,g)P + 7f0g0 + 1(f0g1+f1g0) + 2f1g1,

K2(x , z) = min(x , z)− .3598xz − .2055(x + z) + .1376.

(f ,g)H

R3P,B

=(f ,g)P + 100f0g0 + 101(f0g1+f1g0) + 102f1g1,

K3(x , z) = min(x , z) + 3.3171xz − 2.1692(x + z) + 1.1006.

(f ,g)H

R4P,B

=(f ,g)P + .001f0g0 + .002(f0g1+f1g0) + .0005f1g1,

K4(x , z) = min(x , z) + .998xz − .9988(x + z) + .9999.



Mathematics and

Computer Science

Numerical Example of Interpolation Without Boundary Data

If boundary data is removed, we see some difference in theinterpolants, attributable to the design of the inner product.

N = 10 evenly spaced points (no boundary), f (x) = sin(2πx) + x .



Mathematics and

Computer Science

Boundary Counditions are a Pain

Not every choice of p1,p2,p3 in

(f ,g)HRP,B

= (f ,g)P + p1f (0)g(0) + p2(f (0)g(1)+f (1)g(0)) + p3f (1)g(1).

allows for interpolation in the full Sobolev spaceHP,B = f |(f , f )P , (f , f )B <∞.

For instance, if we work with p1 = p2 = p3 = 0, thenK (x , z) = min(x , z) and only functions with f (0) = 0 are in the Nativespace, as discussed by Dr. Erickson last week.

Similarly, if we choose p1 = p3 = 1 and p2 = 0, thenK (x , z) = min(x , z)− xz + 1/2 and only periodic functions are in theNative space.



Mathematics and

Computer Science

Expanding the Inner Product Structure

Some of you may find the structure

(f ,g)HRP,B

= (f ,g)P + p1f (0)g(0) + p2(f (0)g(1)+f (1)g(0)) + p3f (1)g(1).

of the inner product rather limiting.

The involvement of the P operator is fixed and to change theinteraction between derivatives throughout the domain we would needto change P.

It would be possible to change the structure of the boundarycontributions, though this would require a low-level change to(f ,g)∂Ω = f (0)g(0) + f (1)g(1).



Mathematics and

Computer Science

Alternate Boundary Inner Products

The definition of the boundary inner product could be modified to, e.g.,(f ,g)∂Ω = f (0)g(0) + σ2f (1)g(1)

(f ,g)∂Ω = f (0)g(1) + f (1)g(0)

(f ,g)∂Ω = f (a)g(a) + σ2(f (0)g(0) + f (1)g(1)), a ∈ (0,1)

(f ,g)∂Ω = f (0)g(0) + f (1)g(1) + σ2(f ′(0)g′(0) + f ′(1)g′(1))

(f ,g)∂Ω =∫ 1

0 f (x) dx∫ 1

0 g(x) dx + σ2(f (0)g(0) + f (1)g(1))

Each of these choices would yield different ψ functions to formR(x , z) =

∑nak=1 akψk (x)ψk (z), and different structure in the Native

space inner product

(f ,g)HRP,B

= (f ,g)P +na∑

k=1

fk gk

ak−

na∑i=1

na∑j=1

fk gk (ψi , ψj)P ,

where fk = (f , ψk )B are Fourier-type coefficients.



an introduction to generalized sobolev spacesmeshfree-methods-seminar/presentations/...2014/07/21...

Documents