APP

LIED

MAT

HEM

ATIC

S

A stochastic modeling methodology based on weightedWiener chaos and Malliavin calculusXiaoliang Wana, Boris Rozovskiib, and George Em Karniadakisb,1

aProgram in Applied and Computational Mathematics, Princeton University, Princeton, NJ 08544; and bDivision of Applied Mathematics, Brown University,Providence, RI 02912;

Edited by George C. Papanicolaou, Stanford University, Stanford, CA, and approved June 16, 2009 (received for review March 4, 2009)

In many stochastic partial differential equations (SPDEs) involvingrandom coefficients, modeling the randomness by spatial whitenoise may lead to ill-posed problems. Here we consider an ellipticproblem with spatial Gaussian coefficients and present a method-ology that resolves this issue. It is based on stochastic convolutionimplemented via generalized Malliavin operators in conjunctionwith weighted Wiener spaces that ensure the ellipticity condition.We present theoretical and numerical results that demonstrate thefast convergence of the method in the proper norm. Our approachis general and can be extended to other SPDEs and other types ofmultiplicative noise.

stochastic elliptic problem | SPDEs | uncertainty quantification

E lliptic partial differential equations perturbed by spatial noiseprovide an important stochastic model in applications, such

as diffusion through heterogeneous random media (1), stationarySchrodinger equation with a stochastic potential (2), etc. A typi-cal model of interest is the following stochastic partial differentialequation (SPDE):

−∇ · (a(x, ω)∇u(x)) = f (x), x ∈ D ⊂ Rd, u|∂D = g(x), [1]

where ω indicates randomness. Recently, this model has beenactively investigated in the context of uncertainty quantification(UQ) for mathematical and computational models, see, e.g.,refs. 3 and 4 and the references therein. The diffusion coeffi-cient a(x, ω) is typically modeled by the Karhunen–Loève typeexpansion a(x, ω) = a(x) + ε(x). Here, a(x) is the mean andε(x) = ∑

k≥1 σk(x)ξk is the noise term, where σk(x) are deter-ministic matrices and ξ := {ξi}i≥1 is a set of uncorrelated randomvariables with zero mean and unit variance. For the problem of Eq.1 to be well posed, the ellipticity condition is required. However, ifthe noise ε(x) is Gaussian, (or any other type with infinite negativepart), the standard ellipticity condition would not hold, no mat-ter how small the variance of ε(x, ω) is. Various modifications ofthe aforementioned setting have been used to mitigate this prob-lem. For example, Gaussian models were replaced by distributionswith finite range (e.g., uniform distribution). We note that fromthe statistical and, by implication, UQ perspectives, Gaussian per-turbations are of paramount importance and should be modeledjudiciously.

In this article, we propose a well-posed modification of themodel of Eq. 1 with Gaussian noise ε = ε(x). More specifically,we replace Eq. 1 by the following SPDE:

∇ · (a(x)∇u(x)) + ∇ · (δε(x)(∇u(x))) − f (x), x ∈ D, u|∂D = g(x),[2]

where “δε(x)” denotes the Malliavin divergence operator (MDO)with respect to Gaussian noise ε(x) (see, e.g., refs. 5 and 6 andthe next section). The MDO δε(ζ) is a stochastic integral andspecifically, in the present setting, a convolution of the integrandζ with the driving Gaussian noise ε. Although replacing the ordi-nary product by stochastic convolution may be surprising at first,it should be appreciated that physical systems rarely produce aninstant response to sharp inputs such as multiplicative forcing.

Therefore, in modeling systems subject to sharp perturbations,convolutions often become a model of choice.

Historically, stochastic convolution-based models were used tobypass the exceeding singularity of models with (literally under-stood) multiplicative noise. In fact, the idea of reduction to MDOcould be traced back to the pioneering work of K. Itô on stochasticcalculus and stochastic differential equations. Specifically, in hisseminal work (7), Itô has replaced the “product” model

u(t) = a(u(t)) + b(u(t)) · W (t) [3]

by the “stochastic convolution” model

u(t) = u0 +∫ t

0a(u(s))ds +

∫ t

0b(u(s))dW (s), [4]

where∫ t

0 b(u(s))dW (s) is the famous Itô’s stochastic integral. Inthis article, we extend Itô’s idea to the infinite dimensional sta-tionary system in Eq. 1 and replace it with Eq. 2. In contrast toItô’s SDEs, elliptic equations like 1 or 2 are not causal. By thisreason, we replace Itô’s integral by the MDO δW . The latter, inthis instance, is equivalent to anticipating (noncausal) Skorohodintegral, (see ref. 6).

A general theory of bilinear elliptic SPDEs that includes, inparticular, Eq. 2 was developed recently in ref. 8. In particular,it was shown that to ensure well-posedness of Eq. 2 it suffices toassume positivity of a(x) rather than of a(x, ω) = a(x) + ε(x). Thisapproach is based on Malliavin calculus and Wiener chaos expan-sion (WCE) with respect to Cameron–Martin basis (see ref. 9 andnext section).

The Cameron–Martin basis consists of random variables ξα,where α = (α1, α2, . . .) is a multiindex with nonnegative integerentries (see the next section for more detail). The WCE solu-tion of Eq. 2 is given by the series u(x) = ∑

α∈J uα(x)ξα, whereuα = E[uξα]. One can view the Cameron–Martin expansion as aFourier expansion that separates randomness from the determin-istic “backbone” of the equation. It was shown in ref. 8 that theWCE coefficients uα(x) for Eq. 2 verify a lower triangular systemof linear deterministic elliptic equations (see Eq. 19 below): Werefer to this system as the (uncertainty) propagator. Under verygeneral assumptions, the propagator is equivalent to SPDE 2 inthe same way as the set of coefficients of a Fourier expansion isequivalent to the underlying function.

In this article, we develop a numerical method for solving Eq. 2with high-order discretization in the physical space and weightedWCE in the probability space. We conduct a theoretical andnumerical investigation of the propagation of the truncation errorsand illustrate the results by numerical simulations. In particu-lar, an a priori error estimate is presented to demonstrate the

Author contributions: X.W., B.R., and G.E.K. performed research and wrote the paper.

The authors declare no conflict of interest.

This article is a PNAS Direct Submission.1To whom correspondence should be addressed. E-mail: gk@dam.brown.edu.

This article contains supporting information online at www.pnas.org/cgi/content/full/0902348106/DCSupplemental.

www.pnas.org / cgi / doi / 10.1073 / pnas.0902348106 PNAS August 25, 2009 vol. 106 no. 34 14189–14194

convergence of the developed numerical algorithm. We also carryout numerical and theoretical comparisons of the model of Eq. 2with direct multiplication model of Eq. 1 for positive (lognormal)coefficient a(ω). To facilitate the theoretical comparison of thesemodels, we have developed a generalization of the MDO δε (basedon Gaussian noise ε) to a divergence operator with respect to aclass of noises including nonlinear transformations of Gaussiannoise (e.g., lognormal).

In addition to its physical meaning, there are other mathemat-ical and computational reasons for employing the “convolution”model of Eq. 2 instead of the “multiplication” model of Eq. 1,specifically: (i) The convolution model is computationally effi-cient due to the lower triangular (in fact, bidiagonal) structureof the propagator; its computational complexity is linear. Onecould also formally define an approximate WCE solution for themultiplication model of Eq. 1; however, its propagator would bea full system with quadratic computational complexity. (ii) Thevariance of WCE solutions for both models is infinite. However,the blow-up of the convolution model is controllable, in that theWCE solution can be effectively rescaled by simple weights rαsuch that

∑α r2

αu2α < ∞ (see Theorem 3). We remark that the

blowup in both models is inevitable even if the perturbation ξ is1-dimensional (see example in the next section). The blowup ofthe multiplication model of Eq. 1 is much more severe and couldhardly be controlled effectively. In fact, we are not aware of anysystematic treatment of Eq. 1.

There are alternative ways to address Eq. 2 and similar bilin-ear equations. In particular, one could attack Eq. 2 using Hida’swhite-noise infinite dimensional calculus (see refs. 10 and 11). InHida’s approach, convolution is interpreted in terms of the Wickproduct, an operator closely related to stochastic integrals. Con-volution models for SPDEs with positive (lognormal) and normalrandom coefficients have been studied extensively (see, e.g., refs.2 and 12–14) by means of white-noise analysis. The white-noiseapproach relies on built-in spaces of stochastic distributions knownas Hida and Kondratiev spaces (see, e.g., refs. 15 and 16). TheMalliavin calculus is more flexible, and in applications to SPDEsit allows us to build solution spaces optimal for the equation athand (see, e.g., refs. 8 and 17). In this article, we take advantage ofthis important feature of Malliavin calculus to obtain more pow-erful numerical approximation schemes and substantially moreprecise estimates of the convergence rates than those reported inliterature on white-noise analysis.

Malliavin Calculus and Elliptic SPDEsLet F := (Ω, F , P) be a probability space, where F is the σ-algebragenerated by ξ := {ξi}i≥1 and U be a real separable Hilbert space.Given a real separable Hilbert space X , we denote by L2(F; X )the Hilbert space of square-integrable F -measurable X -valuedrandom elements f . When X = R, we write L2(F) instead ofL2(F; R). A formal series W = ∑

k≥1 ξkuk, where {uk, k ≥ 1} is acomplete orthonormal basis in U , is called Gaussian white noise onU . Specifically, for the elliptic SPDE we consider here the properspaces are X = H1(D) and U = L2(D).

To explain the exact nature of solution to SPDE 2, we recallthe notion of Cameron–Martin basis. Let J be the collectionof multiindices α with α = (α1, α2, . . .) so that αk ∈ {0, 1, 2, . . .}and |α| := ∑

k≥1 αk < ∞. For α, β ∈ J , we define α + β =(α1 + β1, α2 + β2, . . .), |a| = ∑

k≥1 αk, and α! = ∏k≥1 αk!. By εi we

denote the multiindex of length 1 and with the single nonzero entryat position i: i.e., the kth coordinate of εi is 1 if k = i and 0 if k �= i.Define the collection of random variables Ξ = {ξα, α ∈ J } as fol-lows: ξα = ∏

k≥1(Hαk (ξk)/√

αk!), where Hn(x) are 1-dimensionalHermite polynomials of order n.

Theorem 1. [Cameron–Martin (9)] The set Ξ is an orthonormalbasis in L2(F): if η ∈ L2(F) and ηα = E[ηξα], then η =∑

α∈J ηαξα = ∑α∈J ηαHα(ξ)/

√α! and E[η2] = ∑

α∈J η2α.

Next, we introduce the Malliavin derivative (MD) and theMDO. To make these notions more transparent, we begin withthe simplest setting: we will differentiate and integrate ξα withrespect to a single normal random variable, e.g. ξk, the kth coor-dinate of ξ. The MD Dξk and divergence operator δξk are defined,respectively, by

Dξk (ξα) := √αkξα−εk

, δξk (ξα) :=√

αk + 1ξα+εk. [5]

In the literature on quantum physics (see, e.g., ref. 18) theoperators δξk and Dξk are often called creation and annihilationoperators, respectively. As intuition suggests, the MDO of Dξk (ξα)recovers the integrand ξα. Indeed, for αk > 0, α−1

k δξk (Dξk (ξα)) =ξα. Also, MD and MDO can be extended to L2(F; X ). For exam-ple, for f ∈ L2(F; X ⊗U), δW (f ) is defined as the unique element ofL2(F; X ) with the property E[ϕδW (f )] = E[(f , DW ϕ)U ] for everyϕ such that ϕ ∈ L2(F) and DW ϕ ∈ L2(F; U) (see, e.g., ref. 17).

Before proceeding with the SPDE 2, let us consider a simpleexample to shed some light on the structure of the spaces withinwhich we could expect existence of solutions of this equation.

Example 1. Let u be a solution of equation

u = 1 + δξ(u), [6]

where ξ ∼ N (0, 1). Simple calculations show that ‖u‖2L2(F) =∑∞

k=1 u2n = ∞, where un = ∑

n≥0 E[uHn(ξ)]/√n!. On the otherhand, taking a weighted norm with weights rn = (n!)−1/22−qn/2,q > 0, we get

‖u‖2RL2(F) :=

∑n

r2nu2

n =∑n≥0

2−qn = (1 − 2−q)−1. [7]

The above example demonstrates that even simple station-ary equations do not have solutions with finite second moments.To overcome this obstacle one should introduce an appropriateweighted version of the solution space. Clearly, introduction ofweights amounts to rescaling of the stochastic Fourier (Wienerchaos) representation of the solution. Below, we discuss brieflythe construction of weighted spaces.

Let R be a bounded linear operator on L2(F) defined byRξα = rαξα for every α ∈ J , where the weights {rα, α ∈ J }are positive numbers. In what follows, we will identify the opera-tor R with the corresponding collection {rα, α ∈ J }. The inverseoperator R−1 is defined as R−1ξα = r−1

α ξα. The elements ofRL2(F; X ) can be identified with a formal series

∑α∈J fαξα,

where∑

α∈J ‖fα‖2X r2

α < ∞. Clearly, RL2(F; X ) is a Hilbert spacewith respect to the norm ‖f‖2

RL2(F;X ) := ‖Rf‖2L2(F;X ). We define

the space R−1L2(F; X ) as the dual of RL2(F; X ) relative to theinner product in the space L2(R; X ). For f ∈ RL2(F; X ) andg ∈ R−1L2(F) we define the scalar product

〈〈f , g〉〉 := E[(Rf )(R−1g)] ∈ X , [8]

where 〈〈f , g〉〉 = ∑α∈J fαgα, with g = ∑

α∈J gαξα.

Remark 1. One could readily check that u = 1 + ξ · u, i.e., the mul-tiplication version of Eq. 6, is much more complicated and blowsup much faster than Eq. 6.

To address SPDEs driven by lognormal and other importanttypes of random perturbations, we need to develop a bilinearsymmetric version of MDO with white noise W replaced by anarbitrary nonlinear transformation of Gaussian random variables.To begin with, assume that vector ξ consists of a single Gaussianrandom variable ξ ∼ N (0, 1). Let ξm = Hm(ξ)/

√m!. Define a

n-tuple MDO by induction: δ⊗1ξ (ξm) := δξ(ξm) and δ⊗n

ξ (ξm) :=

14190 www.pnas.org / cgi / doi / 10.1073 / pnas.0902348106 Wan et al.

APP

LIED

MAT

HEM

ATIC

S

δξδ⊗(n−1)ξ (ξm) for n > 1. Let δξn (ξm) := δ⊗n

ξ (ξm)/√

n!. It is readilychecked that

δξn (ξm) =√

(m + n)!m!n! ξm+n = 1√

m!n!Hm+n(ξ). [9]

Now, for ξ := (ξ1, ξ2, . . .) and multiindices α and β, defineδξα

(ξβ) := ∏∞k=1 δ

⊗αkξk

(ξβk (ξk)). The formula of Eq. 9 translatesinto the multidimensional case as follows:

δξα(ξβ) =

√(β + α)!/α!β!ξβ+α. [10]

Remark 2. (Wick product) The Wick product (�), can be definedas follows: for α, β ∈ J , Hα(ξ) � Hβ(ξ) := Hα+β(ξ), whereHγ(ξ) := ∏

k≥1 Hγk (ξk). It follows from Eq. 10 that Hα(ξ)�Hβ(ξ) =δHα(ξ)(Hβ(ξ)).

Theorem 2. If θ , η ∈ RL2(F; X ), set δξα(θ) := ∑

β∈J θβδξα(ξβ) and

δη(θ) := ∑α∈J ηαδξα

(θ). Then δη is a bounded linear operator onR′L2(F; X ) and

δη(θ) = δθ (η). [11]

The proof follows from Eq. 10.

Example 2. Let ξ ∼ N (0, 1) and c is a constant. Let θ = exp(−cξ−c22 ) and η = exp(cξ− c2

2 ). Then, δη(θ) = 1. This relation is impor-tant for the comparison of the product of Eq. 1 and convolutionof Eq. 2 with lognormal coefficient a(ω) (see Numerical Results).

Elliptic SPDE Model. Let us rewrite Eq. 2 as an SPDE in the senseof Malliavin calculus:{

Au(x) + δW (Mu(x)) = f (x) on D,u(x) = g(x) on ∂D, [12]

where D denotes the physical domain, W is white noise on U =L2(D), the Hilbert space of square summable sequences of realnumbers,

Au(x) := −∑

i,j

Di(aij(x)Dju(x)), [13a]

Mku(x) :=∑

i,j

Di(σk

ij(x)Dju(x)), [13b]

and Mu = ∑k≥1 Mku ⊗ uk. We note that uk in L2(D) can be iden-

tified with εk, see Eq. 5. For simplicity, we assume that g = 0 andthat f is deterministic. We can now rewrite the term δW (Mu) as

δW (Mu) =∑k≥1

δξk (Mku). [14]

Everywhere below we assume that: (i) the functions aij(x) andσkij(x)

are measurable and bounded in the closure D of D; and (ii) thereexist positive numbers A1, A2 such that A1|y|2 ≤ aij(x)yiyj ≤ A2|y|2for all x ∈ D and y ∈ R

d.

Definition 1. A solution to Eq. 12 is a random element u ∈ H10 (D)

such that the equality

〈〈Au, v〉〉 + 〈〈δW (Mu), v〉〉 = 〈〈f , v〉〉 [15]

where f ∈ H−1(D) and

v ∈ R−1L2(F) and DW v ∈ R−1L2(F; U). [16]

Analytical issues related to Eq. 15 have been investigated in ref.8. In particular, the following result holds:

Theorem 3. There exists an operator R and a unique solution u ∈RL2(F; H1

0 (D)) of Eq. 12 such that:

(i) The operator R is defined by the weights rα given by

rα = qα

√|α|! , with qα =∞∏

k=1

qαkk ,

where the numbers qk are chosen so that the “renormalizationcondition” ∑

k≥1

C2kq2

k < 1 [17]

holds, and Ck are defined by

‖A−1Mkv‖H10 (D) ≤ Ck‖v‖H1

0 (D), ∀v ∈ H10 (D).

(ii) With the definition of rα given in (i), uα satisfy

r2α‖uα‖2

H10 (D)

≤ C2A‖f‖2

H−1(D)

|α|!α!

∏k≥1

(Ckqk)2αk ,

where the constant CA is defined by

‖u0‖H10 (D) ≤ CA‖f‖H−1(D). [18]

Because the expectation of the MDO is zero, we have

E[Au + δW (Mu)] = AE[u] = f (x),

which is the unperturbed (deterministic) version of elliptic Eq. 1.By substituting the WCE u = ∑

α∈J uαξα into Eq. 15 and perform-ing a Galerkin projection in the probability space, we can establishthe equivalence between Eqs. 2 and 15 and derive the followinguncertainty propagator:

Auα +∑k≥1

√αkMkuα−εk = fα, [19]

which is a system of deterministic partial differential equations(PDEs). In the next section, we will develop an efficient numericalalgorithm to solve Eq. 19.

Numerical MethodWe will employ WCE in the probability space and a high-orderfinite element discretization in the physical space. Our methodexploits the recursive structure of the propagator of Eq. 19.This remarkable property of the propagator significantly reducesthe complexity of the algorithm and provides opportunities forimproving numerical efficiency. For example, the coefficients uα,satisfying |α| = p, can be solved in parallel because they allrely only on already computed coefficients uβ with β < α and|α − β| = 1.

For simplicity and without loss of generality, we consider a2-dimensional physical domain D. Let Th be a family of triangula-tions of D with straight edges and h the maximum size of elementsin Th. We assume that the family is regular, in other words, the min-imal angle of all the triangles is bounded from below by a positiveconstant. We define the finite element space as

V Kh = {

v : v ◦ F−1K ∈ Pp(R)

},

and

Vh = {v ∈ H1(D) : v|K ∈ V K

h , K ∈ Th},

where FK is the mapping function for the element K which mapsthe reference element R to element K and Pp(R) denotes theset of polynomials of degree up to p over R. We assume thatvh|�D = 0, ∀vh ∈ Vh. Thus, Vh is an approximation of H1

0 (D) based

Wan et al. PNAS August 25, 2009 vol. 106 no. 34 14191

on piecewise polynomials. There exist many choices of basis func-tions on the reference elements, such as h-type finite elements(19), spectral/hp elements (20, 21), etc.

Let JM , p be a finite dimensional subset of J given by JM , p :={α|α ∈ N

M0 , |α| ≤ p, p ∈ N} and RM , p be a given set of weights rα ,

α ∈ JM , p. We define

Vc :=⎧⎨⎩f =

∑α∈JM , p

fαξα : ‖f‖RM , pL2(F) < ∞⎫⎬⎭ ,

V −1c :=

⎧⎨⎩f =

∑α∈JM , p

fαξα : ‖f‖R−1M , pL2(F) < ∞

⎫⎬⎭ .

Then, the stochastic finite element method (sFEM) can bedefined as: Find uM , p

h ∈ Vh ⊗ Vc such that⟨⟨AuM , p

h +M∑

k=1

δξk

(MkuM , p

h

), v

⟩⟩H1

0 (D)

= 〈〈f , v〉〉H10 (D), [20]

for any v ∈ Vh ⊗ V −1c where

〈〈g1, g2〉〉H10 (D) = E[(Rg1, R−1g2)H1

0 (D)].Note that we truncate the expansion of the white noise up to M

terms and the WCE up to polynomial order p.Let H = H1

0 (D), and rα = qα/√|α|! as in Theorem 3. The

main result regarding convergence of the stochastic finite elementmethod (sFEM) is given by the following theorem:

Theorem 4. For u ∈ RL2(F; H) ∩ RL2(F; Hm+1(D)), the error ofapproximation of the sFEM is given by∥∥u − uM , p

h

∥∥RL2(F; H)

≤ C

(hm‖u‖RL2(F; Hm+1) +

√qW

(1 − q)2 + qp+1

1 − q

), [21]

where the constant C is independent of h, q = ∑k≥1 C2

kq2k < 1,

qW = ∑k>M C2

kq2k, and Ck are constants defined in Theorem 3.

We remark that the 3 main components of the error O(hm),O(qW ), and O(qp+1) are due to the finite element discretization,the approximation of white noise, and truncation of the WCE,respectively. We also note that spectral convergence is obtainedin the weighted norm ‖ · ‖RL2(F;H). Next, we present a sketch ofthe proof whereas all technical details can be found in supportinginformation (SI) Appendix.

The approximation error can be decomposed as

u − uM , ph =

∑α∈JM , p

(uα − uα)ξα +∑

α∈J \JM , p

uαξα,

where uα and uα are the coefficients of chaos expansions of u anduM , p

h , respectively. Correspondingly, we obtain∥∥u − uM , ph

∥∥2RL2(F;H) ≤

∑α∈JM , p

‖uα − uα‖2H‖ξα‖2

RL2(F)

+∑

α∈J \JM , p

‖uα‖2H‖ξα‖2

RL2(F)

= I1 + I2

To obtain the error contribution I1, we need to estimate the finiteelement approximation error ‖uα − uα‖. Since uα depends on uβ

with |α − β| = 1, we should consider the error propagation in the

uncertainty propagator. The key observation is that for |α| > 0,the following equations are satisfied in the weak sense

Auα +M∑

k=1

√αkMkuα−εk = 0, ∀vh ∈ Vh,

Auα +M∑

k=1

√αkMkuα−εk = 0, ∀v ∈ H ,

from which we can obtain a recursive inequality for the errorpropagation as

‖uα − uα‖H ≤ C infvh∈Vh

‖uα − vh‖H +M∑

k=1

ck‖uα−εk − uα−εk ‖H .

We then use results from the approximation theory to bound thefinite element approximation error. The error contribution fromI2 is from the truncation of WCE and the approximation of whitenoise, which can be estimated using Theorem 3. More details aregiven in SI Appendix.

Numerical ResultsWe consider the 2-dimensional stochastic elliptic problem{−∇ · [(E[a](x) + δW (∇u(x))] = f (x), x ∈ D,

u(x) = 0, x ∈ ∂D,

where E[a](x) denotes the mean field of coefficient and W (x)the random perturbation. For simplicity, we choose the physicaldomain D = (0, 1)2, E[a](x) = 1, and f (x) = 1.

To represent the white noise on L2(D), we select the followingorthonormal basis

wm,n(x) =

⎧⎪⎪⎨⎪⎪⎩

1, m = n = 0√2 cos(mπx), n = 0√2 cos(nπy), m = 0

2 cos(mπx) cos(nπy), m, n = 1, 2, . . . , ∞.

Hence, we approximate the white noise as

W (x) =M∑

m+n=0

wm,n(x)ξm,n.

For convenience, we use an 1-dimensional index and rewrite theabove equation as

W (x) =M∑

k=1

wk(x)ξk.

Let u(x) = ∑p|α|=0 uαξα be the truncated WCE of the solution up

to polynomial order p. The uncertainty propagator takes the form

−∇ · (E[a](x)∇uα) −M∑

k=1

√αk∇ · (wk(x)∇uα−εk ) = fα,

where fα = 0 if |α| > 0, because f (x) is assumed to be determinis-tic, and the operator Mk is defined as Mku = −∇ · (wk(x)∇u). Tochoose proper rα for the weighted Wiener chaos space, we needto specify the constant Ck defined in Theorem 3. For our case, wehave

Ck = ‖wk(x)‖L∞(D)/A1.

Here, ‖wk(x)‖L∞(D) is uniformly bounded and A1 = 1 becauseE[a](x) = 1. Hence, the weights can be defined as rα = qα/

√|α|!with qα = ∏∞

k=1 qαkk , if qk is chosen as qk = 1/(k + 1)Ck.

We now examine the convergence of the sFEM. We employthe spectral/hp element method to solve the uncertainty

14192 www.pnas.org / cgi / doi / 10.1073 / pnas.0902348106 Wan et al.

APP

LIED

MAT

HEM

ATIC

S

Fig. 1. Convergence of the proposed stochastic finite element method forthe model problem. (A) Weighted L2 norm of approximate white noise. (B)Spectral convergence of the stochastic finite element method with respect tothe weighted norm. M = 21.

propagator. The physical domain (0, 1)2 is discretized in 64 quadri-lateral elements, where 12th-order piecewise polynomials are usedin each element. Numerical tests show that errors associated withthe physical discretization are close to machine accuracy and hencenegligible. If we fix the number of random variables in the approx-imation of white noise and increase the polynomial order of WCE,the dominant error in the error estimate of Eq. 21 should be thetruncation error of the WCE. In Fig. 1, we plot the convergencebehavior of the approximate white noise in the weighted normRL2(F, L2(D)) with respect to M . We also plot the errors of theapproximate solution in the weighted norm RL2(F, H1

0 (D)) withrespect to the polynomial order of WCE. For the numerical sim-ulation, we choose M = 21, corresponding to m + n ≤ 5. Theerrors are approximated as ‖uM , p+1

h − uM , ph ‖RL2(F;H1

0 (D)). We seethat spectral convergence is obtained, which is consistent with theerror estimate of Eq. 21.

Model Comparison. Next, we present a comparison of the con-volution and multiplication models by considering the following1-dimensional problem⎧⎨

⎩− ddx

(K(x) ∗ d

dxu)

= f (x),

u(0) = 0, u(1) = 0,[22]

where “∗” indicates stochastic convolution or ordinary multiplica-tion. We choose f (x) = 1, and we consider first the simple modelKI (x, ξ) = exp(cξ − 1

2 c2), where ξ ∼ N (0, 1) is a normal randomvariable and c is a constant indicating the degree of perturbation.In particular, KI (x) can be regarded as a simplified version of thelognormal noise eW . In addition, we consider a second model KIIwith space-dependent lognormal noise of the type

KII (x, ξ) = exp[c(ξ1 + √

2 cos(πx)ξ2 + √2 cos(2πx)ξ3

)−1

2c2 (1 + 2 cos2(πx) + 2 cos2(2πx)

)], [23]

where ξi, i = 1, 2, 3, are normal random variables. In otherwords, we take the first 3 modes of white noise W (x) = ξ1 +∑∞

i=2

√2 cos((i − 1)πx)ξi on L2(0, 1). It is easy to show that

E[KII (x, ξ)] = 1, and that Var(KII (x, ξ)) = exp(c2 +2c2 cos2(πx)+2c2 cos2(2πx)) − 1. In Fig. 2, we plot the variances of solutionscorresponding to 2 different models obtained from solving boththe convolution and the multiplication cases; results for both KIand KII are shown. We observe that when the noise is small, thevariances given by the 2 models are about the same; however,when the noise is large a large difference exists, with the variancegiven by the multiplication model increasing much faster thanthat given by the convolution model. For KI , when its standarddeviation increases from 0.1003 to 22.7379, the solution varianceincreases from O(10−5) to O(106) for the multiplication model,and from O(10−5) to O(101) for the convolution model—a 5-order difference in magnitude! For KII the variace achieves largervalues as we include more terms in the expansion for the whitenoise. More results and analysis for both models are included inSI Appendix.

This exponential increase of the variance with respect toperturbation can be explained by referring to the aforemen-tioned Example 2. Specifically, we consider the simple problem(δa(ux(x, ξ)))x = f (x), x ∈ (x0, x1), u(x0) = u(x1) = 0 with log-normal but space-independent coefficient a(ω) = exp(cξ − c2

2 );the corresponding solution is u(x) = θδ−1f (x) with δ beingthe Dirichlet Laplacian on (x0, x1) and δa(θ) = 1. Example 2shows that θ = exp(−cξ − c2

2 ). On the other hand, the solu-tion to the corresponding multiplication model (a · vx(x, ξ))x =f (x) is v(x, ξ) = a−1δ−1f (x). Hence, the rapid increase in thevariance observed in the computations is related to the ratio

Fig. 2. Variance versus perturbation at x = 0.5. For model II, it is veryexpensive to compute the solution beyond c = 1 for the multiplication model.

Wan et al. PNAS August 25, 2009 vol. 106 no. 34 14193

a−1/θ , which is equal to exp(c2). Clearly, v(x) = exp(c2)u(x)and E[|v(x)|k] = exp(kc2)E[|u(x)|k]. This establishes that solu-tions of the above equation with lognormal permeability cor-responding to multiplication model are exceedingly singular ascompared to a similar equation with stochastic convolution basedon MDO.

Summary and DiscussionIn this article, we have developed an efficient numerical methodfor solving second-order elliptic SPDEs with Gaussian coeffi-cients. We introduced the concept of “stochastic convolutionmodel” and employed Malliavin calculus to implement it. Thestochastic solution was represented by a weighted WCE in theappropriate norm. It was shown that the coefficients of the WCEcan be obtained by solving a lower triangular system of determin-istic elliptic PDEs, i.e., the uncertainty propagator. We derived aa priori error estimate to measure the rate of convergence withrespect to appropriately weighted L2 norm. We have also carriedout numerical and theoretical comparisons of the model of Eq. 2with the direct multiplication model of Eq. 1 for positive (lognor-mal) coefficient a(ω). To facilitate the theoretical comparison ofthese models, we have developed a generalization of the MDOδε (based on Gaussian noise ε) to a divergence operator withrespect to a class of noises including nonlinear transformationsof Gaussian noise (e.g., lognormal).

It might be instructive to reexamine the differences and simi-larities between the convolution model Au(x) = δW (Mu(x)) andits multiplication counterpart Au(x) = Mu(x) · W (x). It is wellunderstood that the multiplication models are exceedingly singu-lar. We are not aware of any systematic theoretical or numericalefforts to investigate such SPDEs. It appears though that the gen-eral methodology developed in this article could be extended toaddress elliptic equations of this type as well. However, in the

multiplication setting, the propagator is not lower triangular, andthe solution spaces are expected to be much larger than in thesetting of this article. In contrast to multiplication models theconvolution models are much more manageable analytically andnumerically and have solid physical meaning. Both models becomemuch easier if one replaces the white noise forcing by lognormal(which is a positive noise). We have shown that in the lognormalsetting the SPDE has reasonable solutions for both models. Still,as the variance of the noise grows, the variance of the solution ofthe multiplication equation scales up exponentially as comparedwith the convolution model.

In spite of the aforementioned differences, the 2 models areclosely related. In fact, the convolution models could be viewedas the highest stochastic order approximations to multiplicationmodels. Indeed, by Remark 2, δHα(ξ)(Hβ(ξ)) = Hα+β(ξ) whileHα(ξ) ·Hβ(ξ) = Hα+β(ξ)+Rα,β, where Rα,β is a linear combinationof Hermite polynomials of orders lesser than α + β.

Finally, we note that the mean in the multiplication modelfor the problems considered here deviate greatly from the corre-sponding deterministic solution (see SI Appendix). Clearly, linearsystems with additive noise are unbiased perturbations of theirdeterministic counterparts with the statistical average of a solutionto randomized system coinciding with the solution of the originalunperturbed system. The convolution bilinear equations consid-ered in this article also enjoy this important property of linearsystems. However, we stress that this property does not hold andshould not be expected from SPDEs with nonlinear operator Aor/and multiplication in the stochastic terms.

ACKNOWLEDGMENTS. We acknowledge the helpful suggestions by thereferees and the useful discussions with our students Z. Q. Zhang andC.-Y. Lee. This work was supported by the National Science Foundation(NSF)/Division of Mathematical Sciences and Army Research Office grants(to B.R.) and NSF/Applied Mathematics Crosscutting–Stochastic Systems andOffice of Naval Research (G.E.K.).

1. Papanicolaou G (1995) Diffusion in random media. Surveys in Applied Mathematics,eds Keller JB, McLaughlin D, Papanicolaou G (Plenum, New York).

2. Holden H, Oksendal B, Uboe Zhang T (1996) Stochastic Partial Differential Equations:A modeling, White Noise Functional Approach (Birkhauser, Boston).

3. Babuška I, Tempone R, Zouraris G (2004) Galerkin finite element approxi-mations of stochastic elliptic differential equations. SIAM J Numer Anal 42:800–825.

4. Frauenfelder P, Schwab C, Todor R (2005) Finite elements for elliptic problems withstochastic coefficients. Comput Methods Appl Mech Eng 194:205–228.

5. Malliavin P (1997) Stochastic Analysis (Springer, Berlin).6. Nualart D (2006) Malliavin Calculus and Related Topics (Springer, New York),

2nd Ed.7. Itô K (1944) Stochastic integral. Proc Imp Acad Tokyo Jpn 20:519–524.8. Lototsky S, Rozovskii B (2009) Stochastic partial differential equations driven by purely

spatial noise. SIAM J Math Anal, in press.9. Cameron R, Martin W (1947) The orthogonal development of nonlinear functionals

in series of Fourier—Hermite functionals. Ann Math 48:385.10. Hida T, Juo H, Potthoff J, Streit L (1993) White Noise (Kluwer Academic, Dordrecht,

The Netherlands).11. Kuo HH (1966) White Noise Distribution Theory (CRC, Boca Raton, FL).

12. Manouzi H, Theting T (2006) Numerical Analysis of the Stochastic Stokes Equations ofWick Type (Wiley InterScience, New York).

13. Theting T (2000) Solving Wick-stochastic boundary value problems using a finiteelement method. Stochastics Stochastic Rep 70:241–270.

14. Våge G (1998) Variational methods for PDEs applied to stochastic partial differentialequations. Math Scand 82:113–137.

15. Kondratiev Y, Samoylenko YS (1978) The spaces of trial and generalized functions ofinfinite number of variables. Rep Math Phys 14:325–350.

16. Kondratiev Y, Leukert P, Potthoff J, Streit L, Westerkamp W (1966) Generalizedfunctionals in Gaussian spaces: The characterization theorem revisited. J Funct Anal141:301–318.

17. Nualart D, Rozovskii BL (1997) Weighted stochastic Sobolev spaces and bilinear SPDE’sdriven by space-time white noise. J Funct Anal 149(1):200–225.

18. Glimm J, Jaffe A (1987) Quantum Physics. A Functional Integral Point of View(Springer, New York).

19. Ciarlet P (2002) The Finite Element Method for Elliptic Problems (Soc for Industrialand Appl Math, Philadelphia).

20. Karniadakis GE, Sherwin SJ (2005) Spectral/hp Element Methods for CFD (Oxford UnivPress, Oxford), 2nd Ed.

21. Schwab C (1998) p and hp- Finite Elements Methods (Oxford Univ Press, Oxford).

14194 www.pnas.org / cgi / doi / 10.1073 / pnas.0902348106 Wan et al.

Online Supplementary Material:

A stochastic modeling methodology based on weighted Wiener chaos

and Malliavin calculus

X. Wan, B. Rozovskii and G.E. Karniadakis

1 Proof of Theorem 4

Let the Wiener-chaos expansion of u and uM,ph be

u =∑

α∈J

uαξα, uM,ph =

∑

β∈JM,p

uβξβ,

respectively. Then the approximation error can be decomposed as

u − uM,ph =

∑

α∈JM,p

(uα − uα)ξα +∑

α∈J\JM,p

uαξα.

Correspondingly we obtain

‖u − uM,ph ‖2

RL2(F;H) ≤∑

α∈JM,p

‖uα − uα‖2H‖ξα‖2

RL2(F)

+∑

α∈J\JM,p

‖uα‖2H‖ξα‖2

RL2(F)

= I1 + I2 (1)

We first look at the term I1. In the uncertainty propagator

Auα +∑

k≥1

√αkMkuα−εk

= fα,

we see that uα is fully determined by the coefficients uβ with |α− β| = 1 and β < α. In other words,the coefficients uβ with β ∈ J \JM,p have no influence on uα with α ∈ JM,p. We know that for |α| > 0,the following equations are satisfied in the weak sense

Auα +M∑

k=1

√αkMkuα−εk

= 0, ∀vh ∈ Vh,

Auα +M∑

k=1

√αkMkuα−εk

= 0, ∀v ∈ H.

1

Let B(v1, v2) =∑

i,j(aijDiv1, Djv2) indicate the bilinear form introduced by the elliptic operator A.We have

B(uα − vh, uα − vh)

= B(uα − vh, uα − vh) + B(uα − uα, uα − vh)

= B(uα − vh, uα − vh) + (M∑

k=1

√αkMk(uα−εk

− uα−εk), uα − vh).

Using the ellipticity condition of operator A, we obtain

A1‖uα − vh‖H ≤ A2‖uα − vh‖H + supwh 6=0,

wh∈Vh

∣

∣

∣(∑M

k=1

√αkMk(uα−εk

− uα−εk), wh)

∣

∣

∣

‖wh‖H.

Combing the above inequality with the triangular inequality

‖uα − uα‖H ≤ ‖uα − vh‖H + ‖uα − vh‖H ,

we obtain

‖uα − uα‖H ≤ (1 +A2

A1) inf

vh∈Vh

‖uα − vh‖H

+1

A1sup

wh 6=0,

wh∈Vh

∣

∣

∣(∑M

k=1

√αkMk(uα−εk

− uα−εk), wh)

∣

∣

∣

‖wh‖H.

Since σij(x) is bounded and wk(x) is Lipschitz continuous, we have

‖uα − uα‖H ≤ C infvh∈Vh

‖uα − vh‖H +M∑

k=1

ck‖uα−εk− uα−εk

‖H ,

where the first term on the right-hand side is the approximation error from finite element approximationin the physical space and the second term is the error from the previous coefficients. We can use thisequation recursively to take into account the errors from previous coefficients uβ with β < α. It isobvious that we can obtain

‖uα − uα‖H ≤∑

β≤α

cα,β infvh∈Vh

‖uβ − vh‖H ,

where cα,β are constants dependent on α and β. If uα ∈ H∩Hm(D), we know from the approximationtheory that

infvh∈Vh

‖uα − vh‖H ≤ Chm‖uα‖Hm+1 .

Thus, we obtain

‖uα − uα‖2H ≤ h2m

∑

β≤α

cα,β‖uβ‖Hm+1

2

≤ h2m∑

β≤α

c2α,β

∑

β≤α

‖uβ‖2Hm+1

≤ h2mcα

∑

β≤α

‖uβ‖2Hm+1 ,

2

where h is independent of constants cα,β. Thus the term I1 in equation (1) can be bounded as

I1 ≤ h2m∑

α∈JM,p

cα

∑

β≤α

‖uβ‖2Hm+1‖ξα‖2

RL2(F)

≤ h2m∑

α∈JM,p

cα

∑

β≤α

‖uβ‖2Hm+1‖ξβ‖2

RL2(F)

= h2m∑

α∈JM,p

cα‖uα‖2Hm+1‖ξα‖2

RL2(F)

≤ maxα∈JM,p

cαh2m‖u‖2RL2(F;Hm+1).

We subsequently examine the term I2. Let α = α(1) ⊕ α(2), where α(1) = (α1, . . . , αM ) andα(2) = (αM+1, αM+2, . . .). We first examine the summation

∑

|α|=n

r2α‖uα‖2

H =n

∑

i=0

∑

|α(1)|=i,

|α(2)|=n−i

r2α‖uα‖2

H .

It is obvious that if n ≤ p, we can separate the terms further into two groups as

∑

|α|=n

r2α‖uα‖2

H =∑

|α(1)|=n,

|α(2)|=0

r2α‖uα‖2

H +n−1∑

i=0

∑

|α(1)|=i,

|α(2)|=n−i

r2α‖uα‖2

H .

The first term on the right-hand side is the truncated Wiener-chaos expansion and the second termgoes to the error estimation. Using theorem 3, we have

∑

|α(1)|=i,

|α(2)|=n−i

r2α‖uα‖2

H ≤ C2A‖f‖2

H−1

n!

i!(n − i)!

∑

|α(1)|=i,

|α(2)|=n−i

i!(n − i)!

α(1)!α(2)!

∏

k≥1

(Ckqk)2αk

= C2A‖f‖2

H−1

n!

i!(n − i)!(q − qW )iqn−i

W ,

where q =∑

k≥1 C2kq2

k and qW =∑

k>M C2kq2

k.

3

We are now ready to estimate the term I2.

I2 =∑

α∈J\JM,p

‖uα‖2H‖ξα‖2

RL2(F)

=

p∑

|α|=1

|α|−1∑

i=0

∑

|α(1)|=i,

|α(2)|=|α|−i

‖uα‖2H‖ξα‖RL2(F) +

∞∑

|α|=p+1

‖u‖2H‖ξα‖RL2(F)

≤ C2A‖f‖2

H−1

[

p∑

n=1

n−1∑

i=0

n!(q − qW )iqn−iW

i!(n − i)!+

∑

n>p

qn

]

= C2A‖f‖2

H−1

[

p∑

n=1

qn − (q − qW )n +∑

n>p

qn

]

≤ C2A‖f‖2

H−1

[

1 − qp

1 − q− 1 − (q − qW )p

1 − (q − qW )+

∑

n>p

qn

]

≤ C2A‖f‖2

H−1

[

qW

(1 − q)2+

qp+1

1 − q

]

,

where we used the mean value theorem for the function h(x) = 1−xp

1−x in the last step. It is easy to verify

that h′(x) ≤ 1(1−q)2

, ∀x ∈ (q − qW , q). Combining the estimates of I1 and I2, the conclusion follows.

2 Numerical Results

We provide here more details on the comparison between the convolution and the standard multipli-cation models using the one-dimensional problem considered in the paper, i.e.

{

− ddx(K(x) ∗ d

dxu) = f(x),u(0) = 0, u(1) = 0,

(2)

where ‘∗’ indicates the stochastic convolution or multiplication ‘·’.We plug into equation (2) the truncated Wiener chaos expansion

u(x) =

p∑

i=0

ui(x)ξi(ξ),

where p is the polynomial order and ξi(ξ) are normalized Hermite polynomials. By performing aGalerkin projection we obtain

−p

∑

i=0

d2

dx2ui(x)E[Kξiξj ] = δ0,j , ∀j = 0, 1, . . . , p (3a)

−p

∑

i=0

d2

dx2ui(x)E[δK(ξi)ξj ] = δ0,j , ∀j = 0, 1, . . . , p (3b)

for the multiplication and the stochastic convolution, respectively.

4

Using the Taylor expansion and the properties of Hermite polynomials, we obtain the followingresults for KI(x, ω)

E[KIξiξj ] = e−c2/2∞

∑

n=0

[n/2]∑

k=0

∑

p≤i∧j

A(n, k)B(i, j, p)δn−2k,i+j−2p, (4a)

E[δKI(ξi)ξj ] =

e−c2/2

√i!

∞∑

n=0

[n/2]∑

k=0

cn

√

(n − 2k + i)!

(n − 2k)!(2k)!!δn−2k+i,j , (4b)

where

A(n, k) =cn

n!

(

n

2k

)

(2k − 1)!!√

(n − 2k)!, B(i, j, p) =

[(

i

p

)(

j

p

)(

i + j − 2p

i − p

)]1/2

.

The above formulas are also valid for log-normal noise taking the form K(x, ω) = eφ(x)ξ, whereξ ∼ N (0, 1) and φ(x) is a deterministic function. It is straightforward to extend them to the high-dimensional case. For simplicity and clarity, our discussions will be mainly focused on the simplermodel KI(x, ω).

It is instructive to study the matrix structures of E[KIξiξj ] and E[δKI(ξi)ξj ]

• Multiplication. Given a finite polynomial order, the matrix E[KIξiξj ] is full, symmetric andnonnegative-definite. For all c1, . . . , cn ∈ R, we have

n∑

i,j=1

cicjE[KIξiξj ] = E

n∑

i,j=1

cicjKIξiξj

= E

[

(n

∑

i=1

K1/2I ξi)

]

≥ 0.

Thus, all the eigenvalues of E[KIξiξj ] are positive. In other words, the system (3a) is well-posed.However, all unknowns in equation (3a) are coupled together, which is not a desirable propertyfor numerical computation.

• Stochastic convolution. By noting that n must be even for i = j to get nonzero terms inequation (4b), we can prove that E[δKI

(ξi)ξi] = 1, ∀i = 0, 1, . . .

E[δKI(ξi)ξi] =

e−c2/2

√i!

∞∑

m=0

m∑

k=0

c2m

√

(2m − 2k + i)!

(2m − 2k)!(2k)!!δ2m−2k+i,i

=e−c2/2

√i!

∞∑

m=0

c2m

√i!

(2m)!!

= e−c2/2∞

∑

m=0

c2m 1

2mm!= e−c2/2ec2/2 = 1.

Furthermore, we note that E[δKI(ξi)ξj ] is a lower-triangular matrix. In other words, the Fourier

coefficients ui only depend on the modes uj with j < i.

5

Next, we examine the spatial distribution and decay of the Fourier coefficients of the solution

for the case KI(x, ξ) (defined in the main paper) and for c = 1. In figure 1 we compare the coefficientsof Fourier expansions for the two solutions given by the multiplication and stochastic convolution,respectively. For a better comparison, we use the same scale for both figures. We note that:

1. The multiplication yields a mean solution different from the deterministic solution based on theaveraged field KI(x) while the stochastic convolution yields the same one. This is due to theproperty of the stochastic convolution, which corresponds to a convolution with respect to whitenoise.

2. All coefficients of Fourier expansion decay. However, we can see that the coefficients correspondingto the stochastic convolution are much smaller than the corresponding ones corresponding to themultiplication.

0 0.2 0.4 0.6 0.8 1−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

x

u i

ud with E[K]

u0

u3

u6

0 0.2 0.4 0.6 0.8 1−0.05

0

0.05

0.1

0.15

0.2

0.25

0.3

x

u i

ud with E[K]

u0

u3

u6

Figure 1: Coefficients of Fourier expansion of solutions based on multiplication and stochastic convo-lution. The deterministic solution (ud) based on the averaged KI(x) is also included for comparison.Left: multiplication; Right: stochastic convolution.

We now study the relation between the degree of perturbation and the statistics of the solution.We vary the value of c from 0.1 to 2.5, indicating the standard deviation of KI(x) from 0.1003 to22.7379 for the statistics of KI(x). We use ud to indicate the deterministic solution corresponding tothe averaged mean field, i.e.,

− d

dx(E[KI(x)]

dud(x)

dx) = f(x). (5)

In figure 2, we show the increasing behavior of mean solution given by the multiplication withthe degree of perturbation, indicating by the value of c. For comparison, we include the deterministicsolution ud based on the averaged mean field E[KI ]. It can be seen that when the degree of perturbationis small (≤ 10%), the mean solution given by the multiplication is close to ud. When the noise becomeslarger, the mean solution increases fast away from ud. We note here the mean solution given by thestochastic convolution is always the same as ud.

Model II: For the space-dependent case KII we plot in figure 4 the mean and variance distributionalong the domain for fixed perturbation c = 1. The behavior is similar to case corresponding toKI(x, ξ).

6

0 0.2 0.4 0.6 0.8 110

−3

10−2

10−1

100

101

102

x

Mea

n

ud with E[K]

c=0.1

c=0.5

c=1

c=1.5

c=2.0

c=2.5

Figure 2: Mean solutions E[u(x)] along the domain for different degrees of the perturbation representedby c corresponding to the multiplication.

0 0.2 0.4 0.6 0.8 110

−10

10−8

10−6

10−4

10−2

100

102

104

106

108

1010

x

Var

ianc

e

c=0.1c=0.5c=1c=1.5c=2.0c=2.5

0 0.2 0.4 0.6 0.8 110

−8

10−6

10−4

10−2

100

102

x

Var

ianc

e

c=0.1c=0.5c=1c=1.5c=2.0c=2.5

Figure 3: Variance along the domain for different degrees of perturbation represented by c. Left:multiplication; Right: stochastic convolution

0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

x

Mea

n

ConvolutionMultiplication

0 0.2 0.4 0.6 0.8 10

0.5

1

1.5

2

2.5

x

Sta

ndar

dev

iatio

n

CovolutionMultiplication

Figure 4: Model II (kernel KII)): Mean (left) and standard deviation (right) when c = 1.

7