poincaré constant estimationpillaud/presentations/poincare... · 2021. 6. 18. · of dµ, our goal...

Poincare InequalityStatistical estimation of the Poincare Constant

Future Work?

Poincare Constant estimation

Loucas Pillaud-Vivien

Joint work with Alessandro Rudi and Francis BachIn collaboration with G.Stolz and T.Lelievre CERMICS/INRIA

January 14, 2020

Loucas Pillaud-Vivien Poincare Constant estimation


Future Work?

Outline

1 Poincare InequalityA historical perspectivePoincare inequalities in the modern frameworkApplication of Poincare inequalities

2 Statistical estimation of the Poincare ConstantSetting of the problemPoincare inequality as a spectral problemStatisical consistency of the estimator

3 Future Work?Learning a reaction coordinateConclusion



Future Work?

A historical perspectivePoincare inequalities in the modern frameworkApplication of Poincare inequalities

A historical perspective

Work of H.Poincare: PDE for mathematical physics Fouriereigenvalue problem for the heat equation:

∆U + kU = 0 Ω ⊂ R3, Ω bounded domain∂U∂n = 0 ∂Ω



Future Work?


A historical perspective

Spectral problem: for j > 1,

∆uj + kjuj = 0∂uj∂n = 0,

and he showed that kj admitted another characterization:

kj =

∫uj(−∆uj)dx∫

u2j dx

=

∫‖∇uj‖2dx∫

u2j dx

,

k2 > κ(Ω) > 0 and kj −→ +∞.



Future Work?




Future Work?


and after come calculations...



Future Work?


Poincare inequality for bounded open convex set in Rn

Theorem (H.Poincare 1890)

For Ω open bounded convex set of Rd , f smooth from Ω to Rsuch that,

∫Ω fdx = 0,∫

Ωf 2dx 6 P

∫Ω‖∇f ‖2dx ,

where P = P(Ω) 6 Kd Diam(Ω)2 < +∞.

(Has been optimized in the 60’s −→ P(Ω) = Diam(Ω)2/π2.)



Future Work?


Poincare inequalities: definition in modern language

Definition (Poincare inequality)µ ∈ P(Rd ) satisfies a Poincare Inequality with constant P if

Varµ(f ) 6 Pµ∫‖∇f ‖2dµ,

for all (bounded) f : Rd −→ R of class C1.

Recall that :

Varµ(f ) =∫

f 2dµ−(∫

fdµ)2

=∫ (

f −∫

fdµ)2

dµ∫‖∇f ‖2dµ = E(f ) is the Dirichlet Energy.

Spectral interpretation: E(f ) =∫∇f · ∇fdµ =

∫f (−Lf )dµ

−→ 1/P = λ2, first non-trivial eigenvalue of L.Loucas Pillaud-Vivien Poincare Constant estimation


Future Work?


Poincare inequalities : Basic examples 1

Poincare-Wirtinger: f : Ω = [0, 1] −→ R, smooth and periodic :f (0) = f (1) and

∫ 10 fdx = 0, then∫ 1

0f 2dx 6

14π2

∫ 1

0f ′2dx ,

Proof: f (x) =∑

m>1 am cos(2πmx) + bm sin(2πmx),a0 =

∫ 10 fdx = 0.

12π

∫ 1

0f 2dx =

∑m>1

a2m + b2

m

12π

∫ 1

0f ′2dx =

∑m>1

4π2m(a2m + b2

m)



Future Work?


Poincare inequalities : Basic examples 2

Poincare-Wirtinger spherical version: σ uniform measure on thesphere of dimension d . f : Sd −→ R, smooth such that∫Sd fdσ = 0, then ∫

Sdf 2dσ 6

1d

∫Sd‖∇f ‖2dσ,

proof by expansion in spherical harmonics.Poincare for gaussian: µ gaussian probability measure:dµ(x) : 1

(2π)d/2 e−‖x‖2/2dx f : Rd −→ R, smooth such that∫Rn fdµ = 0, then ∫

Rdf 2dµ 6

∫Rd‖∇f ‖2dµ,

proof by expansion in Hermite polynomials.Loucas Pillaud-Vivien Poincare Constant estimation


Future Work?


Ok, fine. But what are the applications of suchinequalities?



Future Work?


Bounding the variance

Poincare’s inequalities are a powerful tool to bound the variance.Assume that X a random vector distributed accorded to µ s.t.:

1 µ satisfies a Poincare inequality Var(f (X )) 6 PE‖∇f (X )‖2

2 f is L-lipschitzThen,

Var(f (X )) 6 PL.

Applications:X standard Gaussian random vector, f Lipschitz, then:Var(f (X )) 6 1The variance of the largest singular value, σ(A), of a randommatrix A taking values in [0, 1] is bounded by 1.



Future Work?


Convergence to equilibrium for Diffusions

Let us consider the overdamped Langevin diffusion in Rd :

dXt = −∇V (Xt)dt +√

2dBt ,

Stationnary measure: dµ(x) = e−V (x)dx .Semi-group: Pt(f )(x) = E[f (Xt)|X0 = x ] −→ ”law of Xt”.Infinitesimal generator: Lφ = ∆φ−∇V · ∇φ.

We can verify that the law of Xt follows the dynamics:

ddt Pt(f ) = LPt(f ).



Future Work?


Convergence to equilibrium for Diffusions

Theorem (Poincare implies convergence to equilibrium)With the notations above, the following propositions areequivalent:

µ satisfies a Poincare Inequality with constant PFor all f smooth, Varµ(Pt(f )) 6 e−2t/PVarµ(f ) for all t > 0.

Proof: Integration by part formula (µ is reversible),

−∫

f (Lg) dµ =∫∇f · ∇g dµ = −

∫(Lf )g dµ, hence,

ddt Varµ(Pt(f )) = d

dt

∫(Pt(f ))2dµ = 2

∫Pt(f )(LPt(f ))dµ

= −2∫‖∇Pt(f )‖2dµ

6 −2/P Varµ(Pt(f ))Loucas Pillaud-Vivien Poincare Constant estimation


Future Work?


Application to the Ornstein-Uhlenbeck process

The diffusion of the Ornstein-Uhlenbeck process follows theSDE in Rd :

dXt = −Xtdt +√

2dBt ,

Denote L the operator Lφ = ∆φ− x · ∇φ, then1 For dµ(x) = 1

(2π)d/2 e−‖x‖2/2dx , L is self adjoint in L2µ

2 µ stationnary measure of O-U process3 µ verifies Poincare inequality with constant 1.4 for all f smooth, for all t > 0.

Varµ(Pt(f )) 6 e−2tVarµ(f ).



Future Work?


Poincare implies Concentration of measure

Definition (Concentration of measure)One says that µ satisfies the concentration of measure property iffor any set A such that µ(A) > 1/2, we have:

µ(Ar ) > 1− exp(−r 2/2),

with Ar = x ∈ Rd | dist(x ,A) 6 r.

Theorem (Poincare implies Concentration of measure)Assume µ satisfies a Poincare Inequality with constant P, then forany set A such that µ(A) > 1/2, we have:

µ(Ar ) > 1− exp(− r

2√P

),



Future Work?


Statistical estimation of the Poincare Constant






Future Work?

Setting of the problemPoincare inequality as a spectral problemStatisical consistency of the estimator

Setting of the problem

Let X a random vector of Rd be distributed according to theprobability measure µ that satisfies the following Poincare :

Varµ (f (X )) 6 Pµ Eµ[‖∇f (X )‖2

]Goal: given (x1, . . . , xn) n i.i.d samples of the probability measureof dµ, our goal is to estimate Pµ.

Two steps approach:1 Construct a estimator Pn

µ

2 Prove its statistical consistency : Pnµ −→n→∞

Pµ



Future Work?


Reformulation of the problem in a RKHS

Let F be a dense RKHS in C1 associated with kernel K , then:

1 F = spanK (·, x), x ∈ Rd, and in particulary −→ K (y , x) ∈ F that we will note Kx .

2 Reproducing property: ∀f ∈ F and ∀x ∈ Rd ,f (x) = 〈f ,K (·, x)〉F . In other words, functions evaluations areequal to dot products with canonical elements of the RKHS.

3 Derivation corresponds to the following: ∀f ∈ F and∀x ∈ Rd , ∂j f (x) = 〈f , ∂jK (·, x)〉F .

Example: gaussian kernel: Kσ(x , y) = exp(‖x−y‖2

2σ2

).



Future Work?


Operators of the problemLet us define the following positive semi-definite operators:

The operators from F to F , Σ and Σ respectively thecovariance and the empirical covariance operators,

Σ = E [Kx ⊗ Kx ] =∫Rd

Kx ⊗ Kx dµ(x),

Σ = 1n

n∑i=1

Kxi ⊗ Kxi

The operators from F to F , ∆ and ∆,

∆ = E [∇Kx ⊗∇Kx ] =∫Rd∇Kx ⊗∇Kx dµ(x),

∆ = 1n

n∑i=1∇Kxi ⊗∇Kxi



Future Work?


Spectral characterization of the Poincare constant

Back to Poincare’s work:

Lemma (Spectral characterization of the Poincare constant)Let Pµ be the Poincare constant of F ⊂ H1(Rd , dµ), then Pµ isthe maximum of the following Rayleigh ratio:

Pµ = supf ∈F

〈f ,Cf 〉〈f ,∆f 〉 =

∥∥∥∆−1/2C∆−1/2∥∥∥ ,

where ‖ · ‖ is the operator norm, C = Σ−m ⊗m andm =

∫Rd Kx dµ(x) ∈ F .



Future Work?


Spectral characterization of the estimator of Poincareconstant

Definition (of the estimator of Poincare constant)

Pµ = Pµn = supf ∈F

〈f , C f 〉〈f , (∆ + λI)f 〉

=∥∥∥∆−1/2

λ C∆−1/2λ

∥∥∥ ,where C = Σ− m ⊗ m, m = 1

n∑n

i=1 Kxi and ∆λ = ∆ + λI is aregularized empirical version of the operator ∆.

Two remarks :1 We need to regularize because the kernel of ∆ is no longer

strictly included in the kernel of Σ.2 We only need the sup over a finite dimensionnal space:

Im(C)⊕ Im(∆). f =∑n

i=1 αi Kxi + βi · ∇Kxi .Loucas Pillaud-Vivien Poincare Constant estimation


Future Work?


Statisical consistency: Bias - Variance decomposition

Let us denote Pλµ the Poincare constant of the regularized problemPλµ = supf ∈F

〈f ,Cf 〉〈f ,(∆+λI)f 〉 =

∥∥∥∆−1/2λ C∆−1/2

λ

∥∥∥, then

|Pµ − Pµ| 6 |Pλµ − Pµ|︸︷︷︸Bias

+ |Pµ − Pλµ |︸︷︷︸Variance

Variance: ‖∆−1/2λ C∆−1/2

λ ‖ −→n→∞

‖∆−1/2λ C∆−1/2

λ ‖. Reliesstrongly on concentration of operators C to C and ∆ to ∆.Bias: ‖∆−1/2

λ C∆−1/2λ ‖ −→

λ→0‖∆−1/2C∆−1/2‖. Need one

assumption to make it converge.



Future Work?


Statisical consistency: Bias - Variance decomposition

Variance Bound proof. Relies strongly on concentration ofoperators C to C and ∆ to ∆.

Lemma (Control of the variance)Let δ ∈ (0, 1), λ & λ0/n, we have the following inequality withprobability 1− δ: ∣∣∣Pµ − Pλµ ∣∣∣ . Pλµ√

λnlog(1/δ).

Bias Bound proof. Let us assume that C 4 κ∆2 (slightlystronger that Poincare Inequality), then

|Pλµ − Pµ| 6 κλ.



Future Work?


Ok... but why do we want to estimate suchconstants?



Future Work?


Statistical estimation of the Poincare Constant






Future Work?

Learning a reaction coordinateConclusion

Back to the initial problem of Molecular Dynamics

Take Xt ∈ Rd a random vector describing a molecule. Its dynamicsis described by the overdamped Langevin diffusion

dXt = −∇V (Xt)dt +√

2β−1dBt .

Goal: Sample (Xt)t>0

Invariant measure: µ(dx) = Z−1 exp(−βV (x))dxProblem: Metastability



Future Work?


Metastability



Future Work?


Escaping for metastability: learning a reaction coordinate

Goal: Find the path ξ : Rd −→ Rr (with r d) of themetastability, ξ is called the reaction coordinate.

” the metastability of the process is along ξ⇐⇒

the measures µ(·|ξ(x) = z) satisfies a Poincare inequality with alittle Poincare constant ”



Future Work?


Escaping for metastability: learning a reaction coordinate

” the metastability of the process is along ξ⇐⇒

the measures µ(·|ξ(x) = z) satisfies a Poincare inequality with a littlePoincare constant ”

Program:1 given x1, . . . , xn i.i.d according to µ, estimate µ(·|ξ(x) = z)2 using Part II estimate the Poincare constant of µ(·|ξ(x) = z):Pµ(·|ξ(x)=z)(ξ)

3 optimize according to ξ to get ξ∗ = argmin Pµ(·|ξ(x)=z)(ξ)



Future Work?


Conclusion

Still a lot to do:

Discuss with CERMICS to have more relevant hypothesis forthe biasSubsampling techniques?Do some simulations!Technically speaking, related to kernel ICA and CCA, may beworth exploring


poincaré constant estimationpillaud/presentations/poincare... · 2021. 6. 18. · of dµ, our goal...

Documents