generalized means in manifolds existence, uniqueness ...jbigot/site/slidesiops2017/...almost...

IntroductionManifolds with convex geometry

Robustness of mediansAlmost everywhere uniqueness for generic data points

Finding general means in compact manifolds with simulated annealing

Generalized means in manifoldsExistence, uniqueness, robustness and algorithms.

Marc Arnaudon

Université de Bordeaux, France

Le Teich, 5-8 July 2017

Marc Arnaudon Generalized means in manifolds Existence, uniqueness, robustness and algorithms.

joint works with

Frédéric Barbaresco (Thalès Air Systems)Clément Dombry (Université de Franche-Comté, Besançon)

Laurent Miclo (Institut de Mathématiques de Toulouse)Frank Nielsen (Ecole Polytechnique et Sony (Tokyo))

Anthony Phan (Université de Poitiers)Le Yang (Université de Poitiers)

General settingParticular casesExample 1Example 2

M Riemannian manifold with Riemannian distance ρ

µ a probability measure on M

κ : M ×M → R continuous

U : M → R

x 7→∫

Mκ(x , y)µ(dy)

Pb: Find a global minimizer of U

for p ∈ [1,∞), Hp,µ(x) :=

∫Mρp(x , y)µ(dy);

H∞,µ(x) := ‖ρ(x , ·)‖L∞(µ).

Qp,µ: set of minimizers of Hp,µ

if Qp,µ has a unique element, we denote it ep,µ and call it the p-mean of µ.

M Riemannian manifold with Riemannian distance ρ

µ a probability measure on M

κ : M ×M → R continuous

U : M → R

x 7→∫

Mκ(x , y)µ(dy)

Pb: Find a global minimizer of U

for p ∈ [1,∞), Hp,µ(x) :=

∫Mρp(x , y)µ(dy);

H∞,µ(x) := ‖ρ(x , ·)‖L∞(µ).

Qp,µ: set of minimizers of Hp,µ

if Qp,µ has a unique element, we denote it ep,µ and call it the p-mean of µ.

For p = 2, e2 is the barycenter or the Fréchet mean of µ

y µ(dy).

In convex domains ∫M

−→e2y µ(dy) = 0.

For p = 1, e1 is the median of µ. In convex domains it is characterized by∥∥∥∥∥∫

−→e1y∥∥−→e1y∥∥ µ(dy)

∥∥∥∥∥ ≤ µ({e1}).

For p =∞, e∞ is the center of the smallest enclosing ball of supp(µ).In statistics we are mainly interested in

µN (ω) = µ(X1(ω), . . . ,XN (ω))

µ(x1, . . . , xN ) =1N

N∑k=1

δxk .

y µ(dy).

−→e2y µ(dy) = 0.

−→e1y∥∥−→e1y∥∥ µ(dy)

∥∥∥∥∥ ≤ µ({e1}).

µN (ω) = µ(X1(ω), . . . ,XN (ω))

µ(x1, . . . , xN ) =1N

N∑k=1

δxk .

y µ(dy).

−→e2y µ(dy) = 0.

−→e1y∥∥−→e1y∥∥ µ(dy)

∥∥∥∥∥ ≤ µ({e1}).

µN (ω) = µ(X1(ω), . . . ,XN (ω))

µ(x1, . . . , xN ) =1N

N∑k=1

δxk .

y µ(dy).

−→e2y µ(dy) = 0.

−→e1y∥∥−→e1y∥∥ µ(dy)

∥∥∥∥∥ ≤ µ({e1}).

µN (ω) = µ(X1(ω), . . . ,XN (ω))

µ(x1, . . . , xN ) =1N

N∑k=1

δxk .

M = Sym+(d) positive definite real symmetric matrices

ρ(P,Q) =

√√√√ d∑i=1

ln2 λi , λi eigenvalues of P−1/2QP−1/2

Riemannian metric〈V ,W 〉P = tr(VP−1WP−1)

is invariant by the action of Gl(d): A ? P = APAT ,

and P 7→ P−1 is an isometry.

M is a Cartan-Hadamard manifold endowed with the Fisher information metric for thecentered Gaussian model.

Auto-regressive centered Gaussian stationary process: Z = (Z1, ...,Zn)T

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

Rn ∈ M = THPDn is a Toeplitz Hermitian Positive Definite matrix

An estimate of Rn is our observation (radar or echo-doppler)

Kähler potential function: Φ(Rn) = − ln(det Rn)− n ln(πe) yields a Riemannianmetric simpler than Fisher information metric

In suitable coordinates (r0, µ1, . . . , µn−1) = ϕ(Rn),

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

THPDn is a Cartan-Hadamard manifold satisfying −4 ≤ K ≤ 0.

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

Covariance matrix

[E[Zi Zj ]

]1≤i, j≤n

r0 r1 . . . rn−1r1 r0 . . . rn−2...

. . .. . .

...rn−1 . . . r1 r0

ds2 = ndr2

n−1∑k=1

(n − k)|dµk |2

(1− |µk |2)2,

Existence and uniquenessDeterministic algorithmsStochastic algorithmsStochastic algorithms, speed of convergence

Assume the sectional curvatures on M satisfy −β2 ≤ Kσ ≤ α2.

Fix a geodesic ball B(a, r) ⊂ M.

Assume suppµ ⊂ B(a, r).

Assumption ∗

The support of µ is not a singleton. p > 1 or the support of µ is not contained in ageodesic, the radius r satisfies

r < rα,p with

{rα,p = 1

2 min{

inj(M), π2α}

if p ∈ [1, 2)

rα,p = 12 min

{inj(M), π

}if p ∈ [2,∞)

Theorem (B. Afsari, 2010)

Under Assumption ∗, the function Hp has a unique minimizer ep in M, the p-mean of µ.Moreover ep ∈ B(a, r).

Case p = 2: W. Kendall (91)

Assumption ∗

r < rα,p with

{rα,p = 1

2 min{

inj(M), π2α}

if p ∈ [1, 2)

rα,p = 12 min

{inj(M), π

}if p ∈ [2,∞)

Assumption ∗

r < rα,p with

{rα,p = 1

2 min{

inj(M), π2α}

if p ∈ [1, 2)

rα,p = 12 min

{inj(M), π

}if p ∈ [2,∞)

Assumption ∗

r < rα,p with

{rα,p = 1

2 min{

inj(M), π2α}

if p ∈ [1, 2)

rα,p = 12 min

{inj(M), π

}if p ∈ [2,∞)

Assumption ∗

r < rα,p with

{rα,p = 1

2 min{

inj(M), π2α}

if p ∈ [1, 2)

rα,p = 12 min

{inj(M), π

}if p ∈ [2,∞)

Assumption ∗

r < rα,p with

{rα,p = 1

2 min{

inj(M), π2α}

if p ∈ [1, 2)

rα,p = 12 min

{inj(M), π

}if p ∈ [2,∞)

Theorem 1

Take p ∈ [1,∞). Let (tk )k≥1 ⊂ (0,Cp,µ,r ] a sequence of positive numbers satisfying∑∞k=1 tk = +∞ and

∑∞k=1 t2

k <∞. Let x0 ∈ B(a, r); define the sequence (xk )k≥0by

xk+1 = expxk

(−tk+1 gradxk

Hp(·)), k ≥ 0;

. Then xk −→ ep .

For 1 ≤ p < 2, the proof relies on

ρ2(xk+1, ep) ≤ ρ2(xk , ep)(1− Cp,µ,r tk+1) + C(β, r , p)t2k+1.

For p ≥ 2, similar inequality with Hp(xk+1)− Hp(ep).D. Groisser (2004 and 2005) and Huiling Le (2005): p = 2, tk constant, slightly smallerdomains.B. Afsari, R. Tron, R.Vidal (2012): p ≥ 2, optimal result in the sphere: tk constant aslarge as possible, domain as large as possible.

Theorem 1

∑∞k=1 t2

xk+1 = expxk

(−tk+1 gradxk

Hp(·)), k ≥ 0;

Theorem 1

∑∞k=1 t2

xk+1 = expxk

(−tk+1 gradxk

Hp(·)), k ≥ 0;

Theorem 1

∑∞k=1 t2

xk+1 = expxk

(−tk+1 gradxk

Hp(·)), k ≥ 0;

Theorem 2

Take p =∞. Let x0 ∈ B(a, r); define the sequence (xk )k≥0 by

xk+1 = expxk

(−tk+1

−−−−→xk yk+1), k ≥ 0,

where yk+1 is one point of supp(µ) which realizes the maximum of the distance to xk .Then xk −→ e∞ .

Theorem 3

Let (Pk )k≥1 an independent sequence of random variables taking their values inB(a, r), and with law µ.

let x0 ∈ B(a, r); define a random walk (Xk )k≥0 with X0 = x0

Xk+1 = expXk

(−tk+1 gradXk

ρp(·,Pk+1))

= expXk

(tk+1pρp−1−−−−→Xk Pk+1

), k ≥ 0;

where by convention gradx ρ1(·, x) = 0.

Then Xk −→ ep in L2 and a.s.

The main advantage of this method is that it does not require the computation ofgrad Hp .For 1 ≤ p < 2 the proof relies on inequality

E[ρ2(Xk+1, ep)|Fk

]≤ ρ2(Xk , ep)(1− Cp,µ,r tk+1) + C(β, r , p)t2

and then convergence of bounded supermartingale.For p ≥ 2, similar inequality with Hp(Xk+1)− Hp(ep).Related results :p = 2, Cartan-Hadamard manifold : K.T. Sturm 2004p = 1, M a Hilbert space : H. Cardot, P. Cenac, P.A. Zitt 2012convex manifold, C3 cost function : S. Bonnabel 2011

Theorem 3

Xk+1 = expXk

(−tk+1 gradXk

ρp(·,Pk+1))

= expXk

), k ≥ 0;

E[ρ2(Xk+1, ep)|Fk

Theorem 3

Xk+1 = expXk

(−tk+1 gradXk

ρp(·,Pk+1))

= expXk

), k ≥ 0;

E[ρ2(Xk+1, ep)|Fk

Theorem 3

Xk+1 = expXk

(−tk+1 gradXk

ρp(·,Pk+1))

= expXk

), k ≥ 0;

E[ρ2(Xk+1, ep)|Fk

Theorem 3

Xk+1 = expXk

(−tk+1 gradXk

ρp(·,Pk+1))

= expXk

), k ≥ 0;

E[ρ2(Xk+1, ep)|Fk

How does the algorithm work?A1, A2, A3 and A4 are data points, M is the p-mean.

Exemple of simulation: the map p 7−→ ep.

median

Theorem 4

Let (Xk )k≥0 the time inhomogeneous Markov process defined in Theorem 3 with

tk = min(δk ,Cp,µ,r

). Assume that Hp is C2 in a neighbourhood of ep and that

δ > Cp,µ,r .

Then the sequence of processes([nt]√

n−−−−→epX[nt]

)t≥0

converges weakly in D((0,∞),Tep M) to a diffusion process yδ with generator

(y − δ∇dHp(y , ·)]

gradepρp(·,P1)⊗ gradep

ρp(·,P1)].

Assume now that M is a complete Riemannian manifold of dimension ≥ 2.Let ∆ be an upper bound for sectional curvatures

Let r > 0 such that δ := µ(B̄(a, r)

Theorem 5

If ∆ > 0 and2δr

2δ − 1≤ rδ,1 then

Q1,µ ⊂ B̄

∆arcsin

(δ sin(

√∆r

√2δ − 1

If ∆ = 0 then

Q1,µ ⊂ B̄(

√2δ − 1

If ∆ < 0 then

Q1,µ ⊂ B̄

1√−∆

argsinh

(δ sinh(

√−∆r

√2δ − 1

Theorem 5

If ∆ > 0 and2δr

Q1,µ ⊂ B̄

∆arcsin

(δ sin(

√∆r

√2δ − 1

If ∆ = 0 then

Q1,µ ⊂ B̄(

√2δ − 1

If ∆ < 0 then

Q1,µ ⊂ B̄

1√−∆

argsinh

(δ sinh(

√−∆r

√2δ − 1

Theorem 5

If ∆ > 0 and2δr

Q1,µ ⊂ B̄

∆arcsin

(δ sin(

√∆r

√2δ − 1

If ∆ = 0 then

Q1,µ ⊂ B̄(

√2δ − 1

If ∆ < 0 then

Q1,µ ⊂ B̄

1√−∆

argsinh

(δ sinh(

√−∆r

√2δ − 1

Theorem 5

If ∆ > 0 and2δr

Q1,µ ⊂ B̄

∆arcsin

(δ sin(

√∆r

√2δ − 1

If ∆ = 0 then

Q1,µ ⊂ B̄(

√2δ − 1

If ∆ < 0 then

Q1,µ ⊂ B̄

1√−∆

argsinh

(δ sinh(

√−∆r

√2δ − 1

Assume that M is complete.

Recall µ(x1, . . . , xN ) =1N

N∑k=1

Theorem 6

Fix p ∈ [1,∞). Then for λ⊗N almost all (x1, . . . , xN ) ∈ MN the p-mean ep ofµ(x1, . . . , xN ) is unique.

Sketch of proof: if y1(x1, . . . , xN ) and y2(x1, . . . , xN ) are two p-means then the gradientof the map

(x1, . . . xN ) 7→ Hp,µ(x1,...,xN )(y1(x1, . . . , xN ))

is not equal to the gradient of the map

(x1, . . . xN ) 7→ Hp,µ(x1,...,xN )(y2(x1, . . . , xN )).

The difficulty comes from the fact that there are many sets on which these gradientsare not well-defined.

Recall µ(x1, . . . , xN ) =1N

N∑k=1

Theorem 6

(x1, . . . xN ) 7→ Hp,µ(x1,...,xN )(y1(x1, . . . , xN ))

(x1, . . . xN ) 7→ Hp,µ(x1,...,xN )(y2(x1, . . . , xN )).

Recall µ(x1, . . . , xN ) =1N

N∑k=1

Theorem 6

(x1, . . . xN ) 7→ Hp,µ(x1,...,xN )(y1(x1, . . . , xN ))

(x1, . . . xN ) 7→ Hp,µ(x1,...,xN )(y2(x1, . . . , xN )).

Corollary 1

Fix p ∈ [1,∞). If (Xn)n≥1 is a sequence of i.i.d. M-valued random variables withabsolutely continuous laws, then the process of empirical p-means ep,µ(X1(ω),...,Xn(ω))

is a.s. well defined.

For M = T, p = 2, continuous law: Bhattacharya and Pantagreanu (2003)For M = T, limit theorems for e2,µ(X1(ω),...,Xn(ω)): Hotz and Huckemann (2011)

Corollary 1

Fix p ∈ [1,∞). If (Xn)n≥1 is a sequence of i.i.d. M-valued random variables withabsolutely continuous laws, then the process of empirical p-means ep,µ(X1(ω),...,Xn(ω))

is a.s. well defined.

For M = T, p = 2, continuous law: Bhattacharya and Pantagreanu (2003)For M = T, limit theorems for e2,µ(X1(ω),...,Xn(ω)): Hotz and Huckemann (2011)

Back to general situation. Assume M is a compact manifold.

U(x) =

∫Mκ(x , y)µ(dy).

M set of minimizers of U.PB: find Xt →MNeed for δ > 0 small, κδ(x , y) =

p(δ, x , z)κ(z, y)λ(dz)

Uδ(x) =

∫Mκδ(x , y)µ(dy).

Gibbs measure νβ,δ =1

Zβ,δexp (−βUδ(x)) λ(dx)

Generator Lβ,δ =12

∆−12β∇Uδ

For large β, νβ,δ concentrates around minimal values of Uδ .

νβ,δ is the invariant measure for the diffusion process associated to Lβ,δ

dXt = ”dBt ”−β

2∇Uδ(Xt ) dt .

U(x) =

Uδ(x) =

∆−12β∇Uδ

2∇Uδ(Xt ) dt .

U(x) =

Uδ(x) =

∆−12β∇Uδ

2∇Uδ(Xt ) dt .

U(x) =

Uδ(x) =

∆−12β∇Uδ

2∇Uδ(Xt ) dt .

U(x) =

Uδ(x) =

∆−12β∇Uδ

2∇Uδ(Xt ) dt .

U(x) =

Uδ(x) =

∆−12β∇Uδ

2∇Uδ(Xt ) dt .

Let nt the density of Xt with respect to νβ,δ , ft =nt

νβ,δand

Jt = varνβ,δ (ft ) =

(ft − 1)2 dνβ,δ .

Then J′t = −νβ,δ(|∇ft |2)Poincaré inequality (Holley-Kusuoka-Stroock)

varνβ,δ (ft ) ≤ CM [1 ∧ β‖∇Uδ‖∞]5m−2 exp(b(Uδ)β)νβ,δ(|∇ft |2)

ConsequenceJ′t ≤ −C(βδ−1)2−5m exp(b(Uδ)β)Jt

Let βt = ln t/k , k > b(U), δt = (ln(2 + t))−1. With this choice Jt → 0 as t →∞. So

Theorem 7

For any neighbourhood N ofM, P(Xt ∈ N )→ 1 as t →∞.

Pb: difficult to compute ∇Uδ .

νβ,δand

(ft − 1)2 dνβ,δ .

Theorem 7

νβ,δand

(ft − 1)2 dνβ,δ .

Theorem 7

νβ,δand

(ft − 1)2 dνβ,δ .

Theorem 7

νβ,δand

(ft − 1)2 dνβ,δ .

Theorem 7

νβ,δand

(ft − 1)2 dνβ,δ .

Theorem 7

νβ,δand

(ft − 1)2 dνβ,δ .

Theorem 7

Let s 7→ φδ(s, x , y) the value at time s of the flow started at x of z 7→ −12∇zκδ(·, y)

Define the generator

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

and Xt an inhomogeneous diffusion process with generator Lαt ,βt ,δt ,αt = (1 + t)−1, βt = ln t/k , k > b(U), δt = (ln(2 + t))−1 :Let (Nt )t≥0 be a standard Poisson process and N(α)

t = N∫ t0 α−1s ds .

Let (Pk )k≥1 an independent sequence of random variables with law µ.Between jump times of Nαt , dXt = ”dBt ”.

If T is a jump time, XT = φδT

(αTβT ,XT−,PN(α)

)Here again Jt → 0 as t →∞.

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

Lα,β,δ(f )(x) =12

∆f (x) +1α

[f (φδ(βα, x , y))− f (x)] µ(dy)

t = N∫ t0 α−1s ds .

Theorem 8

generalized means in manifolds existence, uniqueness ...jbigot/site/slidesiops2017/...almost...

Documents