some nonstandard distributions and asymptotics for...

Some nonstandard distributions and asymptotics formultivariate analysis

Akimichi Takemura

Univ. of Tokyo

August 10, 2010

A.Takemura (Univ. of Tokyo) August 10, 2010 1 / 67

Introduction

Introduction

In this talk I give a survey on some nonstandard techniques formultivariate distribution theory.

Based on joint works with Satoshi Kuriki, Yo Sheena, HidehikoKamiya and Tomonari Sei.

We cover the following topics.1 Tube formula approximation to maximum type test statistics

2 Wishart matrix when population eigenvalues are infinitely dispersed

3 Star-shaped distributions

4 Multivariate distributions defined via optimal transport


Introduction

Introduction

The four topics can be classified into two groups.

Based on normality (including asymptotic normality):

Tube formulaWishart matrix when population eigenvalues are infinitely dispersed

Construction of non-normal distribution:

Star-shaped distributionsDistributions via optimal transport


Tube formula approximation to maximum type test statistics

1. Tube formula approximation to maximum type teststatistic



Tube formula: some historical background

Jacob Steiner already had Steiner’s formula (1840) for the volume ofa tube of a convex set.

Minkowski defined mixed volumes.

Hotelling (1939) derived the tube formula for a one-dimensional curveand then H.Weyl immediately generalized it to a general dimension.

Hotelling’s motivation was a nonlinear regression problem (explainedlater).

Revival of tube formula in statistics around 1990.(Knowles-Siegmund(1989), J.Sun(1991,93) and many other people).



Euler characteristic method (independent development)

Euler characteristic heuristic was initiated by R.J.Adler forapproximating the distribution of the maximum of a random randomfield (Adler-Hasofer(1976), Adler’s book(1981)).

This method has been vigorously developed by Adler and KeithWorsley1.

Some important foundational work was done by Jonathan Taylor(2001 thesis).

A standard textbook now is Random Fields and Geometry by Adlerand Taylor, 2007, Springer.

1Keith Worsley passed away in February of 2009, which is a great loss.A.Takemura (Univ. of Tokyo) August 10, 2010 6 / 67


Two methods are equivalent

Around 2000, I and Kuriki were sitting in a talk by Keith Worsley inISM (Institute of Statistical Mathematics, Tokyo, Japan) and wassurprised that he was doing the same computations as us.

Takemura and Kuriki (2002) proved the equivalence of these twomethods by using Morse theorem (for finite dimensional case).

Tube method can be understood as finite dimensional specializationof Euler characteristic method.

I should also mention that “abstract tube” by Naiman and Wynn is adiscrete analog of tube formula.



Back to Hotelling’s original problem

Consider a nonlinear regression model:

Yi = µ+ βf (xi , θ) + ǫi , i = 1, . . . , n, ǫi ∼ N(0, σ2)

For example f (xi , θ) = cos(θxi ), i = 1, . . . , n. (Nonlinear in θ)

Consider testing the null hypothesis

H : β = 0

Let (µ, β, θ) be the maximum likelihood estimate.

Residuals:ei = Yi − µ− βf (xi , θ), i = 1, . . . , n.




The likelihood ratio test is written as∑n

i=1 e2i

∑ni=1(Yi − Y )2

< c ⇒ reject H.

The null distribution of the LRT is not standard (i.e. note a betadistribution) because of the singularity.

Singurity: θ can not be estimated under the null hypothesis H.

Let us consider the problem from a geometric viewpoint.

Recap of simple regression

For simplicity omit µ from the model (as in Hotelling’s original paper):

Yi = βf (xi , θ) + ǫi , i = 1, . . . , n, ǫi ∼ N(0, σ2)




For a given θ, the LSE β(θ) of β is

β(θ) =

∑ni=1 Yi f (xi , θ)

∑ni=1 f (xi , θ)

2=

Y ′f(θ)

‖f(θ)‖2 ,

where Y = (Y1, . . . ,Yn)′, f(θ) = (f (x1, θ), . . . , f (xn, θ))

′.

Let Y (θ) = β(θ)f(θ), e(θ) = Y − Y (θ). Then

‖Y ‖2 = ‖Y (θ)‖2 + ‖e(θ)‖2

and minimizing ‖e(θ)‖2 is equivalent to maximizing ‖Y (θ)‖2.

Y (θ) =Y ′f(θ)

‖f(θ)‖ = Y ′u(θ), u(θ) =f(θ)

‖f(θ)‖ ∈ Sn−1, (1)

where Sn−1 ⊂ Rn is the unit sphere.




The rejection region of LRT is written as

minθ

‖e(θ)‖2‖Y ‖2 < c

This is equivalent to

c ′ < maxθ

(Y ′u(θ))2

‖Y ‖2 = maxθ

(Y ′u(θ))2, Y = Y /‖Y ‖ ∈ Sn−1.

Write M = u(θ) | θ ∈ Θ, where Θ is the range of θ. ThenM ⊂ Sn−1. If θ is a scalar, then M is a curve in Sn−1.

Maximization in (1) is the projection of Y onto M.



Projection onto M

0

Mz

u

Z(u)=u’z

Sn-1

Figure: Projection onto M



Canonical form of tube formula

Let z = (z1, . . . , zn)′ ∼ Nn(0, In).

Let M ⊂ Sn−1 be a C 2-submanifold of dimension d = dimM withpiecewise smooth boundaries.

Let

Z (u) = u′z =n

∑

i=1

uizi , u = (u1, . . . , un)′ ∈ M.

Also consider a standardized random field

Y (u) = u′z/‖z‖, u ∈ M, ‖z‖ =√z ′z .



Canonical form of tube formula

We want to evaluate the distributions of maxima, corresponding tomaximum type test statistics:

T = maxu∈M

Z (u), U = maxu∈M

Y (u).

The tube method gives an approximation of the tail probabilities

P(T ≥ x), x ↑ ∞, and P(U ≥ x), x ↑ 1.



Spherical tube and its volume

Evaluation of the distribution reduces to the evaluation of the volumeof a spherical tube around M.

0

M θ

Sn-1

M

Figure: Spherical tube around M



Spherical tube and its volume

LetMθ =

v ∈ Sn−1 | minu∈M

cos−1(u′v) ≤ θ

denote the tube around M with radius θ.

Let Vol(Mθ) denote the (n − 1)-dimensional spherical volume of Mθ.

By definition

P(

maxu∈M

Y (u) ≥ cos θ)

= Vol(Mθ)/Ωn,

where

Ωn = Vol(Sn−1) =2πn/2

Γ(n/2)

and Ba,b(·) denotes the upper probability of beta distribution withparameter (a, b).



Tube formula for the volume of a spherical tube

Tube formula: For θ smaller than the critical radius,

Vol(Mθ) = Ωn

wd+1B d+12

, n−d−12

(cos2 θ) + wd B d2, n−d

2(cos2 θ)

+ · · ·+ w1B 12, n−1

2(cos2 θ)

,

where w1, . . . ,wd+1 are geometric invariants of M, which can beevaluated by differential geometric methods.

In particular wd+1 = Vol(M)/Ωn, wd = Vol(∂M)/Ωn.

The Euler characteristic of χ(M) of M:

χ(M) = 2m∑

e=0m−e:even

wm+1−e =

2(w1 + w3 + · · ·+ wm+1) (m: even)

2(w1 + w3 + · · ·+ wm) (m: odd),



Critical radius

Critical radius of the tube Mθ is the supremum of θ, such that Mθ

does not have a self-intersection.

M

M

Figure: Tubes with a radius equal to the critical radius.



Tail probability for T (non-standardized maximum)

For T = maxu∈M Z (u) we need integration of the tube formula in‖z‖.By integration on ‖z‖ we have

P(

maxu∈M

Z (u) ≥ x)

= wd+1Gd+1(x2) + wd Gd(x

2) + · · ·

+w1G1(x2) + O(Gn(x

2(1 + tan2 θc))),

where Ga(·) is the upper probability of χ2 distribution with a degreesof freedom and θc is the critical radius.



Application to a multilinear form

Consider a model for interaction in the two-way layout (Johnson andGraybill (1972)):

xij = αi + βj + φuivj + εij ,

εij ∼ N(0, σ2), i = 1, . . . , I , j = 1, . . . , J.

This is a “rank one” interaction model

Consider testing the null hypothesis of no interaction H : φ = 0against this model.

The LR statistic is given by the largest singular value of (doublycentered) data matrix

(zij)I×J , with zij = xij − xi · − x·j + x··.




Consider an extension to the 3-way layout (Boik and Marasinghe,Kawasaki and Miyakawa):

xijk = (αβ)ij + (βγ)jk + (γα)ki + φuivjwk + εijk ,

εijk ∼ N(0, σ2), i = 1, . . . , I , j = 1, . . . , J, k = 1, . . . ,K .

Letzijk = xijk − xij · − xi ·k − x·jk + xi ·· + x·j · + x··k − x··· .

The LR statistic for testing H0 :φ = 0 is given by the “largest singularvalue of 3-way array”:

T = maxijk

(uivjwkzijk)

subject to∑

u2i =∑

v2j =∑

w2k = 1.




The null distribution of the LR statistic T has the canonical formT = maxu∈M u′z , where

z ∈ R(I−1)(J−1)(K−1) ∼ N(0, I(I−1)(J−1)(K−1))

and M = S I−2 ⊗ SJ−2 ⊗ SK−2 ⊂ S (I−1)(J−1)(K−1)−1.

For example, when I = J = K = 3,

P(T ≥ c) ∼ πG4(c2)− 3π

2G2(c

2)

as c → ∞;

P(U ≥ c) ∼ πB4,4(c2)− 3π

2B2,6(c

2)

for c ≥ 2/√7.




x

tail

prob

abili

ty

Monte Carlo simulationasymptotic expansionupper/lower bounds

x

tail

prob

abili

ty

0 1 2 3 4 5 6

0.0

0.2

0.4

0.6

0.8

1.0

x

tail

prob

abili

ty

x

tail

prob

abili

ty

Figure: Tail probability of T (I = J = K = 3)



Other uses of tube formula by Kuriki and Takemura

Other uses of tube formula by Kuriki and Takemura include

Test of multivariate normality based on maximized higher ordercumulants

Anderson-Stephens statistic for testing uniformity on the sphere

Maximum covariance difference test for equality of two covariancematrices

Distribution of projection pursuit index


Wishart matrix when population eigenvalues are dispersed

2. Asymptotic distribution of Wishart matrix when thepopulation eigenvalues are infinitely dispersed



Various asymptotics for Wishart distribution

Let W = (wij) be distributed according to Wishart distributionWp(n,Σ), where p is the dimension, n is the degrees of freedom andΣ is the covariance matrix.

We are interested in the joint distribution of the eigenvalues and theeigenvectors of W .

Exact distribution in terms of hypergeometric function of matrixarguments is still difficult.

Various approximations

Classical (large n, fixed p): Anderson (1963) and many subsequentauthors.Random matrix theory (large p) is now a large research field.

Here we propose yet another type of asymptotics.



Infinitely dispersed population eigenvalues

Denote the spectral decompositions of W and Σ by

W = GLG ′, Σ = ΓΛΓ′.

G and Γ: p × p orthogonal matrices.

L = diag(l1, . . . , lp), Λ = diag(λ1, . . . , λp) are diagonal with theeigenvalues l1 ≥ . . . ≥ lp > 0, λ1 ≥ . . . ≥ λp > 0 of W and Σ.

The population eigenvalues become infinitely dispersed if

ρ = ρ(Σ) = max(λ2λ1,λ3λ2, . . . ,

λpλp−1

) → 0.



Correct normalizations for infinite dispersion

LetG = (gij) = Γ′G .

Columns of G are eigenvectors in a standardized coordinate system.G is close to the identity matrix Ip.

Define

fi =liλi, 1 ≤ i ≤ p,

qij = gij l12j λ

− 12

i = gij f12j λ

12j λ

− 12

i , 1 ≤ j < i ≤ p.

(lower triangular part for G .)



Asymptotic distribution under infinite dispersion

Theorem 1 (Takemura-Sheena (2005))

As ρ = max(λ2/λ1, . . . , λp/λp−1) → 0,

fid→ χ2

n−i+1, 1 ≤ i ≤ p,

qijd→ N(0, 1), 1 ≤ j < i ≤ p,

and fi (1 ≤ i ≤ p), qij (1 ≤ j < i ≤ p) are asymptotically mutuallyindependently distributed.

Note the similarity of this result to the triangular decomposition of Wwhen Σ = Ip.



Why should the result hold?

It just comes from the Wishart density.

Let Σ = Λ be diagonal without loss of generality. Then gij = gij .

The joint density of l1, . . . , lp and gij , i > j , is

c|L|(n−p−1)/2

|Λ|n/2 exp(−1

2trGLG ′Λ−1)

∏

i<j

(li − lj)× J(G ),

where J(G ) is the Jacobian from lower triangular part of G to theuniform measure on the orthogonal group. This Jacobian actuallydoes not matter.




The Jacobian of the transformation

fi =liλi, qij = gij

√

fjλj/λi

is given as

∣

∣

∣

∣

∂(li , gij)

∂(fi , qij)

∣

∣

∣

∣

=

p∏

i=1

λi−(p−1)/2i

p∏

i=1

f−(p−i)/2i .

Hence the joint density of (fi , qij) is written as

c

p∏

i=1

f(n−i−1)/2i exp(−1

2trGLG ′Λ−1)

∏

i<j

(1− λj fjλi fi

)J(G )




As ρ→ 0∏

i<j

(1− λj fjλi fi

)J(G ) → J(Ip).

The essential part is exp(−12trGLG

′Λ−1):

trGLG ′Λ−1 =∑

i ,j

g2ij lj/λj =

∑

i

g2ii li/λi +

∑

i>j

q2ij +∑

i<j

g2ij lj/λj

→∑

i

fi +∑

i>j

q2ij

Hence the joint density converges to

c ′p∏

i=1

f−(n−i−1)/2i exp(−1

2

∑

i

fi −1

2

∑

i>j

q2ij).



Some generalizations

The result can be generalized to blockwise dispersion of populationeigenvalues, e.g., for some q

λ1 > · · · > λq ≫ λq+1 > · · · > λp.

Then blockwise asymptotic independence result holds.

We can also derive an asymptotic expansion in terms ofλ2/λ1, . . . , λp/λp−1.



Some uses of infinite dispersion

Our original motivation for the result was decision theoreticinvestigation of risk functions of estimators of the populationcovariance matrix at the boundary of the parameter space.

We proved results on “tail minimaxity” and non-minimaxity of someestimators.

We here present an application of blockwise result to a hypothesistesting problem.

Consider the null hypothesis on the mth (m = 1, . . . , p) populationeigenvalue

H(m)0 : λm ≥ λ∗m

against the alternative H(m)1 : λm < λ∗m.



Some uses of infinite dispersion

Theorem 2 (Sheena-Takemura (2007))

For testing hypothesis H(m)0 against H

(m)1 , a test with significance level γ is

given with the rejection region

lm ≤ l∗m(γ),

where l∗m(γ) is the lower 100γ% point of the smallest eigenvalue ofWm(n, λ

∗mIm).

This theorem follows from the fact that the limiting case λm+1/λm → 0,λ1 = · · · = λm, λm+1 = · · · = λp, gives the least favorable distribution forthe testing problem.



Some thoughts on different asymptotics

Are various asymptotics mutually exclusive?

The mixed asymptotics may be interesting

Prof. Fujikoshi and his collaborators are working on asymptoticexpansions where both n and p are large. They report good numericalresults.

It may be feasible to mix infinite dispersion and large n.


Star-shaped distributions

3. A generalization of elliptically contoured distributionsto star-shaped distributions



From elliptical contours to star-shaped contours

Contours of the density of elliptically contoured distributions areproportional ellipsoids.

How about densities with square contours?

How about something like the following?

Figure: From elliptical contours to general contours



From elliptical contours to star-shaped contoured

Suppose that the contours of the density are star-shaped andproportional with respect to the origin (concentric around the origin).

Every contour is obtained by a magnification of one specific contour,obtained by multiplication by a positive constant.

Then the independence of “length” and “direction” holds!

“Length”: on which contour the observation falls“Direction” : the relative position on the contour

Example in R2

g = g(x , y) = max|x |, |y |, θ = θ(x , y) = “angle”



From elliptical contours to star-shaped contours

g=1

g=2

g=3

x

y

θ

θ

Figure: Square contours

Under this kind of density, g and θ are independent.

Distribution of θ only depends on the shape (i.e. square) of thecontour.



General theory

The results on the previous page can be proved in general by the theory ofinvariant measures.

Let a group G act on the sample space X from the left:

(g , x) 7→ gx : G × X → X .

Gx = gx : g ∈ G : the orbit containing x ∈ XX/G = Gx : x ∈ X: the orbit space.

In the previous example, G = R∗+ is the multiplicative group of

positive reals and the orbits are rays emanating from the origin.



General theory

Let Gx = g ∈ G | gx = x denote the isotropy subgroup (stabilizer)of x .

When Gx = e for all x ∈ X , the action is said to be free, where edenotes the identity element of G.For discussing star-shaped distributions, we only need to consider thecase of free action.

For simplicity we assume free action in this talk. However the wholetheory can be generalized to the case of non-free action.



General theory

A cross section Z ⊂ X : a set Z intersects each orbit Gx , x ∈ X ,exactly once.

Orbital decomposition: each x ∈ X is uniquely written as

x = gz = g(x)z(x), g ∈ G, z ∈ Z.

g(x): equivariant part of x , “length”z(x): invariant part of x , “direction”



General theory

A relatively invariant dominating measure λ on X with multiplier χ :

λ(d(gx)) = χ(g)λ(dx), g ∈ G.

(χ(g) can be thought of as the Jacobian)

We call a density f (x) w.r.t. λ cross-sectionally contoured if it is ofthe form

f (x) = fG(g(x)).

Namely, f (x) only depends on the “length” g(x).

The contours of f are x | g(x) = c.



Theorem

Theorem 3 (Kamiya-Takemura-Kuriki (2008))

Suppose that x is distributed according to a cross-sectionally contoureddistribution fG(g(x))λ(dx). Under some regularity conditions on thetopology of X and the orbits, we have:

1 g = g(x) and z = z(x) are independently distributed.2 The distribution of z only depends on the cross section Z.



Back to the star-shaped distribution

G = R∗+, X = R

p − 0.G freely acts on X as

(g , (x1, . . . , xp)) 7→ (gx1, . . . , gxp).

The orbits under this action are rays emanating from the origin

A cross section Z is the boundary of a star-shaped set.

The Lebesgue measure dx is relatively invariant with multiplierχ(g) = gp.

We call a distribution with the density of the form f (x) = fG(g(x)) astar-shaped distribution.

From Theorem 3, under the star-shaped distribution g(x) and z(x)are independent and the distribution of z(x) depends only on Z.



Application to random matrices

The general theory can be applied to distributions other thanstar-shaped distributions.

We consider a pair of Wishart matrices.

Let W1 = (w1,ij) and W2 = (w2,ij) be two p × p positive definitematrices.

The sample space X is W = (W1,W2) : W1,W2 ∈ PD(p).As a dominating measure we consider

λ(dW ) = (detW1)a−

p+12 (detW2)

b−p+12 dW1dW2, (2)

where a, b > (p − 1)/2 anddW1 =

∏

1≤i≤j≤p dw1,ij , dW2 =∏

1≤i≤j≤p dw2,ij .




We consider the action of the lower triangular group.

Let LT (p) denote the group consisting of p × p lower triangularmatrices with positive diagonal elements.

G = LT (p) freely acts on X by

(T , (W1,W2)) 7→ (TW1T′, TW2T

′), T ∈ LT (p).

We consider the Cholesky decomposition of W1 +W2 = TT ′. Thenthe following beta-type cross section is standard:

Z ′ = (U, Ip − U) : 0 < U < Ip ⊂ PD(p)× PD(p),




The orbital decomposition of W = (W1,W2) w.r.t. Z ′

(W1,W2) =(

TUT ′, T (Ip − U)T ′)

, T = T (W ), U = U(W ). (3)

W1 +W2 and hence T (W ) is considered as “length”.

U(W ) = T−1W1(T−1)′ is considered as “direction”.




We now consider a general cross section.

Let S(U) be a function from U : 0 < U < Ip to LT (p).

Define a cross section

Z =(

S(U)US(U)′, S(U)(Ip − U)S(U)′)

, 0 < U < Ip

Corresponding to Z define

g(W ) = T (W )S(U(W ))−1. (4)




Consider a densityf (W ) = fG(g(W )) (5)

with respect to λ(dW ) in (2),

Let W have the density (5). By Theorem 3, g(W ) and U(W ) areindependently distributed.

Note that Theorem 3 states the independence of g(W ) andS(U(W ))U(W )S(U(W ))′. However since S(U(W ))U(W )S(U(W ))′

and U(W ) are in one-to-one correspondence (Z and Z ′ are inone-to-one correspondence), g(W ) and U(W ) are independentlydistributed.


Multivariate distributions defined via optimal transport

4. Multivariate distributions defined via optimal transport



Multivariate distributions via optimal transport

Transformation of a variable is convenient for univariate statisticalanalysis

→ ex. Box-Cox transformation

In the univariate case, by a monotone transformation of the formF−1(G (Y )), Y ∼ G , we can transform any cumulative distributionfunction G to any other cumulative distribution function F .

A multivariate generalization?→ The optimal transport theory is one answer.

HINT: A monotone function is a derivative of a convex function in R1.




x

ψ(x)

Figure: The slope is strictly increasing




Let Y = (Y1, . . . ,Yp)′ ∼ Np(0, I ).

Let ψ(x) = ψ(x1, . . . , xp) be a smooth and strictly convex function.We call ψ a “potential function”.

Assume that the gradient map

x 7→ ∇ψ(x) = (∂ψ

∂x1, . . . ,

∂ψ

∂xp)′

is onto Rp. (“co-finiteness” of ψ.)

∇ψ(x) is injective because ψ is strictly convex.

For x , y ∈ Rp, x 6= y , let ψ(t) = ψ(x + t(y − x)), which is smooth and

strictly convex in t ∈ R. Then ψ′(t) is strictly increasing.ψ′(0) = (y − x)′∇ψ(x), ψ′(1) = (y − x)′∇ψ(y).If ∇ψ(x) = ∇ψ(y), then ψ′(0) = ψ′(1) (a contradiction).

Therefore under co-finiteness, ∇ψ : Rp → Rp is bijective and smooth.




Then we can define a random vector X on Rp by the solution of

∇ψ(X ) = Y (6)

Let

∇∇′ψ(x) =

(

∂2ψ(x)

∂xi∂xj

)

i ,j=1,...,p

denote the Hessian matrix of ψ.

Then the density function of x is explicitly written as

p(x) =1

(2π)p/2exp(−1

2‖∇ψ(x)‖2) det(∇∇′ψ(x)).

(No need to worry about the normalizing constant.)




A trivial example: ψ(x) = a′x + 12x

′Bx (B is positive definite)

∇ψ(x) = a+ Bx

Hence a positive definite quadratic form corresponds to an affinetransformation.

The basic fact is that any distribution can be obtained from Np(0, I )by this kind of transformation.



Basic theorem

Theorem 4 (McCann (1995))

Let P and Q be two (arbitrary) probability measures on Rp absolutely

continuous w.r.t. the Lebesgue measure. Let Y be a random vector withthe distribution Q. Then there exists a convex function ψ on R

p such thatthe solution X of ∇ψ(X ) = Y has the distribution P. The function ψ isunique up to an arbitrary constant.



Basic theorem

This theorem is a consequence of the theory of optimal transport.

Consider the problem minimizing the cost

∫

Rp

‖x − T (x)‖2P(dx)

subject to a one-to-one map T that pushes P forward to Q:

X ∼ P ⇒ T (X ) ∼ Q.

It can be shown that the optimal map T is the gradient map ∇ψ inTheorem 4.



An example of 3-dimensional interaction

Let p = 3 and consider the potential function

ψ(x1, x2, x3) =1

2x ′x + θ

4∑

λ=1

arctan(e ′λx), |θ| < 2

3√3,

where e1 = (1, 1, 1)′, e2 = (−1, 1,−1)′, e3 = (−1,−1, 1)′,e4 = (1,−1,−1)′.

Note that ψ is convex for sufficiently small |θ|.Then the solution of ∇ψ(X ) = Y has the 3-dimensional interactionas long as θ 6= 0.



An example of 3-dimensional interaction

x1

x2

p(x1,x2|x3=1)

x1

x2

p(x1,x2|x3=−1)

(a) p(x1, x2|x3 = 1). (b) p(x1, x2|x3 = −1).

Figure: Conditional densities of 3-dim. interaction model (θ = 0.15).



An example of data analysis

We applied this model with a general quadratic form to detectthree-dimensional interaction of a real data.

The data consists of scores of high-jump (X1), 400m (X2), and 110mhurdle (X3) of 50 decathlon players (Miyakawa 1997).

Preprocess: We first normalized the data such that the sample meanand variance of each variable are 0 and 1, respectively.

Then the estimated potential function ψ(x1, x2, x3) was

1

2x ′

1.054 −0.094 0.137−0.094 1.138 −0.3070.137 −0.307 1.148

x + 0.1864

∑

λ=1

arctan(e ′λx).

AIC of this model was 10.4 less than the Gaussian model.



An example of data analysis

Indeed two empirical conditional correlations are quite different:cor(X2,X3|X1 > 0) = 0.688 and cor(X2,X3|X1 < 0) = −0.049.

-3 -2 -1 0 1 2 3

-3-2

-10

12

3

400m

110m

H

-3 -2 -1 0 1 2 3

-3-2

-10

12

3

400m

110m

H

-3 -2 -1 0 1 2 3

-3-2

-10

12

3

400m

110m

H

(a) all (b) X1 > 0 (c) X1 < 0“good high-jumper” “bad high-jumper”

Cor.= 0.465 Cor.= 0.688 Cor.= −0.049

Figure: 110mH v.s. 400m.



Exact sampling

Sampling from Np(0, I ) is easy.

Once we determine the gradient map ∇ψ, we can sample from p(x)exactly by solving ∇ψ(X ) = Y , where Y ∼ Np(0, I ).

The equation is equivalent to the following convex optimizationproblem:

X = argminx∈Rpψ(x)− Y ′x.We can efficiently solve this optimization problem by Newton’smethod.



A family of distributions based on gradient maps

Let ψ1(x), . . . , ψm(x) be smooth convex functions such that ∇ψi isonto R

p, i = 1, . . . ,m.

Let Θ be a convex subset of Rm.

Consider

ψ(x) = ψ(x , θ) = θ1ψ1(x) + · · ·+ θmψm(x), x ∈ Rp, θ ∈ Θ. (7)

This defines a family of distributions, which we call a “g -model”.

This family has the following nice property.

Theorem 5 (Sei (2010))

The log-likelihood function of g-model is concave.



Transformation from a uniform distribution

We can also take the uniform distribution on the unit cube [0, 1]p asthe standard distribution: Y ∼ U[0, 1]p.

If we appropriately specify ψ, we can obtain a class of multivariatedistributions on the cube.

Example in p = 2.

Let

ψ(x1, x2 | θ) =1

2(x21 + x22 )−

θ

π2cos(πx1) cos(πx2)

Then the density of x1, x2 is given as

p(x1, x2 | θ) = det

(

1 + θ cos(πx1) cos(πx2) −θ sin(πx1) sin(πx2)−θ sin(πx1) sin(πx2) 1 + θ cos(πx1) cos(πx2)

)

= 1 + 2θ cos(πx1) cos(πx2) +θ2

2

(

cos(2πx1) + cos(2πx2))

.



Transformation from a uniform distribution

x1 x2

density

x1 x2

density

(a) θ = 0.5. (b) θ = −0.5.

Figure: The probability density p(x1, x2|θ) for θ = ±0.5.



R. J. Adler. The Geometry of Random Fields. Wiley, Chichester. 1981.

R. J. Adler and A. M. Hasofer. Level crossings for random fields. Ann. Probab., 4, 1–12.1976.

Robert J. Adler and Jonathan E. Taylor. Random Fields and Geometry, Springer, NewYork. 2007.

T. W. Anderson. Asymptotic theory for principal component analysis. Ann. Math. Statist.,36, 413–432. 1963.

R. J. Boik and M. G. Marasinghe. Analysis of nonadditive multiway classifications. J.Amer. Statist. Assoc., 84, 1059–1064. 1989.

H. Hotelling, H. Tubes and spheres in n-spaces, and a class of statistical problems. Amer.

J. Math., 61, 440–460. 1939.

D. E. Johnson and F. A. Graybill. An analysis of a two-way model with interaction and noreplication. J. Amer. Statist. Assoc., 67, 862–868. 1972.

Hidehiko Kamiya and A. Takemura. Hierarchical orbital decompositions and extendeddecomposable distributions. Journal of Multivariate Analysis, 99, 339–357. 2008.

Hidehiko Kamiya, A. Takemura and Satoshi Kuriki. Star-shaped distributions and theirgeneralizations. Journal of Statistical Planning and Inference, 138, 3429–3447. 2008.

M. Knowles and D. Siegmund. On Hotelling’s approach to testing for a nonlinearparameter in regression. Internat. Statist. Rev., 57, 205–220. 1989.



H. Kawasaki and M. Miyakawa. A test of three-factor interaction in a three-way layoutwithout replication. Quality, JSQC , 26, 97–108 (in Japanese). 1996.

Satoshi Kuriki. Asymptotic distribution of inequality-restricted canonical correlation withapplication to tests for independence in ordered contingency tables. Journal of Multivariate

Analysis, 94, 420–449. 2005.

Satoshi Kuriki and A. Takemura. Distribution of the maximum of Gaussian random field:tube method and Euler characteristic method. (in Japanese.) Proceedings of the Institute

of Statistical Mathematics. 47, 201–221. 1999.

Satoshi Kuriki and A. Takemura. Some geometry of the cone of nonnegative definite

matrices and weights of associated χ2 distribution, Ann. Inst. Statist. Math. 52, 1–14.2000.

Satoshi Kuriki and A. Takemura. Shrinkage estimation towards a closed convex set with asmooth boundary. Journal of Multivariate Analysis. 75, 79–111. 2000.

Satoshi Kuriki and A. Takemura. Tail probabilities of the maxima of multilinear forms andtheir applications, Annals of Statistics, 29, 328–371. 2001.

Satoshi Kuriki and A. Takemura. Application of tube formula to distributional problems inmultiway layouts. Applied Stochastic Models in Business and Industry, 18, 245–257. 2002.

Satoshi Kuriki and A. Takemura. Tail probabilities of the limiting null distributions of theAnderson-Stephens statistics. Journal of Multivariate Analysis, 89, 261–291. 2004.

Satoshi Kuriki and A. Takemura. The tube method for the moment index in projectionpursuit. Journal of Statistical Planning and Inference, 138, No.9, 2749–2762. 2008.



Satoshi Kuriki and A. Takemura. Euler characteristic heuristic for approximating thedistribution of the largest eigenvalue of an orthogonally invariant random matrix. Journalof Statistical Planning and Inference, 138, 3357–3378. 2008.

Satoshi Kuriki and A. Takemura. Volume of tubes and distribution of the maxima ofGaussian random fields. Selected Papers on Probability and Statistics, AmericanMathematical Society Translations, Series 2, Volume 227, pp.25–48. (Translation of thearticle in Sugaku, 60, 2008). 2009.

Robert J. McCann.Existence and uniqueness of monotone measure-preserving maps.Duke Math. J., 80(2):309–323, 1995.

M. Miyakawa. Graphical Modeling . (in Japanese), Asakura Shoten, Tokyo. 1997.

D. Q. Naiman and H. P. Wynn. Inclusion-exclusion-Bonferroni identities and inequalitiesfor discrete tube-like problems via Euler characteristics. Ann. Statist, 20, 43–76. 1992.

D. Q. Naiman and H. P. Wynn. Abstract tubes, improved inclusion-exclusion identities andinequalities and importance sampling. Ann. Statist, 25, 1954–1983. 1997.

R. Tyrrell Rockafellar.Convex Analysis.Princeton University Press, Princeton, 1972.

Tomonari Sei. Gradient modeling for multivariate quantitative data. Annals of the Institute

of Statistical Mathematics (published online). 2010.



Tomonari Sei. A structural model on a hypercube represented by optimal transport. Toappear in Statistica Sinica. 2010.

Yo Sheena and A. Takemura. An asymptotic expansion of Wishart distribution when thepopulation eigenvalues are infinitely dispersed. Statistical Methodology, 4, 158–184. 2007.

Yo Sheena and A. Takemura. Inference on eigenvalues of Wishart distribution usingasymptotics with respect to the dispersion of population eigenvalues. Sankhya, 69, No.4,717–733. 2007.

Yo Sheena and A. Takemura. Asymptotic distribution of Wishart matrix for block-wisedispersion of population eigenvalues. Journal of Multivariate Analysis, 99, 751–775. 2008.

J. Sun. Significance levels in exploratory projection pursuit. Biometrika, 78, 759–769. 1991.

J. Sun, J. Tail probabilities of the maxima of Gaussian random fields. Ann. Probab., 21,34–71. 1993.

A. Takemura and Satoshi Kuriki. Weights of χ2 distribution for smooth or piecewisesmooth cone alternatives. Annals of Statistics, 25, 2368–2387. 1997.

A. Takemura and Satoshi Kuriki. Shrinkage to smooth non-convex cone: principalcomponent analysis as Stein estimation. Commun. Statist. – Theory Meth. (in honor of N.Sugiura), 28, 651–669. 1999.

A. Takemura and Satoshi Kuriki. Maximum covariance difference test for equality of twocovariance matrices. in Algebraic Methods in Statistics and Probability. pp.283–301, M.Viana and D. Richards eds., Contemporary Mathematics Vol. 287, American MathematicalSociety. 2001.



A. Takemura and Satoshi Kuriki. On the equivalence of the tube and Euler characteristicmethods for the distribution of the maximum of Gaussian fields over piecewise smoothdomains, Annals of Applied Probability, 12 , 768–796. 2002.

A. Takemura and Satoshi Kuriki. Tail probability via the tube formula when the criticalradius is zero. Bernoulli, 9, No.3, 535–558. 2003.

A. Takemura and Yo Sheena. Distribution of eigenvalues and eigenvectors of Wishartmatrix when the population eigenvalues are infinitely dispersed and its application tominimax estimation of covariance matrix. Journal of Multivariate Analysis, 94, 271–299.2005.

Jonathan Taylor, A. Takemura and Robert J. Adler. Validity of the expected Eulercharacteristic heuristic. Annals of Probability, 33, 1362–1396. 2005.

Cedric Villani.Topics in Optimal Transportation.AMS, Providence, 2003.

Cedric Villani.Optimal Transport - Old and New.Springer, Berlin, 2009.

H. Weyl. On the volume of tubes. Amer. J. Math., 61, 461–472. 1939.


some nonstandard distributions and asymptotics for...

Documents