Partially Linear Additive Gaussian GraphicalModels

Presented by: Sinong GengPrinceton University

June 13th, 2019 @ ICML 2019

Authors

Sinong GengPrincetonUniversity

Minhao YanCornell

University

Mladen KolarThe University of

Chicago

Sanmi KoyejoUniversity of

Illinois

Poster: Partially Linear Additive Gaussian Graphical ModelsThu Jun 13th 06:15 – 09:00 PM @ Pacific Ballroom

2

Brain Functional Connectivity Analysis

Estimation is distorted by physiological noise [Van Dijk et al.,2012, Goto et al., 2016].The noise sources are observable e.g. motion, breathing…

3

Brain Functional Connectivity Analysis

Estimation is distorted by physiological noise [Van Dijk et al.,2012, Goto et al., 2016].

The noise sources are observable e.g. motion, breathing…

3

Brain Functional Connectivity Analysis

Estimation is distorted by physiological noise [Van Dijk et al.,2012, Goto et al., 2016].The noise sources are observable e.g. motion, breathing… 3

Model Formulation: Goals

→ A general formulation of the effects caused by the noise.

→ Stronger theoretical guarantees compared tp methodswith hidden variables.

4

Model Formulation

→ Z denotes the observed fMRI data, and random variableG, the physiological noise.

→ Z | G = g follows a Gaussian graphical model [Yang et al.,2015] with a parameter matrix, denoted by Ω(g):

P(Z = z;Ω(g) | G = g) ∝ exp

p∑

j=1

Ωjj(g)zjp∑

j=1

p∑j′>j

Ωjj′(g)zjzj′ −1

2

p∑j

z2j

.

→ Parameter matrices are additive:

Ω(g) := Ω0 + R(g).

5

Model Formulation: Ω(g)Goals:

• Identifiable parameters• A general formulation

Assumptions:• R(g) = 0 for any g satisfying |g| ≤ g∗.• R(g), and Ω(g) are smooth enough to be recoveredby kernel methods.

Existing assumptions:• R(g) = 0 [Van Dijk et al., 2012, Power et al., 2014].• E(R(g)) = 0 [Lee and Liu, 2015, Geng et al., 2018].

6

Model Formulation: Ω(g)Goals:

• Identifiable parameters• A general formulation

Assumptions:• R(g) = 0 for any g satisfying |g| ≤ g∗.• R(g), and Ω(g) are smooth enough to be recoveredby kernel methods.

Existing assumptions:• R(g) = 0 [Van Dijk et al., 2012, Power et al., 2014].• E(R(g)) = 0 [Lee and Liu, 2015, Geng et al., 2018].

6

Model Formulation: Ω(g)Goals:

• Identifiable parameters• A general formulation

Assumptions:• R(g) = 0 for any g satisfying |g| ≤ g∗.• R(g), and Ω(g) are smooth enough to be recoveredby kernel methods.

Existing assumptions:• R(g) = 0 [Van Dijk et al., 2012, Power et al., 2014].• E(R(g)) = 0 [Lee and Liu, 2015, Geng et al., 2018].

6

Parameter EstimationLog Pseudo Likelihood:

•We summarize the varying effects as Mij := x⊤ij Ωi·j,where x⊤ij denotes the ith row vector of xj.•

ℓPL

(zi,gii∈[n] ;R(·),Ω0

)=

n∑i=1

p∑j=1

zij

(x⊤ij Ω0·j +Mij

)− 1

2z2ij

− 1

2

(x⊤ij Ω0·j +Mij

)2.

7

Parameter Estimation• Pseudo-Profile Likelihood [Fan et al., 2005]• Suppose that Assumptions are satisfied. Then, for anyϵ > 0, with probability of at least 1− ϵ, there exists C4 > 0,so that Ω0 shares the same structure with the underlyingtrue parameter Ω∗

0, if for some constant C5 > 0,

C5

√logpn

≥ λ ≥ 4

αC4

√logpn

,

r := 4C2λ ≤ ∥Ω∗0S∥∞ ,

and n ≥(64C5C2

2C3/α)2 logp.

8

Parameter Estimation

Sparsistency: The underlying structure can be recoveredwith a high probability.

√n Convergence: The smallest scale of the non-zero

component that the PPL method can distinguish fromzero converges to zero at a rate of

√n.

9

Overall Performance

→ LR-GGM→ fMRI dataset with control

subjects and those withSchizophrenia.

→ Diagnosis using the re-covered structure by twodifferent methods. 0

10

20

0.50 0.55 0.60 0.65 0.70 0.75

AUCs

Frequ

en

cy

MethodsPLA−GGMLR−GGM

10

Thank you!

References I

J. Fan, T. Huang, et al. Profile likelihood inferences onsemiparametric varying-coefficient partially linearmodels. Bernoulli, 11(6):1031–1057, 2005.

S. Geng, M. Kolar, and O. Koyejo. Joint nonparametricprecision matrix estimation with confounding. arXivpreprint arXiv:1810.07147, 2018.

M. Goto, O. Abe, T. Miyati, H. Yamasue, T. Gomi, andT. Takeda. Head motion and correction methods inresting-state functional mri. Magnetic Resonance inMedical Sciences, 15(2):178–186, 2016.

12

References II

W. Lee and Y. Liu. Joint estimation of multiple precisionmatrices with common structures. The Journal of MachineLearning Research, 16(1):1035–1062, 2015.

J. D. Power, A. Mitra, T. O. Laumann, A. Z. Snyder, B. L.Schlaggar, and S. E. Petersen. Methods to detect,characterize, and remove motion artifact in resting statefmri. Neuroimage, 84:320–341, 2014.

K. R. Van Dijk, M. R. Sabuncu, and R. L. Buckner. Theinfluence of head motion on intrinsic functionalconnectivity mri. Neuroimage, 59(1):431–438, 2012.

13

References III

E. Yang, P. Ravikumar, G. I. Allen, and Z. Liu. Graphicalmodels via univariate exponential family distributions.Journal of Machine Learning Research, 16(1):3813–3847,2015.

14