Likelihood Ratio Testing under Non-identifiability:

Theory and Biomedical Applications

Kung-Yee Liang and Chongzhi DiDepartment Biostatistics

Johns Hopkins University

July 9-10, 2009National Taiwan University

Outline• Challenges associated with likelihood inference• Nuisance parameters absent under null hypothesis

– Some biomedical examples

– Statistical implications

• Class I: alternative representation of LR test statistic– Implications

• Class II– Asymptotic null distribution of LR test statistic

– Some alternatives

• A genetic linkage example• Discussion

Likelihood InferenceLikelihood inference has been successful in a

variety of scientific fields• LOD score method for genetic linkage

– BRCA1 for breast cancerHall et al. (1990) Science

• Poisson regression for environmental health– Fine air particle (PM10) for increased mortality in total

cause and in cardiovascular and respiratory causes Samet et al. (2000) NEJM

• ML image reconstruction estimate for nuclear medicine– Diagnoses for myocardial infarction and cancers

Challenges for Likelihood Inference

• In the absence of sufficient substantive knowledge, likelihood function maybe difficult to fully specify– Genetic linkage for complex traits– Genome-wide association with thousands of SNPs– Gene expression data for tumor cells

• There is computational issue as well for high-dimensional observations– High throughput data

Challenges for Likelihood Inference (con’t)

• Impacts of nuisance parameters– Inconsistency of MLE with many nuisance

parameters (Neyman-Scott problem)– Different scientific conclusions with different

nuisance parameter values– Ill-behaved likelihood function

• Asymptotic approximation not ready

Parametric Linkage Analyses - 123 multiplex families - 4 models

0

1

2

3

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150 160 170

genetic distance in CentiMorgans

HL

OD

Dominant1_HLOD recessive1_HLOD dominant2_HLOD recessive2_HLOD

0.0 0.2 0.4 0.6 0.8 1.0

-12

0-1

10

-10

0-9

0-8

0-7

0Beta-Binomial Log-Likelihood

Lo

g li

kelih

oo

d

L0.80.8L0.60.8

0.056

L0.80.1L0.60.1

336.9

0.1

0.5

0.3

0.8

Challenges for Likelihood Inference (con’t)

• There are situations where some of the “regularity” conditions may be violated– Boundary issue (variance components, genetic

linkage, etc.)Self & Liang (1987) JASA

– Discrete parameter spaceLindsay & Roeder (1986) JASA

– Singular information matrix (admixture model)Rotnitzky et al (2000) Bernoulli

Nuisance Parameters Absent under Null

Ex. I.1 (Polar coordinate for bivariate normal)

Under H0: δ = 0, γ is absent

Davies (1977) BiometrikaAndrews and Ploberger (1994) Econometrica

10

01,

sin

cos...

2

1 ~

BVNY

Y dii

i

i

Examples (con’t)

Ex. I.2 (Sterotype model for ordinal categorical response)

For Y = 2, .., C,

log Pr(Y = j)/Pr(Y = 1) = αj + βjtx, j = 2,..,C

= αj + φj βtx

0 = φ1 ≤ φ2 … ≤ φC = 1

Under H0: β = 0, φj ’s are absent

Anderson (1984) JRSSB

Examples (con’t)

Ex. I.3 (Variance component models)In certain situations, the covariance matrix of

continuous and multivariate observations could be expressed as

δM(γ) + λ1M1 + … +λqMq

A hypothesis of interest is H0: δ = 0 (γ is absent)

Ritz and Skovgaard (2005) Biometrika

Examples (con’t)

Ex. I.4 (Gene-gene interactions)

Consider the following logistic regression model:logit Pr(Y = 1|S1, S2) = α + Σk δkS1k + Σj λkS2j

+ γ Σk Σj δk λj S1k S2j

To test genetic association between gene one (S1) by taking into account potential interaction with gene two (S2), the hypothesis of interest is

H0: δ1 = … = δK = 0 (γ is absent)

Chatterjee et al. (2005) American Journal of Human Genetics

Examples (con’t)

Ex. II.1 (Admixture models)

f(y; δ, γ) = δ p(y; γ) + (1 – δ) p(y; γ0)

δ: proportion of linked families

γ: recombination fraction (γ0= 0.5)

Smith (1963) Annals of Human Genetics

The null hypothesis of no genetic linkage can be cast asH0: δ = 0 (γ is absent) or H0: γ = γ0 (δ is absent)

Examples (con’t)

Ex. II.2 (Change point)

logit Pr(Y = 1|x) = β0 + βx + δ(x – γ)+

(x – γ)+ = x – γ if x – γ > 0 and 0 if otherwise

• Alcohol consumption protective for MI when consuming less than γ, but harmful when exceeding the threshold

Pastor et al. (1998) American Journal of Epidemiology

Hypothesis of no threshold existing can be cast asH0: δ = 0 (γ is absent) or H0: γ = ∞ (δ is absent)

Examples (con’t)

Ex. II.3 (Non-linear alternative)

logit Pr(Y = 1|x) = β0 + βx + δh(x; γ)

e.g., h(x; γ) = exp(xγ) – 1

• The effect of alcohol consumption on risk, through log odds, of MI is non-linear if γ ≠ 0

The hypothesis of linearity relationship with a specific non-linear alternative can be cast asH0: δ = 0 (γ is absent) or H0: γ = 0 (δ is absent)

Gallant (1977) JASA

Characteristics of ExamplesMajority of examples can be characterized as

f(y, x; δhy,x(γ, β), β)

Class I Class II

H0: δ = 0 H0: δ = 0 or γ = γ0 ( hy,x(γ0, β) = 0 )

Figure: expected log likelihood function for three cases

Class I: Asymptotic

For H0: δ = 0,

1. LRT = 2{logL( ) – logL(0, •)}

= supγ 2{logL( , γ) – logL(0, •)}

= supγ LRT(γ)

2. LRT(γ) = S(γ)t I-1(γ)S(γ) + op(1) = W2(γ) + op(1),

where S(γ) = ∂logL(δ, γ)/∂δ|δ=0, I(γ) = var{S(γ)}

W(γ) = I-1/2(γ) S(γ)

and W(γ) is a Gaussian process in γ with mean 0, variance 1 and autocorrelation ρ(γ1, γ2) = cov{W(γ1), W(γ2)}

ˆ,ˆ

Class I: Asymptotic (con’t)

• Results were derived previously by Davies (1977, Biometrika)

• No analytical form available in general

• Approximation, simulation or resampling methods Kim and Siegmund (1989) Biometrika

Zhu and Zhang (2006) JRSSB

Q.: Can simplification be taken place for

• Asymptotic null distribution?

• Approximating the p-value?

Class I: Principal Component Representation

Principal component decomposition

• K could be finite or ∞• {ξ1, …,ξK} are independent r.v.’s• ξk ~ N(0, λk), k = 1, .., K• ρ(γ, γ) = Σk λk ωk(γ)2 = 1

)()(),(,)()( 21

1211

k

K

kkk

K

kkkW

Class I: Principal Component Representation (con’t)

W2(γ) = {Σk ξk ωk(γ)}2

≤ {Σk ξk2/λk} {Σk λk ωk(γ)2} = {Σk ξk

2/λk}

Consequently, one has

supγ W2(γ) = supγ {Σk ξk ωk(γ)}2 ≤ {Σk ξk2/λk} ~

• The asymptotic distribution of LRT under Ho is bounded by

2K

2K

Class I: Simplification

• Simplify to if K < ∞ and for almost every

(ξ1, …, ξK), there exists γ such that

λ1ω1(γ)/ξ1 = ….. = λKωK(γ)/ξK Ex. I.1. (Polar coordinate for bivariate normal)

For any , there exists γ ε [0, π) such that

• LRT ~ instead of

H0: δ = 0 ↔ H0: μ1 = μ2

2K

21

22

)()sincos()(

)cos(),(),sincos()(2

22

12

212

2121212/1

YYnYYnW

YYnW

221 ),( RYY

sin/cos/ 21 YY

Class I: Simplification (con’t)

• Simplify to if S(γ) = h(γ) g(Y)– ρ(γ1, γ2) = 1

Ex.: Modified admixture models

γ p(y; δ) + (1 – γ) p(y; δ0), γ ε [a, 1] with a > 0 fixed

H0: δ = δ0 (γ is absent) and the score function for δ at δ0 is

S(γ) = γ ∂logp(y; δ0)/∂δ• Known as restricted LRT for testing H0: δ = δ0

• Has been used in genetic linkage studiesLamdeni and Pons (1993) BiometricsShoukri and Lathrop (1993) Biometrics

21

Class I: Approximation for P-values

When simplification fails: Step 1: Calculate W(γ) and ρ(γ1, γ2)

Step 2: Estimate eigenvalues {λ1, …, λK} and eigenfunctions {ω1, …, ωK}, where K is chosen so that first K components explain more than 95% variation

Step 3: Choose a set of dense grid {γ1, …, γM} and for i = 1, …, N, repeat the following steps:– Simulate ξik ~ N(0, λk) for k = 1, …, K

– Calculate Wi(γm) = {Σk ξik ωk(γm)}2 for each m

– Find the maximum of {Wi(γ1), …, Wi(γM)}, Ri say

{R1, …, RN} approximates the null distribution of LRT

Class II: Some New Results

• Consider the class of familyf(y, x; δhy,x(γ, β), β),

where hy,x(γ0, β) = 0 for all y and xH0: δ = 0 or γ = γ0

Tasks: 1. Derive asymptotic distribution of LRT under H0

2. Present alternative approaches• Illustrate through Ex. II.1 (Admixture models)

δ Binom (m, γ) + (1 – δ) Binom (m, γ0) δ ε [0, 1] and γ ε [0, 0.5], hy,x(γ, β) = p(y; γ) – p(y; γ0) – For simplicity, assuming β is absent

Class II: LRT Representation

Under H0, f(y, x; δ, γ) = f(y, x; 0, • ) = f(y, x; • , γ0 ),

LRT = supδ,γ 2{logL(δ, γ) – logL(0, • )} = supδ,γ LRT(δ, γ)

= max {sup1,4 LRT(a), sup2,4 LRT(b), sup3 LRT(a, b)},

here for fixed a, b > 0 and γ0 = 0.5:

Region 1: δ ε [a, 1], γ ε [0.5 – b, 0.5]

Region 2: δ ε [0, a], γ ε [0, 0.5 – b]

Region 3: δ ε [0, a], γ ε [0.5 – b, 0.5]

Region 4: δ ε [a, 1], γ ε [0, 0.5 – b]

Class II: Four Sub-Regions of Parameter Spaces

Class II: Regions 1 & 4

With δ ε [a, 1], this reduces to Class I, and sup1,4 LRT(a)= supδ { + op(1)}

W1(δ) = I1-½(δ)S1(δ)

S1(δ) = ∂logL(δ, γ)/∂γ|γ = γ0, I1(δ) = var(S1(δ))

For the admixture models,

• S1(δ) = δ ∂log p(y; γ0)}/∂γ, which is proportional to δ

• is independent of δ

)(21 W

)(21 W

Class II: Regions 2 & 4

With γ ε [0, γ0 – b], this reduces to Class I, and

sup2,4 LRT(b)= supγ { + op(1)}

W2(γ) = I2-½(γ)S2(γ)

S2(γ) = ∂logL(δ, γ)/∂δ|δ = 0, I2(γ) = var(S2(γ))

For the admixture models,

• S2(γ) = {p(y; γ) – p(y; γ0)}/p(y; γ0)

• W2(γ) → ∂log p(y; γ0)}/∂γ = W1 as γ → γ0 (or b → 0)

)(22 W

Class II: Region 3

With δ ε [0, a], γ ε [0.5 – b, 0.5], expand at [0, 0.5]sup3 LRT(a, b) = supδ,γ { + op(1)}

W3 = I3-½S3

S3 = ∂2logp(y; 0, 0.5)/∂δ∂γ, I3 = var(S3)

For the admixture models,

• S3 = ∂log p(y; γ0)}/∂γ and W3 = W1

23W

Class II: Asymptotic Distribution of LRT

Combining three regions and let a, b → 0,LRT = max {sup1,4 LRT(a), sup2,4 LRT(b), sup3 LRT(a,b)}

= max { , sup [W2(γ)]2, }

→ sup {W2(γ)}2 ,

where • The asymptotic null distribution of LRT is supremum of

squared Gaussian process w.r.t. γ• Simplification (null distribution and approximation to p-

values) can be adopted from Class I

23W

3122 )(lim)5.0(0

WWWW

21W

Class II: Alternatives

Question: Can one find alternatives test statistics with conventional asymptotic null distributions?

1. Restricted LRT: limit range for δ to [a, 1] with a > 0 (Region 1 and 4)

TR(a) = sup1,4 LRT(a) =

• TR(a) decreases in a

• How to choose a?– Smaller the a, the better

– Chi-square approximation maybe in doubt

21

21 ~)1( poW

Class II: Alternatives (con’t)

2. Smooth version (penalized LRT)

Instead of excluding (δ, γ) values in Regions 2 & 3, they are “penalized” toward δ = 0 by considering penalized log-likelihood:

PL(δ, γ; c) = log L(δ, γ) + c g(δ),

where g(δ) ≤ 0 is a smooth penalty with , maximized at δ0 and c > 0 controlling the magnitude of penalty

• Bayesian interpretation – g(δ) could be “prior” on δ

)(lim0

g

Class II: Penalized likelihood

Define the penalized LR test statistic for H0: γ = γ0

PLRT(c) = 2 {sup PL(δ, γ; c) – PL(δ0, γ0; c)}

Under H0: PLRT(c) → as n → ∞• Proof is more demanding*• Similar concerns to restricted LRT

– Decreasing in c– Chi-square approximation maybe invalid with small c– Lose power if δ is small (linked families is small in

proportion)*Different approach for mixture/admixture model was provided by

by Chen et al. (2001, 2004) and Fu et al. (2006)

21

Application: Genetic Linkage for Schizophrenia

• Conducted by Ann Pulver at Hopkins

• 486 individuals from 54 multiplex families

• Interested in marker D22S942 in chromosome 22

• Schizophrenia is relatively high in prevalence with strong evidence of genetic heterogeneity

• To take into account of this phenomenon, consider

δ Binom (m, γ) + (1 – δ) Binom (m, γ0),

where δ ε [0, 1], γ ε [0, 0.5] with γ0 = 0.5

Genetic Linkage Study of Schizophrenia (con’t)

With this admixture model considered,•

• LRT = 6.86, p-value = 0.007• PLRT with

– PLRT(3.0) = 5.36, p-value = 0.010– PLRT(0.5) = 5.49, p-value = 0.009– PLRT(0.01) = 6.84, p-value = 0.004

• Which p-value do we trust better?

06.0ˆ,4.0ˆ

)1,loglim(log)( 00

g

Figure: asymptotic vs empirical distribution of the LRT for genetic linkage example

Discussion

• Issue considered, namely, nuisance parameters absent under null, is common in practice

• Examples can be classified into Class I and II– Class I: H0 can only be specified through δ (= 0)– Class II: H0 can specified either in δ (= 0) or γ (= γ0)

• For Class I, asymptotic distribution of LRT is well known, and through principal component representation– Deriving sufficient conditions for simple null

distribution– Proposing a means to approximate p-values

Discussion (con’t)

• For Class II, less well developed– Deriving asymptotic null distribution of LRT

• Through this derivation, we observe– Connection with Class I– Connection with RLRT and PLRT

• Proof on asymptotic of PLRT non-trivial• Shedding light on why penalty applied to δ not to γ?• Pointing out some peculiar features and shortcoming of

these two approaches

• A genetic linkage example on schizophrenia was presented for illustration

Discussion (con’t)

Some future work:

• Constructing confidence intervals/regions

• Generalizing to partial/conditional likelihood– Cox PH model with change point (nuisance function)

• Extending to estimating function approach in the absence of likelihood function to work with– Linkage study of IBD sharing for affected sibpairsE(S(t)) = 1 + (1 - 2θt,γ)2 E(S(γ)) = 1 + (1 - 2θt,γ)2 δ

θt,γ = (1 – exp(–0.02|t – γ|)/2