Implications of Implications of Model Model

Specification and Specification and Temporal Revisit Temporal Revisit Designs on Trend Designs on Trend

DetectionDetectionLeigh Ann Starcevich (OSU)Leigh Ann Starcevich (OSU)

Kathryn M. Irvine (USGS) Kathryn M. Irvine (USGS)

Andrea M. Heard (UCR, NPS)Andrea M. Heard (UCR, NPS)

OutlineOutline Question of interestQuestion of interest Case studyCase study Trend modelsTrend models Simulation resultsSimulation results

Question of interestQuestion of interest Trend estimation and testingTrend estimation and testing Components of variation impact power Components of variation impact power

to detect trend to detect trend Three approaches to trend estimation Three approaches to trend estimation

and testing suggested by:and testing suggested by: Urquhart, Birkes, and Overton (1993)Urquhart, Birkes, and Overton (1993) Piepho and Ogutu (2002)Piepho and Ogutu (2002) Kincaid, Larsen, and Urquhart (2004)Kincaid, Larsen, and Urquhart (2004)

Which approach has highest power for Which approach has highest power for trend detection for a given Type I error trend detection for a given Type I error level?level?

MotivationMotivation SIEN lake chemistry monitoringSIEN lake chemistry monitoring Trend and status Trend and status Annual effort is limited in vast Annual effort is limited in vast

landscapelandscape ~800 lakes in network~800 lakes in network

Sierra Nevada Network Sierra Nevada Network LakesLakes

Sequoia-Kings Canyon NP: ~ 860,000 acres Yosemite NP: ~ 760,000 acres

Trend modelTrend model Linear mixed model used to estimate Linear mixed model used to estimate

trend and components of variancetrend and components of variance Trend models contain both fixed and Trend models contain both fixed and

random effectsrandom effects Fixed effects Fixed effects describe the mean describe the mean Random effects Random effects describe the variance describe the variance

structurestructure

10 β j i i ijijk i kj jj b ay w w t c e

Variance componentsVariance components Site-to-site variation (Site-to-site variation (σσaa

22) ) Does not affect the trend estimate or SE when Does not affect the trend estimate or SE when

same sites visited annually (Piepho and Ogutu, same sites visited annually (Piepho and Ogutu, 2002)2002)

Year-to-year variation (Year-to-year variation (σσbb22) )

Visiting additional sites will not increase power to Visiting additional sites will not increase power to detect trend when year-to-year variation is highdetect trend when year-to-year variation is high

Site-by-year variation (Site-by-year variation (σσcc22) )

Requires within-year visits to a site (not estimable Requires within-year visits to a site (not estimable in SIEN lakes survey)in SIEN lakes survey)

Random slope variation (Random slope variation (σσtt22) )

Might indicate subpopulations with different Might indicate subpopulations with different trendstrends

Residual error variation (Residual error variation (σσee22) )

Power to detect trendPower to detect trend Affected by:Affected by:

Type I error level (Type I error level (αα)) Trend magnitude (Trend magnitude (ββ11)) Variance compositionVariance composition

Associated with a particular hypothesis Associated with a particular hypothesis testtest One-sided vs. two-sided alternative hypothesis?One-sided vs. two-sided alternative hypothesis? Reflects monitoring goalsReflects monitoring goals We examine: We examine:

1 1:β 0 vs. :β 0O AH H

Revisit designsRevisit designs

Panel designs allow more sites to be Panel designs allow more sites to be visited over timevisited over time

When all sites are not visited annually, When all sites are not visited annually, data are purposefully unbalanceddata are purposefully unbalanced

Serially-alternating augmented designs Serially-alternating augmented designs consideredconsidered Connected across time for powerful trend Connected across time for powerful trend

teststests Incorporates more sites for status estimatesIncorporates more sites for status estimates

[1-0][1-0]

11 22 33 44 55 66 77 88

XX XX XX XX XX XX XX XX

Notation of McDonald (2003)Notation of McDonald (2003) Urquhart and Kincaid (1999) showed Urquhart and Kincaid (1999) showed

that [1-0] is best for trend estimationthat [1-0] is best for trend estimation

[(1-0),(1-3)][(1-0),(1-3)]

11 22 33 44 55 66 77 88

XX XX XX XX XX XX XX XX

XX XX

XX XX

XX XX

XX XX

[(1-0),(1-8)][(1-0),(1-8)]11 22 33 44 55 66 77 88 99

XX XX XX XX XX XX XX XX XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

Estimation of fixed Estimation of fixed effectseffects

Generalized least squares (GLS) used to estimate fixed effectsGeneralized least squares (GLS) used to estimate fixed effects

ˆ ˆ ˆ -1-1 -1

= X Φ θ X X Φ θ Y

wherewhere

2 2 2 2ˆ , , , ,

ˆ Y

a b t e

Var

X

Φ

1,w

θ

θ

Estimation of RE variance Estimation of RE variance componentscomponents

ANOVA Type III when data are balancedANOVA Type III when data are balanced REML for unbalanced dataREML for unbalanced data

Piepho & Ogutu (2002)Piepho & Ogutu (2002); Spilke, et al. (2005) ANOVA Type III and REML provide the ANOVA Type III and REML provide the

same estimates when data are balancedsame estimates when data are balanced Satterthwaite degrees of freedom with Satterthwaite degrees of freedom with

Geisbrecht-Burns approximationGeisbrecht-Burns approximation

Approach 1: Urquhart, et Approach 1: Urquhart, et al.al.

Variance components obtained from Variance components obtained from model model withoutwithout fixed trend slope fixed trend slope Construct Construct ΦΦ((θθ)=Var(Y) from variance )=Var(Y) from variance

componentscomponents Estimate Estimate ββ and SE( and SE(ββ) with GLS) with GLS This approach assumed the variance This approach assumed the variance

components were knowncomponents were known Did not address estimationDid not address estimation We use REMLWe use REML

Approach 2: Piepho and Approach 2: Piepho and Ogutu (2002)Ogutu (2002)

Extension of VanLeeuwen, et al (1996)Extension of VanLeeuwen, et al (1996) Random slope effect incorporatedRandom slope effect incorporated Trend and variance component estimation Trend and variance component estimation Trend testing conducted using a synthetic F Trend testing conducted using a synthetic F

test test Wald F-test not invariant to location shiftsWald F-test not invariant to location shifts

P&O relaxed assumption of P&O relaxed assumption of independence between random site independence between random site effect and random slope for invariant effect and random slope for invariant Wald F-testWald F-test , 0i i atCov a t

Approach 3: Kincaid, et Approach 3: Kincaid, et al. (2004)al. (2004)

Two models usedTwo models used Trend model omits RE for yearTrend model omits RE for year Variance components model omits fixed Variance components model omits fixed

linear trendlinear trend RE’s for site, year, interactionRE’s for site, year, interaction This paper focused on status This paper focused on status

estimationestimation Trend approach mentioned incidentallyTrend approach mentioned incidentally

Desirable propertiesDesirable properties

Trend testTrend test PowerfulPowerful Nominal test sizeNominal test size

Trend modelTrend model Ability to accurately estimate trend Ability to accurately estimate trend Nominal CI coverage for trendNominal CI coverage for trend Variance component estimationVariance component estimation

SIEN Lake Chemistry SIEN Lake Chemistry Pilot data: Seven lakes study & Western lakes studyPilot data: Seven lakes study & Western lakes study Three outcomes chosen for studyThree outcomes chosen for study

Ca: high random site Ca: high random site variabilityvariability Cl: high random slope Cl: high random slope variabilityvariability NO3: high year-to-year NO3: high year-to-year and and residual error variation residual error variation

Indicator 3: high year-Indicator 3: high year-to-year variationto-year variation Indicator 4: high Indicator 4: high residual error variationresidual error variation

Monte Carlo power Monte Carlo power simulationsimulation

Simulate population of lakes from estimated Simulate population of lakes from estimated fixed effects and variance components fixed effects and variance components obtained from the case study dataobtained from the case study data 1000 populations generated1000 populations generated 3 independent random samples selected from 3 independent random samples selected from

each populationeach population Generate known trendGenerate known trend

Annual decline of 1% or 4%Annual decline of 1% or 4% 10 years10 years

Impose revisit design Impose revisit design 1/3 of effort to 1/3 of effort to annual panelannual panel

Simulation power is proportion of times that Simulation power is proportion of times that null hypothesis is correctly rejected at the null hypothesis is correctly rejected at the αα = 0.10 level= 0.10 level

Test sizeTest size

2t

LargeVar.

Comp.Revisit design

s = 10 s = 60

Approach Approach

1 2 3 1 2 3

[1-0] 0.396 0.104 0.512 0.162 0.096 0.600

[(1-0),(1-3)] 0.228 0.110 0.400 0.104 0.096 0.514

[(1-0) ,(1-8)] 0.270 0.142 0.396 0.140 0.124 0.464

[1-0] 0.596 0.098 0.602 0.394 0.108 0.570

[(1-0) ,(1-3)] 0.360 0.094 0.408 0.214 0.084 0.428

[(1-0) ,(1-8)] 0.350 0.096 0.440 0.212 0.096 0.384

[1-0] 0.100 0.094 0.552 0.058 0.082 0.792

[(1-0) ,(1-3)] 0.094 0.110 0.556 0.078 0.126 0.808

[(1-0) ,(1-8)] 0.084 0.104 0.490 0.064 0.094 0.768

[1-0] 0.322 0.088 0.374 0.292 0.098 0.392

[(1-0) ,(1-3)] 0.210 0.094 0.302 0.174 0.114 0.288

[(1-0) ,(1-8)] 0.176 0.080 0.292 0.110 0.070 0.238

2b

2a

2e

Simulation resultsSimulation results

Power approximations are too high Power approximations are too high when test size exceeds nominal ratewhen test size exceeds nominal rate

Estimates of Estimates of ββ11 are generally unbiased are generally unbiased Bias of Bias of ββ1 1 most sensitive tomost sensitive to revisit design, revisit design,

not trend approachnot trend approach Observed that bias of SE(Observed that bias of SE(ββ11) was less ) was less

severe as revisit cycle length severe as revisit cycle length increasedincreased

Rel. Bias of SE(Rel. Bias of SE(ββ11): ): σσaa22

large, p =-1%large, p =-1%

Rel. Bias of SE(Rel. Bias of SE(ββ11): ): σσtt22

large, p= -1%large, p= -1%

Rel. Bias of SE(Rel. Bias of SE(ββ11): ): σσbb22

large, p= -1%large, p= -1%

Rel. Bias of SE(Rel. Bias of SE(ββ11): ): σσee22

large, p= -1%large, p= -1%

DiscussionDiscussion Approach 2 has most stable test sizeApproach 2 has most stable test size When When σσee

22 high, Approach 2 high, Approach 2 overestimates overestimates σσaa

22, , σσtt22, , σσbb

22, and SE(, and SE(ββ11)) Poor indicator for monitoringPoor indicator for monitoring

Simulation power is almost always Simulation power is almost always lower than power approximations lower than power approximations assuming large-sample theoryassuming large-sample theory

ConclusionsConclusions Trend test size should be assessed Trend test size should be assessed

when examining power to detect when examining power to detect trendtrend

Including a random slope effect that Including a random slope effect that is correlated with the random site is correlated with the random site effect in the mixed model approach effect in the mixed model approach provides nearly-nominal trend testsprovides nearly-nominal trend tests

Examining variance components is Examining variance components is useful for choosing monitoring useful for choosing monitoring indicators and revisit designsindicators and revisit designs

Ongoing workOngoing work

Determine if bias of variance Determine if bias of variance components estimates may be components estimates may be reducedreduced

Incorporate autocorrelation Incorporate autocorrelation estimationestimation

Examine relationship between revisit Examine relationship between revisit cycle length and SE(cycle length and SE(ββ11))

AcknowledgementsAcknowledgements

NPS Vital Signs Monitoring NPS Vital Signs Monitoring AgreementAgreement

Linda MutchLinda Mutch James Sickman, John Melack, and James Sickman, John Melack, and

Dave Clow Dave Clow Kirk SteinhorstKirk Steinhorst N. Scott Urquhart N. Scott Urquhart Tom KincaidTom Kincaid

Questions?Questions?