statistical inference for food websfaculty.washington.edu/gchiu/talks/bio08.pdf · 2009-10-04 ·...

Statistical Inference for Food WebsPart I: Bayesian Melding

Grace Chiu∗

andJosh Gould

∗ Department of Statistics & Actuarial Science

CMAR-Hobart Science Seminar, March 6, 2009 1

Outline

Overview: existing vs. statistical approaches forTrophic contextWhole-system context: ecological network analysis (ENA)

Present ENA techniquesENA statistical inference

statistical perspective of mass balanceBayesian melding

Example: Chesapeake Bay Mesohaline NetworkSummary and Conclusion

Outline

Present ENA techniques

ENA statistical inferencestatistical perspective of mass balanceBayesian melding

Outline

Example: Chesapeake Bay Mesohaline Network

Summary and Conclusion

Outline

Aspects of a Food Web

... that was only for trophicrelations ...

Aspects of a Food Web: (1) Trophic

Food web, trophically

structure of interdependence among speciespredator-prey links =⇒ feeding patternslinks can be weighted:e.g. predation frequency

Existing work for trophic analyses:examine interactions between providers (prey) andbenefactors (predators)(semi-)quantitative techniques for systematic extraction ofthe meaning of these interactionse.g. trophic hierarchy / compartments

Food web, trophicallystructure of interdependence among speciespredator-prey links =⇒ feeding patternslinks can be weighted:e.g. predation frequency

Existing work for trophic analyses:examine interactions between providers (prey) andbenefactors (predators)(semi-)quantitative techniques for systematic extraction ofthe meaning of these interactions

e.g. trophic hierarchy / compartments

and now, for whole-systemrelations...

Aspects of a Food Web: (2) Whole System

Ecological Network (whole-system food web)

trophic compartments and susbstance / energy throughputthis interdependence is subject to system balancenotion of balance based on thermodynamics

Existing Work: Ecological Network Analysis (ENA)deterministic biophysical theory in a balance modelquantity of exchange of substance / energy amongcompartmentsextract characteristics of these quantities

=⇒ describe interactions among compartments

Ecological Network (whole-system food web)trophic compartments and susbstance / energy throughputthis interdependence is subject to system balancenotion of balance based on thermodynamics

Disclaimer

Images of food webs were generated by aGoogle search.

Trophic analysis and ENA

Main Goal:to understand / predict (e.g. over time) within-web interactionsbased on the quantities associated with the edges (arrows)between pairs of species / compartments

ISSUES:randomness of quantities ignoredno formal statistical inference of interaction patterns orpredictionsfor ENA,

randomness =⇒ observed quantities don’t balancesome quantities are unobservable from field —computer algorithms generate missing quantities tominimize imbalance (e.g. EcoSim / EcoPath)

=⇒ further complicating any inference attempts!

A statistician’s ideas...

Alternative Trophic Analysis:take a completely quantitative regression approach:

accounts for randomness of data!accounts for substance exchange! and other variables

proper inference possible from regression modellingincludes prediction inference —for pairwise links and compartments AND over time

simple scatterplots to identify and interpret compartments

that’ll be Statistical InferencePart II

Statistical Inference Part I

ENA Statistical Inference:can overcome empirical imbalance by incorporatingrandomness through Bayesian Meldingcan fill in missing quantities by prediction inference withinBayesian frameworkcan be extended to temporal model without explicitcalibration (as opposed to, e.g. morphing multiple staticanalyses into a single dynamics model)

ENA Statistical Inference:can overcome empirical imbalance by incorporatingrandomness through Bayesian Meldingcan fill in missing quantities by prediction inference withinBayesian frameworkcan be extended to temporal model without explicitcalibration (as opposed to, e.g. morphing multiple staticanalysis into a single dynamics model)

Conventional ENA

Simple e.g. from Ulanowicz (Comput. Biol. & Chem., 2004):

Fix a transfer medium, e.g. nitrogen or heat. Let

i , j = compartment labelTij = rate of transfer of medium from i to jXi = rate of exogenous input of medium to iEi = rate of external transfer of medium from iRi = rate of dissipation of medium from i

Balance Equation:

total inflow rate = total outflow rate=⇒ Xi +

Tj i =∑

Ti j + Ei + Ri

Ideally, do this over all n compartments and K media.

Conventional ENA

Balance Equation:

Tj i =∑

Ti j + Ei + Ri

Conventional ENA

Balance Equation:

Tj i =∑

Ti j + Ei + Ri

Conventional ENA: Ideally

System of n × K equations:

X (1)1 + T (1)

+1 = T (1)1+ + E (1)

1 + R(1)1

...X (1)

n + T (1)+n = T (1)

n+ + E (1)n + R(1)

X (2)1 + T (2)

+1 = T (2)1+ + E (1)

1 + R(2)1

...X (2)

n + T (2)+n = T (2)

n+ + E (1)n + R(2)

...X (K )

1 + T (K )+1 = T (K )

1+ + E (K )1 + R(K )

X (K )n + T (K )

+n = T (K )n+ + E (K )

n + R(K )n

Randomness =⇒ field data never satisfy equality

Worse yet, only some quantities are observable from field ...

Conventional ENA

: Ideally

X (1)1 + T (1)

+1 = T (1)1+ + E (1)

1 + R(1)1

...X (1)

n + T (1)+n = T (1)

n+ + E (1)n + R(1)

X (2)1 + T (2)

+1 = T (2)1+ + E (1)

1 + R(2)1

...X (2)

n + T (2)+n = T (2)

n+ + E (1)n + R(2)

...X (K )

1 + T (K )+1 = T (K )

1+ + E (K )1 + R(K )

X (K )n + T (K )

+n = T (K )n+ + E (K )

n + R(K )n

Conventional ENA

: Ideally

X (1)1 + T (1)

+1 = T (1)1+ + E (1)

1 + R(1)1

...X (1)

n + T (1)+n = T (1)

n+ + E (1)n + R(1)

X (2)1 + T (2)

+1 = T (2)1+ + E (1)

1 + R(2)1

...X (2)

n + T (2)+n = T (2)

n+ + E (1)n + R(2)

...X (K )

1 + T (K )+1 = T (K )

1+ + E (K )1 + R(K )

X (K )n + T (K )

+n = T (K )n+ + E (K )

n + R(K )n

Conventional ENA

: Ideally

X (1)1 + T (1)

+1 = T (1)1+ + E (1)

1 + R(1)1

...X (1)

n + T (1)+n = T (1)

n+ + E (1)n + R(1)

X (2)1 + T (2)

+1 = T (2)1+ + E (1)

1 + R(2)1

...X (2)

n + T (2)+n = T (2)

n+ + E (1)n + R(2)

...X (K )

1 + T (K )+1 = T (K )

1+ + E (K )1 + R(K )

X (K )n + T (K )

+n = T (K )n+ + E (K )

n + R(K )n

Conventional ENA

Example for n=4, K =1Observe: Xi , Ei , Ri for all i ; Tij for all (i , j) except i=3Unknown: T31, T32, T34 =⇒ T3+

X1 + T21 + T31 + T41 = T12 + T13 + T14 + E1 + R1

X2 + T12 + T32 + T42 = T21 + T23 + T24 + E2 + R2

X3 + T13 + T23 + T43 = T31 + T32 + T34 + E3 + R3

X4 + T14 + T24 + T34 = T41 + T42 + T43 + E4 + R4

no balance =⇒ no theoretical solution for T3+

Conventional ENA

Example for n=4, K =1Observe: Xi , Ei , Ri for all i ; Tij for all (i , j) except i=3Unknown: T31, T32, T34 =⇒ T3+

X1 + T21 + T31 + T41 = T12 + T13 + T14 + E1 + R1

X2 + T12 + T32 + T42 = T21 + T23 + T24 + E2 + R2

X3 + T13 + T23 + T43 = T31 + T32 + T34 + E3 + R3

X4 + T14 + T24 + T34 = T41 + T42 + T43 + E4 + R4

no balance =⇒ no theoretical solution for T3+

Conventional ENA

Remedy: deduce T3j ’s from observable auxiliary quantities

e.g. theoretical relationship amongTijcompartment productionbiomass...

f (Pij , Bij , . . .) = Tij

even if a deduced Tij is free of uncertainty ...System of equations fails regardless ... what then?

Conventional ENA

f (Pij , Bij , . . .) = Tij

Conventional ENA

f (Pij , Bij , . . .) = Tij

even if a deduced Tij is free of uncertainty ...

System of equations fails regardless ... what then?

Conventional ENA

f (Pij , Bij , . . .) = Tij

Conventional ENA

Computer software to the rescue!

tinker with quantities subject to certain criteriauntil equality (almost) holdscriteria built into software — can be mysterious to userrestrict tinkering to deduced quantities only, if possible

(Program) Input: observed and deduced quantities(Program) Output: balanced quantities

Conventional ENA

In light ofim

(possible)

balancetheory-based deductioncoerced balance ...

Million $ question:How confident are we in the numbers??

Conventional ENA

In light ofim(possible)balancetheory-based deductioncoerced balance ...

Conventional ENA

In light ofim(possible)balancetheory-based deductioncoerced balance ...

Conventional ENA: confidence

Existing attempts: Sensitivity analysesperturb program inputmonitor behavior of program output

but ...

How to make inference for underlying network structure?

Conventional ENA: confidence

Existing attempts: Sensitivity analysesperturb program inputmonitor behavior of program output

but ...

How to make inference for underlying network structure?

Perspectives of Mass Balance

Physics: in = outStatistics: E( in ) = E( out )

Simplest example

Let Wi = Xi + T+i w/ mean µW

Ui = Ti+ + Ei w/ mean µU

Ri w/ mean µR

=⇒ balance model: µW = µU + µR

single balance equation of unobservable quantitiesestimation and confidence statements via statisticalinference

Physics: in = out

Statistics: E( in ) = E( out )

Simplest example

Ri w/ mean µR

Simplest example

Ri w/ mean µR

Simplest example

Ri w/ mean µR

Simplest example

Ri w/ mean µR

Simplest example

Ri w/ mean µR

Bayesian Melding for ENA Inference

Elements of deterministic modelingdeterministic model M( )

model input θ

= E(observables)

model output φ

= E(unobservables)

M : θ 7−→ φ or φ := M(θ)

Rationale for choice of input/output:to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)

model input θ

= E(observables)

model output φ

= E(unobservables)

M : θ 7−→ φ or φ := M(θ)

model input θ

= E(observables)

model output φ

= E(unobservables)

M : θ 7−→ φ or φ := M(θ)

model input θ = E(observables)model output φ = E(unobservables)

M : θ 7−→ φ or φ := M(θ)

Rationale for choice of input/output:

to make statistical inference, need assumptions aboutstatistical behavior for quantitiesexperience with observed quantities can be basis of suchassumptionsthen statistical behavior of E(unobservables) isdefined by M through those of E(observables)

model input θ = E(observables)model output φ = E(unobservables)

M : θ 7−→ φ or φ := M(θ)

Among Wi , Ui , Ri , suppose Ri is unobservable

=⇒ rewrite balance model as

µR = µW − µU

or φ = M(θ)

where φ = µR model output

θ = (µW , µU)′ model input

M(θ) = (1,−1)θ the model M : θ → φ

Among Wi , Ui , Ri , suppose Ri is unobservable=⇒ rewrite balance model as

µR = µW − µU

or φ = M(θ)

Among Wi , Ui , Ri , suppose Ri is unobservable=⇒ rewrite balance model as

µR = µW − µU

or φ = M(θ)

Bayesian Inference

specify prior and likelihood for φ,θ

obtain posterior for φ,θ =⇒ posterior inference

Bayesian Melding (due to Poole & Raftery, 2000)two priors for φ:

specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M

combine both priors =⇒ melded prior for φ, q̃(φ)

crank melded prior q̃(φ) through M−1

=⇒ melded prior for θ, p̃(θ)

posterior inference based on p̃ and M

there are tricks to overcome non-invertibility of M

Bayesian Inferencespecify prior and likelihood for φ,θ

Bayesian Melding (due to Poole & Raftery, 2000)

two priors for φ:specified prior, h(φ)induced prior, h∗(φ), from cranking prior for θ through M

g(θ) h∗(φ)input prior induced output prior

θ(1) → M(θ(1)) = φ(1)

...θ(m) → M(θ(m)) = φ(m)

Bayesian Melding for ENA

obtain posterior for φ,θ =⇒ posterior inferencepredictions for “missing” T3i ’s via posterior predictive

Bayesian Melding for ENA

Melded prior for φ is

q̃(φ) ∝ h∗(φ)γ h(φ)1−γ

for some γ ∈ (0,1).

What’s γ?arbitrary in principlecan be specified to reflect expert opinions on relativereliability between h and h∗ (or M)

Melded prior for φ is

q̃(φ) ∝ h∗(φ)γ h(φ)1−γ

for some γ ∈ (0,1).

What’s γ?arbitrary in principlecan be specified to reflect expert opinions on relativereliability between h and h∗ (or M)

specified prior, f (φ)induced prior, f ∗(φ), from cranking prior for θ through M

there are tricks to overcome non-invertibility of MCMAR-Hobart Science Seminar, March 6, 2009 27

posterior for θ: πθ(θ) ∝ p̃(θ) Lθ(θ) Lφ

(M(θ)

posterior for φ: M : πθ(θ) → πφ(φ)

Hallelujah!

(M(θ)

)posterior for φ: M : πθ(θ) → πφ(φ)

Hallelujah!

(M(θ)

)posterior for φ: M : πθ(θ) → πφ(φ)

Hallelujah!

Application: Chesapeake Bay Summer Network

based on

Baird & Ulanowicz (1989), Ecological Monographs 59, 329–364

http://www.cbl.umces.edu/˜ulan/ntwk/datall.zip

transfer medium: carbon (g/m2/summer)i=1,. . . , 36 compartments

Elements of Bayesian inferenceLikelihood:

Wi , Ui , Ri |θ, φ ∼ independent exponentials

specified prior:

(θ, φ)′ ∼ trivariate log-normal

=⇒ Wi , Ui , Ri are marginally dependent(as would be necessary due to theoretical balance)

specified prior:

NOTE: Instead of painstaking extraction of (unbalanced) datafrom the flow diagram, we adopted the online data (alreadybalanced) — would ideally use former.

0 100 300 500

0 50 100 150 200 250

0 20 40 60 80 100

φφ0 10 20 30 40 50 60

Specified prior ( —— ) and Melded Posterior ( —— ):

Estimates and 95% Confidence intervals:

θ1 = µW θ2 = µU φ = µRMelding

Posterior Mean 89.03 64.11 24.92HPD interval (72.69, 107.26) (47.99, 80.84) (17.57, 33.23)

ClassicalCLT interval (33.24, 146.06) (20.70, 110.56) (4.24, 43.80)

Classical intervals are MUCH WIDER!

Estimates and 95% Confidence intervals:

θ1 = µW θ2 = µU φ = µRMelding

Posterior Mean 89.03 64.11 24.92HPD interval (72.69, 107.26) (47.99, 80.84) (17.57, 33.23)

ClassicalCLT interval (33.24, 146.06) (20.70, 110.56) (4.24, 43.80)

Classical intervals are MUCH WIDER!

Dependence among θ1, θ2, φ:

60 70 80 90 100 110 120

θθ 2

60 70 80 90 100 110 120

40 50 60 70 80 90 100

High dependence is presumed among all 3 due totheoretical balance.Inference indicates dependence is only high betweenµW and µU =⇒ new insight!Note: inference on dependence structure NOT possiblewith classical inference (which treats µ’s as constants).

Dependence among θ1, θ2, φ:

60 70 80 90 100 110 120

θθ 2

60 70 80 90 100 110 120

40 50 60 70 80 90 100

High dependence is presumed among all 3 due totheoretical balance.Inference indicates dependence is only high betweenµW and µU =⇒ new insight!Note: inference on dependence structure NOT possiblewith classical inference (which treats µ’s as constants).

Summary

Physics Statistics(1) each quantity (variable) • no random variable needs

must satisfy within- to satisfy balancecompartment balance =⇒ no compartment needs=⇒ collapse compart- to satisfy balance§

ments to allow balance • random variables have=⇒ wrong biology expectations that satisfy

within-system balance=⇒ the more compartments(i.e. data) the better!

§ can be a need as long as replicated observations areavailable per compartment

Summary

Physics Statistics(1) each quantity (variable) • no random variable needs

must satisfy within- to satisfy balancecompartment balance =⇒ no compartment needs=⇒ collapse compart- to satisfy balance§

ments to allow balance • random variables have=⇒ wrong biology expectations that satisfy

within-system balance=⇒ the more compartments(i.e. data) the better!

§ can be a need as long as replicated observations areavailable per compartment

Summary

Physics Statistics(2) unobservable variables statistical prediction

are deduced from (inferential) possibleauxiliary theory in certain scenarios

e.g. variable observed forsome compartments

e.g. deduce unobservedthrough formal regression(both in progress)

(3) no formal inference / possible forconfidence statements (a) any quantity infor any quantity system-balance equation

(b) unobservable variablein some cases (see (2))

(4) multiple media on perceivably straight-same system hard forwardto impossible (in progress)

Conclusion

So why statistical inference for food webs?statistical models are more honest:

properly acknowledge uncertaintystatistical inference-based techniques are tractableproper prediction inference is possible

(straightforward within Bayesian framework)ENA with Bayesian melding additionally

overcomes empirical imbalance under theoretical balancesoundly integrates statistical practice with physical theory

— often preferred by scientists over purely empirically drivenapproaches

Conclusion

So why statistical inference for food webs?

statistical models are more honest:properly acknowledge uncertainty

statistical inference-based techniques are tractableproper prediction inference is possible

Conclusion

(straightforward within Bayesian framework)

ENA with Bayesian melding additionallyovercomes empirical imbalance under theoretical balancesoundly integrates statistical practice with physical theory

Conclusion

Thank you!

This presentation is available from:http://www.stats.uwaterloo.ca/˜gchiu/Talks/csiro-hobart09.pdf

Articles:Chiu & Gould (submitted).Chiu & Gould (2008),U of Waterloo Working Paper #2008-07.

www.stats.uwaterloo.ca/stats navigation/techreports/08WorkingPapers/08-07.pdf

statistical inference for food websfaculty.washington.edu/gchiu/talks/bio08.pdf · 2009-10-04 ·...

Documents

19 inference

observation & inference

tutorial on approximate bayesian computation · inference...

observation of optimal gecko’s adhesion on nanorough...

introduction to statistical...

variational inference and mean field · variational...

asymptotic theory for bent-cable regression | the basic...

low precision inference on gpu - nvidia · 3 inference •...

unit 5: inference for categorical variables lecture 1:...

variational inference - marc deisenrothvariational inference...

4 : exact inference: variable elimination 1 probabilistic...

automatisierte logik und programmierung ii · gui evaluator...

induction (probable inference) : inference moving from...

1 probabilistic inference and...

name definitions: evidence inference period inference vs

direct inference and inverse inference teddy … inference...

graphical inference

stats 200: introduction to statistical inference 200:...

m ethods of inference hasan zafari. m ethods of inference...

extended bayesian information criteria for model selection...