mcmc software options - biostatisticsbrad/7440/slide_hatfield_mcmc_software.pdfmcmc software options...
TRANSCRIPT
MCMC Software Options
I BUGS and BUGS-esque: WinBUGS, OpenBUGS, and JAGS
I R Calling Functions for BUGS: BRugs, R2WinBUGS, rbugs,rjags, R2jags, and runjags
I R-based: mcmc
I Python-based: PyMC
All of these can produce so-called ”coda” files (i.e., raw MCMCchain output) that may be processed with coda package, amongothers. Saving model output in this format is the most portable.
Toy Model to Illustrate Differences
We will fit the following toy model as an example in each of thesoftware packages described.
yi ∼ N(β0 + β1xi , σ2y )
β0 ∼ N(µ1, σ21)
β1 ∼ N(µ2, σ22)
where the hyperparameters (σ2y , σ21, σ
22, µ1, µ2)′ are fixed at
(2, 10, 10, 5, 1)′, i = 1, . . . , 100, and the xi are randomly generatedfrom N(0, 1).
Generating the Data
For purposes of the R code snippets below, assume we have runthe following:
N <- 100
set.seed(221983)
x <- rnorm(N,0,1)
beta <- c(4,2)
y <- rnorm(N,beta[1]+beta[2]*x,sqrt(2))
bprec <- c(1/10,1/10)
mu <- c(5,1)
prec <- 1/2
WinBUGS
Dave Lunn & Nicky Best, Imperial College Londonhttp://www.mrc-bsu.cam.ac.uk/bugs/winbugs/
I Stand-alone GUI software, written in Component Pascal
I Runs natively in Windows and in Linux/Mac via WINE
I Includes spatial models (GeoBUGS)
I “Point-and-click” interface, but can call from R
WinBUGS model code
model{
for (i in 1:N){
y[i] ~ dnorm(meen[i],prec)
meen[i] <- beta[1] + beta[2]*x[i]
}
beta[1] ~ dnorm(mu[1],bprec[1])
beta[2] ~ dnorm(mu[2],bprec[2])
}
# Data
list(list(y=c(-4.79537E-01, 4.91775E+00, ...),
x=c(-1.21613E+00, 6.44501E-01, 3.61216E-01, ...),
mu=c(5,1),prec=.5,bprec=c(.1,.1),N=100)
# Inits
list(beta=c(0,0))
list(beta=c(1,1))
WinBUGS
OpenBUGS
Andrew Thomas, University of Helsinkihttp://www.openbugs.info/w/
I Open-source successor to WinBUGS, still Component Pascal
I Runs in Windows, Mac (via WINE), and Linux (no GUI)
I Minor differences vs WinBUGS model specification
I No spatial models
OpenBUGS model code
model{
for (i in 1:N){
y[i] ~ dnorm(meen[i],prec)
meen[i] <- beta[1] + beta[2]*x[i]
}
beta[1] ~ dnorm(mu[1],bprec[1])
beta[2] ~ dnorm(mu[2],bprec[2])
}
(Note: identical to WinBUGS for this model.)
OpenBUGS
JAGS
Martyn Plummer, International Agency for Research on Cancerhttp://calvin.iarc.fr/~martyn/software/jags/
I Open-source C++ software with BUGS-type modelspecification
I Runs natively on many OSs (Win, Mac, various Linux flavors)
I Distinguishes between censoring and truncation (unlikeWinBUGS)
JAGS
(Model code is the same; data and inits format slightly different.)
y <- c(3.06582, 1.26428, 0.54347, ...)
x <- c(-0.03570, -1.27062, -0.99170, ...)
mu <- c(5, 1)
prec <- 0.5
bprec <- c(0.1, 0.1)
N <- 100
JAGS
model in "JAGS_model.txt"
data in "JAGS_data.txt"
compile, nchains(2)
parameters in "JAGS_inits.txt"
initialize
monitor beta
update 10000
coda *
JAGS
rjags
library(rjags)
mymodel <- jags.model(file="JAGS_model.txt",
data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),
inits=list(list(beta=c(0,0)),list(beta=c(1,1))),n.chains=2)
mysamples <- jags.samples(mymodel,
variable.names=’beta’,n.iter=10000)
summary(mysamples$beta,quantile,
prob=c(.025,.5,.975))
rjags
R2WinBUGS
Andrew Gelman, Columbiahttp://www.stat.columbia.edu/~gelman/bugsR/
I Calls WinBUGS or OpenBUGS from within R
I (Using OpenBUGS option requires BRugs)
I Includes ability to run WinBUGS via WINE
R2WinBUGS
fit <- bugs(
data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),
inits=list(list(beta=c(0,0)),list(beta=c(1,1))),
parameters.to.save=’beta’,model.file="WB_model.txt",
n.chains=2,n.iter=10000,n.burnin=0,program="WinBUGS")
plot(fit)
R2WinBUGS
WBDev
Dave Lunn, Cambridgehhttp://www.winbugs-development.org.uk/
I Allows users to extend WinBUGS using new functions anddistributions
I Works within BlackBox Component Builder 1.5 (Windows)http://www.oberon.ch/blackbox.html
WBDev
MODULE WBDevMyNorm;
...
PROCEDURE LogFullLikelihood
(node: WBDevUnivariate.Node; OUT value: REAL);
VAR
x, meen, prec: REAL;
BEGIN
x := node.value;
meen := node.arguments[location][0].Value();
prec := node.arguments[inverseScale][0].Value();
value := 0.5 * (Math.Ln(prec) -
prec * Math.IntPower(x - meen, 2) - log2Pi);
value := value - Math.Ln(1 -
WBDevSpecfunc.Phi(-Math.Sqrt(prec) * meen));
END LogFullLikelihood;
...
WBDev
model{
for (i in 1:N){
y[i] ~ MyNorm(meen[i],prec)
meen[i] <- beta[1] + beta[2]*x[i]
}
beta[1] ~ dnorm(mu[1],bprec[1])
beta[2] ~ dnorm(mu[2],bprec[2])
}
WBDev
BRugs
http://www.openbugs.info/w/UserContributedCode
http://www.biostat.umn.edu/~brad/software/BRugs/
I (Silently) calls OpenBUGS from within R
I No longer distributed on CRAN, bugg on Windows 7 (?)
I Includes handy data formatting function bugs.data
BRugs
myfit <- BRugsFit(
modelFile="WB_model.txt",
data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),
inits=list(list(beta=c(0,0)),list(beta=c(1,1))),
nIter=10000,nBurnin=0,
numChains=2,parametersToSave=’beta’)
myfit$Stats
samplesHistory(’beta’)
samplesDensity(’beta’)
BRugs
rbugs
Jun Yan & Marco Prates, UConn
I Calls either Win/OpenBUGS from R
I Possibly better for Mac/Linux using WINE
rbugs
library(rbugs)
rbugsfit <- rbugs(
data=list(y=y,x=x,mu=mu,prec=prec,bprec=bprec,N=N),
inits=list(list(beta=c(0,0)),list(beta=c(1,1))),
paramSet=’beta’, model=’OB_model.txt’,
n.chains=2,n.iter=10000,n.burnin=0,
bugs="C:/Program Files (x86)/OpenBUGS/OpenBUGS321/OpenBUGS.exe",
OpenBugs=T,bugsWorkingDir=getwd())
rbugs
mcmc
Charlie Geyer, University of Minnesotahttp://www.stat.umn.edu/geyer/mcmc/
I Simple random-walk Metropolis algorithm
I User supplies a function that evaluates the log unnormalizedtarget density
I A few control parameters
mcmc
library(mcmc)
myll <- function(params,data){
meen <- params[1] + params[2]*data$x
ll <- sum(dnorm(data$y,meen,data$sigy,log=T))+
sum(dnorm(params,data$mu,data$sigb,log=T))
return(ll)
}
metropfit <- metrop(myll,initial=c(0,0),
nbatch=10000,blen=1,scale=.2,
data=list(x=x,y=y,mu=mu,sigy=sqrt(1/prec),
sigb=sqrt(1/bprec)))
metropfit$accept
mcmc
PyMC
Anand Patil (Univ Oxford), David Huard (McGill), and ChrisFonnesbeck (Vanderbilt)http://code.google.com/p/pymc/
I Wrapper Python code that calls updating routines written inC and Fortran
I Open-source, extensible, fast