Bayesian Model Robust and Model Discrimination Designs
William Li
Operations and Management Science Department
University of Minnesota
(joint work with Chris Nachtsheim and Vincent Agtobo)
Outline
• Introduction
• Bayesian model-robust designs
• Bayesian model discrimination designs
Part I: Introduction• Main objective: find designs that are efficient over a
class of models– Model estimation: Are all models estimable?– Model discrimination: Can estimable models be
discriminated?
• Brief literature review– Early work: Lauter (1974), Srivastava (1975), Cook and
Nachtsheim (1982)– Cheng, Steinberg, and Sun (1999)– Li and Nachtsheim (2000)– Jones, Li, Nachtsheim, Ye (2006)
More literature review
• Bingham and Chipman (2002): Bayesian Hellinger distance
• Miller and Sitter (2005): the probability that the true model is identified
• Montgomery et al. (2005): application of new tools in model-robust designs
• Loeppky, Sitter, and Tang (2005): projection model space
• Jones, Li, Nachtsheim, Ye (2006): model-robust supersaturated designs
A general framework
• Li (2006): a review on model-robust designs• Framework: three main elements
– Model space: F={f1, f2, …, fu}
– Criterion (e.g., EC, D-, EPD)
– Candidates designs (e.g., orthogonal designs)
• Objective (rephrase): select an optimal design from candidate designs, such that it is optimal over all models in F, with respect to a criterion
Model spaces
• Srivastava (1975): search designs– F = {all effects of type (ii) + up to g effects of type
(iii)}
• Sun (1993), Li and Nachtsheim (2000)– Fg = {all main effects + up to g 2f interactions}
• Supersaturated designs– F = {any g out of m main effects}
• Loeppky, Sitter, and Tang (2005)– Fg = {g out of m main effects + all 2f interactions}
Criteria and candidate designs
• Criteria– Bayesian model-robust criterion (related to EC
and IC of LN)– Bayesian model discrimination criteria (related
to EPD of Jones et al.)
• Candidate designs– Orthogonal designs– Optimal designs
Bayesian optimal designs
• Main elements– Prior distribution: p()– Distribution of data: p(y | )– Utility function: U(d, y)– Design space
Selected literature
• “Bayesian Experimental Design: A Review” – Chaloner and Verdinelli (1995)
• DuMouchel and Jones (1994): Bayesian D-optimal designs
• Jones, Lin, and Nachtsheim (2006): Bayesian supersaturated designs
Part II: Bayesian model-robust designs
• Focus: estimability of designs– Estimation capacity (EC): percentage of estimable
models• Model-robust designs: EC=100%
– Information capacity (IC): average D-criterion value over all models
• Model space– LN (2000): main effects + g 2fi’s– Loeppky et al. (2006): g main effects + all 2fi’s among g factors
Bayesian criterion
Bayesian criterion for model-robust designs
Bayesian model-robust design
• Prior probabilities– Uniform prior– Hierarchical prior
• Chipman, Hamada, and Wu (1997)
• Bayesian model-robust (BMR) criterion
• Bayesian model-robust design (BMRD)
Design evaluations
• Evaluating existing orthogonal designs – 12-, 16-, and 20-run designs (Sun, Li, and Ye,
2002)– Two model spaces– Compute BMR values and rank designs– Compare BMR ranks with generalized WLP
ranks • Generalized WLP: Deng and Tang (1999)• Ranks for GWLP: given in Li, Lin, and Ye (2003)
Rank comparison plot (for 16*7 designs)
Design constructions
• Optimal designs– Balanced (equal # of +’s and –’s)
• CP algorithm of Li and Wu (1997)
– General (unbalanced) optimal designs• Coordinate-exchange algorithm of Meyer and
Nachtsheim (1995)
Part III: Bayesian model discrimination designs
• Issues beyond model estimation– How well can estimable models be distinguished from
each other?
– If true model is known, is it fully aliased with other models through the design?
Criteria
• Atkinson and Fedorov (1975)
• EPD (expected prediction differnce) criterion (Jones et al. 2006)
Expected non-centrality parameter (ENCP) criterion
Bayesian EPD criterion
Design results
• Evaluating orthogonal designs– A comprehensive study of designs– Candidate designs: 12-, 16-, 20-run designs – Model space: both LN and the projected space of
Loeppky et al. (2006)– Criteria: all model discrimination criteria (Bayesian and
non-Bayesian)
• Constructing optimal designs– CP: balanced– Coordinate-exchange: general (unbalanced)
An example
mEPD aEPD mAF aAF mENCP aENCP
----------------------------------------
(n=16, m=5, g=2)
1-3 EC < 100%
4 0.125 0.205 0.347 0.567 16.000 26.182
5 0.063 0.187 0.173 0.520 4.000 22.109
6 EC < 100%
7 0.000 0.184 -9.999 -9.999 0.000 19.806
8 0.094 0.198 0.173 0.495 4.000 19.673
9 EC < 100%
10 0.094 0.198 -9.999 -9.999 8.000 17.673
11 0.058 0.177 -9.999 -9.999 4.667 15.702
THANK YOU!
More information: www.csom.umn.edu/~wli
