1

Myopic Policies for Budgeted Optimization

with Constrained Experiments

Javad Azimi, Xiaoli Fern, Alan Fern

Oregon State University AAAI, July 2010

2

Motivation: Electricity Production in a Microbial Fuel Cell (MFC)

AnodeCathode

bact

eria

Oxidation products

(CO2)

Fuel (organic matter)

e-

e-

O2

H2OH+

This is how an MFC works

SEM image of bacteria sp. on Ni nanoparticle enhanced carbon fibers.

Nano-structure of anode significantly impact the electricity production.

We should optimize anode nano-structure to maximize power.

3

Experiment Selection• Experiments are costly and there is a fixed budget.• How to select the best sequence of experiments.

Current Experiments Scientist selects Experiment

Run Experiment

Bayesian Optimization (BO)• Since Running experiment is very expensive we use BO.

• Select one experiment to run at a time based on results of previous experiments.Current Experiments Gaussian Process Surface Select Single Experiment

Run Experiment 4

5

Bayesian Optimization (BO)• BO assumes that we can ask for specific experiment.• This is unreasonable assumption in many applications.

– In Fuel Cell it takes many trials to create a nano-structure with specific requested properties.

– Costly to fulfill.Space of Experiments

Average Circularity

Ave

rage

d A

rea

6

Constrained Experiments• It is less costly to fulfill a request that specifies ranges for the nanostructure

properties

• E.g. run an experiment with Averaged Area in range r1 and Average Circularity in range r2

• We will call such requests “constrained experiments”Space of Experiments

Average Circularity

Ave

rage

d A

rea

Constrained Experiment 1• large ranges • low cost• high uncertainty about which experiment will be run

Constrained Experiment 2• small ranges• high cost• low uncertainty about which experiment will be run

7

BO for Constrained Experiment

• Given a fixed budget, select the best constrained experiments.

Run Experiment

Current Experiments Gaussian Process Surface Select Single ExperimentSelect Constrained Experiment

8

Constrained Experiment• We generalized BO heuristics to constrained

experiments.

• Two challenges:– How to compute heuristics for constrained experiment?– How to take experimental cost into account?(which has

been ignored by most of the approaches in BO).

9

Standard BO Heuristics• Standard heuristics are statistics of the posterior p(y|x,D) where D is our

current observation.

• Maximum Upper bound Interval (MUI)– Select point with highest 95% upper confidence bound– Purely explorative approach.

• Maximum Probability of Improvement (MPI)– It computes the probability that the output is more than (1+m) times of the best current

observation , m>0. – Explorative and Exploitative.

• Maximum Expected of Improvement (MEI)– Similar to MPI but parameter free– It simply computes the expected amount of improvement after sampling at any point.

10

Generalizing BO for Constrained Experiment

• Having the posterior distribution of p(y|x,D) and px(.|D) we can calculate the posterior of the output of each constrained experiment which has a closed form solution.

• Therefore we can compute standard BO heuristics for constrained experiments.– There are closed form solution for these heuristics.

Input spaceDiscretization

Level

11

Budgeted Constrained Experiments• We are limited with Budget B.• Unfortunately heuristics will typically select the smallest and most

costly constrained experiments which is not a good use of budget.

• How can we consider the cost of each constrained experiment in making the decision?– Cost Normalized Policy (CN)– Constraint Minimum Cost Policy(CMC)

-Low uncertainty

-High uncertainty

-Better heuristic value

-Lower heuristic value

-Expensive -Cheap

12

Cost Normalized Policy• It selects the constrained experiment achieving

the highest expected improvement per unit cost.

• We report this approach for MEI policy only.

13

Constraint Minimum Cost Policy (CMC)• Motivation:

1. Approximately maximizes the heuristic value.2. Has expected improvement at least as great as spending

the same amount of budget on random experiments.• Example:

Very expensive: 10 random experiments likely to be better

Selected Constrained experiment

Poor heuristic value: not select due to 1st

condition

Cost=4 random Cost=10 random Cost=5 random

14

Experimental Results(Setup)

• Gaussian process is used as our model with squared exponential kernel.

• Cost function is defined as:

– There is a constant cost for running any constrained experiment plus an additional cost depending on the size of the experiment.

– The value of slope dictates how fast the cost increases as the size of a constrained experiment decreases.

Space of Experiments

r1

r2

15

Experimental Results(Real Applications)

• Two Real data sets:– Fuel Cell:• Fuel Cell electricity generation

– Hydrogene:• Biosolar hydrogen production

16

Experimental Results(Benchmarks Functions)

• 3 popular benchmarks used in BO literature.

Rosenbrock Discontinuous Cosines

17

Overall Performance• The normalized regret of each framework is shown which is calculated

as y*- ymax over Random performance for budget 15 where y* is the highest possible output.

Average regret of each approach over all frameworks.

Different Budget(1)

18

Random

Cosines

Fuel CellReal

Rosenbrock

CMC-MUI

Different Budget(2)

19

CN-MEI

Cosines

Fuel CellReal

Rosenbrock

Different Budget(3)

20

CMC-MPI(0.2)

Cosines

Fuel CellReal

Rosenbrock

21

Different Budget(4)

CMC-MEICosines

Fuel CellReal

Rosenbrock

22

Conclusion• We introduced a new constrained experiment

framework which asks for hyper-rectangle rather than exact point.

• We extended model-free BO heuristics to our frame work.

• We introduced two approaches to optimize our budgeted framework.

• CMC-MEI is working better than other approaches.

23

Thanks for attendance

Question?