1 hybrid methods for solving large-scale parameter estimation problems carlos a. quintero 1 miguel...
Post on 27-Dec-2015
221 Views
Preview:
TRANSCRIPT
1
Hybrid methods for solving large-scale
parameter estimation problems Carlos A. Quintero1 Miguel Argáez1 Hector Klie2
Leticia Velázquez1 Mary Wheeler2
1 Department of Mathematical Sciences
University of Texas at El Paso
13th GAMM - IMACS International Symposium on Scientific Computing, Computer Arithmetic, and Verified
Numerical Computations SCAN'2008
El Paso, Texas, USA
October 2 , 2008
2 ICES-The Center for Subsurface Modeling The University of Texas at Austin
2
Outline
Introduction Problem Formulation
Parameter Estimation Optimization Framework
Global Search Surrogate Model Local Search
Numerical Results Conclusion and Future Work
3
Goal
We present a hybrid optimization approach for solving global optimization problems, in particular automated parameter estimation models.
The hybrid approach is based on the coupling of the Simultaneous Perturbation Stochastic Approximation (SPSA) and a Newton-Krylov Interior-Point method (NKIP) via a surrogate model.
We implemented the hybrid approach on several test cases.
4
Problem Formulation
minimize : R R (1)nf x f
where the global solution x* is such that
We are interested in problem (1) that have many local minima.
We consider the global optimization problem in the form:
nxxfxf R allfor *
5
Types of local minima
x
global minimum
local minima
6
Hybrid Optimization Framework
• Global Method: SPSA
• Surrogate Model
• Local Method: NKIP
7
Global Method: SPSA
Stochastic steepest descent direction algorithm • (James Spall, 1998)
•Advantage• SPSA gives region(s) where the function value is low, and this allows to conjecture in which region(s) is the global solution. • Uses only two objective function evaluations at each iteration to obtain the update of the parameters.
Disadvantages•Slow method•Do not take into account equality/inequality constraints
8
Global Search: SPSA
SPSA performs random simultaneous perturbations of all model parameters to generate a descent direction at each iteration
This process may be performed by starting with different initial guesses (multistart).
Multistart increases the chances for finding a global solution, and yields to find a vast sampling of the parameter space.
9
SPSA: Pseudocode
end
on x sconstraintlower andupper Check :8
xa-x x Update:7
))./(2cy-(yx stepSet :6
);f(xy and )f(xy evaluationFunction :5
;c-x xand cxon xPerturbati usSimultaneo :4
1) - 1)) nd(n;2(round(ra on vector perturbati random Set the :3
k
c c and
A) (k
a aSet :2
k:1kfor :1
),c,A,a,,kn,SPSA(x,[x]function
k
k-
--
k-
k
k
kk
max
max
0.6021.101,/10),k(10,A :Note max
10
Observations
SPSA gives region(s) where the function value is low, and this allows to conjecture in which region(s) is the global solution.
This give us a motivation to apply a local method in the region(s) found by SPSA.
11
Local Method
Advantages Fast Method: Newton Type Methods Interior-Point Methods allow to add
equality/inequality constraints
Disadvantage Needs first/second order information
Solution Construct a Surrogate Model using the
SPSA function values inside the conjecture region(s)
12
A surrogate model is created by using an interpolation method with the data, , provided by SPSA.
This can be performed in different ways, e.g., radial basis functions, kriging, regression analysis, or using artificial neural networks.
pkxfx kk ,...,1 )),(,(
)( ks xf
Surrogate Model
13
Why Surrogate Models?
Most real problems require thousands or millions of objective and constraint function evaluations, and often the associated high cost and time requirements render this infeasible.
14
RBF is typically parameterized by two sets of parameters: the center c which defines its position, and shape r that determines its width or form.
An RBF interpolation algorithm (Orr,1996) characterizes the uncertainty parameters:
.,..,1 ,,...,1 , , , mjniwrc jijij
Radial Basis Function (RBF)
15
Our goal is to optimize the surrogate function
Where the radial basis functions can be defined as:
that are the multiquadric and the Gaussian basis functions, respectively
Surrogate Model
1
( ) ( ),m
s j jj
f x w h x
2
21
2( )
1( )( ) 1 or ( )
ni ij
ji ij
x cn i ijrijix c
h x h x ejr
16
Hybrid Optimization Scheme (1)
Global Search Via SPSA
FilteringFiltering
Surrogate Model
Explore Parameter space
Multistart (x0)1(x0)2
.
.
.
(x0)k
SamplingSampling
Target Region
.
.
.
1
( ) ( ),m
s j jj
f x w h x
+
17
Test Case 2
Rastrigin’s Problem:
21 2
1
min ( , ,..., ) 10 ( 10cos(2 )),
s.t. 5.12 5.12
for 1,2,...,
n
n i ii
i
f x x x n x x
x
i n
18
Test Case 2: Rastringin
19
Test Case 2: Rastringin
x*=(0,0) global solutionf(x*)=0
20
Sampling Data from SPSA
Test Case 2: Restringin
21
x*=(0,0) global solutionf(x*)=0
Sampling Data from SPSA
Test Case 2: Restringin
22
Restringin: Surrogate ModelWe plot the original model function and the surrogate function:
23
Restringin: Surrogate Model
24
Local Search: NKIP
NKIP is a globalized and derivative dependent optimization method based on the global strategy introduced by Miguel Argaéz and Richard Tapia in 2002.
This method calculates the directions using the conjugate gradient algorithm, and a linesearch is implemented to guarantee a sufficient decrease of the objective function.
25
Local Search: NKIP
We consider the optimization problem in the form:
where a and b are determined by the sampled points given by SPSA.
minimize ( )
subject to Sf x
a x b
26
Hybrid Optimization Scheme (2)
27
Hybrid Optimization Scheme
Convergence history for minimizing Rastringin’s function (n=50).
28
2|| d- G(x)|| minimize T
where is a nonlinear functionand is a set of data observations
The objective function can be written as a nonlinear least squares problem:
mT Rd
mn RR:G(x)
Parameter Estimation Problem
2|| d- G(x)|| minimize T
The objective function can be written as a nonlinear least squares problem:
mn RR:G(x)
2|| d- G(x)|| minimize T
The objective function can be written as a nonlinear least squares problem:
29
Our goal is to estimate the permeability based on sensor pressure data.
Despite having full information of the pressure field, the inverse problem is highly ill-posed and has multiple local minima.
Application
30
Large Scale Two-Phase Flow Problem
3D permeability field
31
Large Scale Two-Phase Flow Problem
Figure 2. The true and initial permeability and pressure fields.
32
High Performance Computing
We run the simulator IPARS (Integrated Parallel Accurate Reservoir Simulator) on a Linux-based multicore network of workstations.
Our goal is to estimate a permeability field that involves 2000 parameters. The field is parameterized by SVD reducing the parameter space to 20 (each parameter represents a scale resolution level).
We choose several initials points by adding a percentage of noise to the true singular values: 5%, 10% and 15%
33
Numerical Results
Noise 5% 10% 15%
Initial starting points by SPSA 30 20 20
Range of f(x0) 8.56 to 10.3 18.72 to 53.9 29.46 to 120.56
SPSA Iterations 3645 2248 1647
Range of fspsa(x) 0.568 to 2.5 0.858 to 6.785 1.541 to 15.89
Best fspsa(x) 0.568 0.858 1.541
Data points used in the surrogate model 156 81 57
NKIP Iterations 94 67 110
fnkip(x) 0.478 0.747 1.465
% Gain by Hybrid Approach 16 13 5
34
We combine SPSA and NKIP strategies into a Hybrid Scheme that exploit the best of these two approaches for a given problem in order to achieve maximum efficiency and robustness.
The hybrid approach finds a good estimate of the global solution for all the test cases.
As the noise level was increased, the hybrid approach presented more difficulties in reproducing the true permeability field. However, the estimation is very accurate in all cases.
Conclusions
35
Improve a parallel version of the multi-start technique
Parallelize the conjugate gradient of NKIP
Add equality constraints to the minimization of the surrogate model
Add more Global Optimization Techniques (Simulated Annealing, Evolutionary Algorithms)
Future Work
3636
Thank you very much for your attention
top related