cubic interpolation report
DESCRIPTION
Cubic Interpolation for Line Search in Optimization.TRANSCRIPT
-
The Cubic Interpolation Method
Erika Paula L. MoisesJoe Mari D. Reyes
Math 288 (Numerical Optimization)
May 4, 2015
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Presentation Outline
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
The cubic interpolation method approximates the objectivefunction
g() = f (x + d)
by a cubic polynomial, where f is continuously differentiable. Bythe chain rule, we have
g () =dg()
d= f (x + d)Td
We assume that g (0) = f (x)Td < 0, that is, d is the descentdirection at x .
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
To construct the cubic polynomial p(t), four interpolationconditions are required.
For example, we may use:
function values at four points, or
function values at three points and a derivative value at onepoint; or
function values and derivative values at two points
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
In this report, we will focus on interpolation using function valuesand derivative values at two points a and b.
To guarantee that the appropriate minimum occurs in the interval[a, b], the following condition should hold:
g (a) < 0 and g (b) > 0
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
The central idea is to use the derivative information in the endpoints of the interval. Together with the function values, theminimum of the given function is approximated by the minimum ofa cubic polynomial.
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
We construct a cubic polynomial of the form
p(t) = c1(t a)3 + c2(t a)2 + c3(t a) + c4 (1)
where ci are the coefficients of the polynomial which are chosensuch that
p(a) = c4 = g(a),
p(a) = c3 = g (a),
p(b) = c1(b a)3 + c2(b a)2 + c3(b a) + c4 = g(b),p(b) = 3c1(b a)2 + 2c2(b a) + c3 = g (b).
(2)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
From the sufficient condition of the minimizer, we have
p(t) = 3c1(t a)2 + 2c2(t a) + c3 = 0 (3)
andp(t) = 6c1(t a) + 2c2 > 0 (4)
Solving Equation (3) yields
t = a +c2
c22 3c1c3
3c1, if c1 6= 0 (5)
t = a c32c2
, if c1 = 0. (6)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
In order to guarantee that condition (4) holds, we only take thepositive in Equation (5).
By equation (5), we get:
t a =c2 +
c22 3c1c3
3c1=
c3c2 +
c22 3c1c3
(7)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
When c1 = 0, Equation (7) is just Equation (6). Then theminimizer of p(t) is
t = a c3c2 +
c22 3c1c3
(8)
The minimizer in Equation (8) is represented by c1,c2 and c3.
We hope to represent t by g(a),g(b), g (a) and g (b) directly.
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
We now let
s = 3g(b) g(a)
b az = s g (a) g (b),w2 = z2 g (a)g (b) (9)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
By use of Equation (2), we get
s = 3g(b) g(a)
b a = 3[c1(b a)2 + c2(b a) + c3],
z = s g (a) g (b) = c2(b a) + c3,w2 = z2 g (a)g (b) = (b a)2(c22 3c1c3).
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
Then
(b a)c2 = z c3c22 3c1c3 =
w
b a
And so
c2 +
c22 3c1c3 =z + w c3
b a (10)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
Using c3 = g(a) and substituting Equation (10) into Equation (8),
we get:
t a = (b a)g(a)
z + w g (a) , (11)
which is
t a = (b a)g(a)g (b)
(z + w g (a))g (b) =(b a)(z2 w2)
g (b)(z + w) (z2 w2)
=(b a)(w z)g (b) z + w (12)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
Note that the denominator in Equation (12) may be equal to zeroor sufficiently small. We can overcome this by the use of Equation(11) on (12). We get
t a = (b a)g(a)
z + w g (a) =(b a)(w z)g (b) z + w
=(b a)(g (a) + w z)
g (b) g (a) + 2w
= (b a)(1 g(b) + z + w
g (b) g (a) + 2w ) (13)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
This gives us the iterates of the Cubic Interpolation Method:
t = a + (b a) w g(a) z
g (b) g (a) + 2w (14)
where
s = 3g(b) g(a)
b a , z = s g(a) g (b)
w2 = z2 g (a)g (b)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
The Cubic Interpolation Method
In Equation (13) and (14), the denominator
g (b) g (a) + 2w 6= 0
.In fact, since g (a) < 0 and g (b) > 0, then
w2 = z2 g (a)g (b) > 0
.Taking w > 0, it follows that the denominator
g (b) g (a) + 2w > 0
.
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Remark
Cubic interpolation has the danger of lack of convergencenumerically, but the theoretical convergence of the iterate is veryfast.
It can be shown that the cubic interpolation method convergessuperlinearly.
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Presentation Outline
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Cubic Interpolation Steps
We now lay down the steps for the Cubic Interpolation Method
Step 1 (Determination of the Initial Interval)
Let s > 0 be some scalar.
Evaluate g(t) and g (t) at the points t = 0, s, 2s, 4s, 8s, ... untiltwo successive points a and b are found such that either g (b) 0or g(b) g(a).
Then, it can be seen that a local minimum of g exists within theinterval (a, b].
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Cubic Interpolation Steps
Step 2 (Updating of the Current Interval)
Given the current interval [a, b], a cubic polynomial is fitted to thefour values g(a), g (a), g(b) and g (b).
The cubic can be shown to have a unique minimum t in theinterval (a, b] given by
t = a + (b a) w g(a) z
g (b) g (a) + 2w ,
where z = 3g(b)g(a)ba g (a) g (b), w =
z2 g (a)g (b)
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Cubic Interpolation Steps
Step 2 (Updating of the Current Interval)
If g (t) 0 or g (t) g(a) replace b by t.
If g (t) < 0 and g (t) < g(a) replace a by t.
Stopping Criterion
In practice, the computation is terminated once the length of thecurrent interval becomes smaller than a specified tolerance or elsewe obtain t = b.
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Presentation Outline
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Example 1
Cubic interpolation is applied to find the minimum f (x) = x + 16x+1with starting interval [2, 4.5] and accuracy tol = 0.01.
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Example 1
One iteration after reaching the stopping criterion has been givenin Table 1. For this case, the interval changes very fast around theminimum point.
k ak bk xk f(xk) f(xk)
1 2.000 4.500 3.024 7.0001 0.0122 2.000 3.024 2.997 7.0000 -0.0013 2.997 3.024 3.000 7.0000 0.0004 2.997 3.000 3.000 7.0000 0.000
Table: Cubic interpolation for f (x) = x + 16x+1 ,[a0, b0] = [2, 4.5],tol = 0.01
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
References:
Bertsekas, Dimitri P. 1999. Nonlinear Programming. 2ndEdition. pp 723-724
Sun, Wenyu. Yuan, Ya-Xiang. 2006. Optimization Theoryand Methods (Nonlinear Programming). Volume 1. pp 98-101
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method
-
Thank you
Erika Paula L. Moises, Joe Mari D. Reyes The Cubic Interpolation Method