certified and fast computation of supremum norms of approximation
TRANSCRIPT
Certi�ed and fast computation of supremumnorms of approximation errors
Sylvain Chevillard Mioara Joldes Christoph Lauter
May 26, 2009
1 / 32
Outline
Motivation
Correctly rounded elementary functionsSupremum norm of error functionsPrevious approaches and di�culties
Our approach
Automatic di�erentiation and Taylor modelsIsolation of roots of polynomialsEnclosure of roots of functions
Results & Conclusion
2 / 32
Motivation
�It makes me nervous to �y on airplanes, since I know they are
designed using �oating-point arithmetic.�
A. Householder
Software systems: scienti�c computing, �nancial, embeddedsystems.
Need to compute sin, cos, exp in �nite precision,�oating-point environment.
Most Mathematical Libraries do not provide correctly roundedfunctions.
IEEE-754 standard revision (June 2008) recommends correctrounding.
Arenaire team develops the Correctly Rounded Libm(CRLibm)1.
1http://lipforge.ens-lyon.fr/www/crlibm/
3 / 32
Motivation
�It makes me nervous to �y on airplanes, since I know they are
designed using �oating-point arithmetic.�
A. Householder
Software systems: scienti�c computing, �nancial, embeddedsystems.
Need to compute sin, cos, exp in �nite precision,�oating-point environment.
Most Mathematical Libraries do not provide correctly roundedfunctions.
IEEE-754 standard revision (June 2008) recommends correctrounding.
Arenaire team develops the Correctly Rounded Libm(CRLibm)1.
1http://lipforge.ens-lyon.fr/www/crlibm/
3 / 32
Motivation
�It makes me nervous to �y on airplanes, since I know they are
designed using �oating-point arithmetic.�
A. Householder
Software systems: scienti�c computing, �nancial, embeddedsystems.
Need to compute sin, cos, exp in �nite precision,�oating-point environment.
Most Mathematical Libraries do not provide correctly roundedfunctions.
IEEE-754 standard revision (June 2008) recommends correctrounding.
Arenaire team develops the Correctly Rounded Libm(CRLibm)1.
1http://lipforge.ens-lyon.fr/www/crlibm/
3 / 32
Motivation
�It makes me nervous to �y on airplanes, since I know they are
designed using �oating-point arithmetic.�
A. Householder
Software systems: scienti�c computing, �nancial, embeddedsystems.
Need to compute sin, cos, exp in �nite precision,�oating-point environment.
Most Mathematical Libraries do not provide correctly roundedfunctions.
IEEE-754 standard revision (June 2008) recommends correctrounding.
Arenaire team develops the Correctly Rounded Libm(CRLibm)1.
1http://lipforge.ens-lyon.fr/www/crlibm/3 / 32
Correctly rounded functions
4 / 32
Correctly rounded functions
4 / 32
Correctly rounded functions
Table Maker's Dilemma
Increase working precision
Worst cases search - V. Lefevre and J-M. Muller
5 / 32
Supremum Norms of Error Functions
ε(x) = f(x)− p(x), x ∈ [a, b] or
ε(x) = p(x)f(x) − 1, x ∈ [a, b]
De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}
Compute a certi�ed bound for the supremum norm of an errorfunction
�Quick and dirty� supremum norms - another class ofalgorithms
6 / 32
Supremum Norms of Error Functions
ε(x) = f(x)− p(x), x ∈ [a, b] or
ε(x) = p(x)f(x) − 1, x ∈ [a, b]
De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}
Compute a certi�ed bound for the supremum norm of an errorfunction
�Quick and dirty� supremum norms - another class ofalgorithms
6 / 32
Supremum Norms of Error Functions
ε(x) = f(x)− p(x), x ∈ [a, b] or
ε(x) = p(x)f(x) − 1, x ∈ [a, b]
De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}
Compute a certi�ed bound for the supremum norm of an errorfunction
�Quick and dirty� supremum norms - another class ofalgorithms
6 / 32
Supremum Norms of Error Functions
ε(x) = f(x)− p(x), x ∈ [a, b] or
ε(x) = p(x)f(x) − 1, x ∈ [a, b]
De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}
Compute a certi�ed bound for the supremum norm of an errorfunction
�Quick and dirty� supremum norms - another class ofalgorithms
6 / 32
Supremum Norms of Error Functions
Error ε(x) = f(x)− p(x) or ε(x) = p(x)f(x) − 1, x ∈ [a, b]
De�ne ‖ε‖∞ = supx∈[a, b]{|ε(x)|}Purpose: Compute a certi�ed bound for the supremum normof an error function
Given p and f �nd a narrow interval r such that ‖ε‖∞ ∈ r.
Need for a fast and certi�ed algorithm:
Correctly rounded elementary functions
For computing the minimum error between a function andthousands of polynomials with �oating-point coe�cients
7 / 32
Supremum norm of error functions
Example:
f(x) = ex, x ∈ [0, 1]p(x) =
∑5i=0 cix
i s.t. ‖f − p‖∞ is as small as possible (Remezalgorithm)ε(x) = f(x)− p(x)
How to obtain a certi�ed and tight bound?
8 / 32
Supremum norm of error functions
Example:
f(x) = ex, x ∈ [0, 1]p(x) =
∑5i=0 cix
i s.t. ‖f − p‖∞ is as small as possible (Remezalgorithm)ε(x) = f(x)− p(x)
How to obtain a certi�ed and tight bound?
8 / 32
Supremum norm of error functions
Example:
f(x) = ex, x ∈ [0, 1]p(x) =
∑5i=0 cix
i s.t. ‖f − p‖∞ is as small as possible (Remezalgorithm)ε(x) = f(x)− p(x)
How to obtain a certi�ed and tight bound?8 / 32
Our Problem - Prone to High Dependency Phenomenon
Example:
f(x) = ex, x ∈ [0, 1]p(x) =
∑5i=0 cix
i s.t. ‖f − p‖∞ is as small as possible (Remezalgorithm)ε(x) = f(x)− p(x)
Using IA, ε(x) ∈ [−0.4, 0.4], but ‖ε(x)‖∞ ' 1.1295e− 6:9 / 32
Our Problem - Prone to High Dependency Phenomenon
Example:
f(x) = ex, x ∈ [0, 1]p(x) =
∑5i=0 cix
i s.t. ‖f − p‖∞ is as small as possible (Remezalgorithm)ε(x) = f(x)− p(x)
Using IA, ε(x) ∈ [−0.4, 0.4], but ‖ε(x)‖∞ ' 1.1295e− 6:9 / 32
Our Problem - Prone to High Dependency Phenomenon
In this case, over [0, 1] we need 107 intervals!
10 / 32
Previous Approaches
Floating-point techniques - not �safe�
Existent Interval Arithmetic methods - Not su�cient
Global optimization software (eg. Globsol) - not tailored forour speci�c problem
Chevillard and Lauter's technique: interval arithmetic, tightbounding of the zeros of the derivative of the error function,removes false singularities (like x = 0 for sin(x)/x). Highcomputation time for deg(p) > 10.Techniques based on a high order Taylor expansion of the errorfunction and a su�ciently close bounding of the remainder
Certi�ed polynomial approximations of analytic functions.
11 / 32
Our Approach
How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?
Use multiprecision IA: MPFI 2
Use a higher degree approximation polynomial T for thefunction f
Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].Break the dependency problem in two:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸
bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸
tightly bounding a polynomial
1 Automatic Di�erentiation/ Taylor Models
2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques
2
http://gforge.inria.fr/projects/mp�/
12 / 32
Our Approach
How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?Use multiprecision IA: MPFI 2
Use a higher degree approximation polynomial T for thefunction f
Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].Break the dependency problem in two:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸
bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸
tightly bounding a polynomial
1 Automatic Di�erentiation/ Taylor Models
2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques
2http://gforge.inria.fr/projects/mp�/12 / 32
Our Approach
How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?Use multiprecision IA: MPFI 2
Use a higher degree approximation polynomial T for thefunction f
Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].
Break the dependency problem in two:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸
bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸
tightly bounding a polynomial
1 Automatic Di�erentiation/ Taylor Models
2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques
2http://gforge.inria.fr/projects/mp�/12 / 32
Our Approach
How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?Use multiprecision IA: MPFI 2
Use a higher degree approximation polynomial T for thefunction f
Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].Break the dependency problem in two:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸
bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸
tightly bounding a polynomial
1 Automatic Di�erentiation/ Taylor Models
2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques
2http://gforge.inria.fr/projects/mp�/12 / 32
Our Approach
How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?Use multiprecision IA: MPFI 2
Use a higher degree approximation polynomial T for thefunction f
Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].Break the dependency problem in two:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸
tightly bounding a polynomial
1 Automatic Di�erentiation/ Taylor Models
2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques
2http://gforge.inria.fr/projects/mp�/12 / 32
Our Approach
How to obtain a certi�ed and tight bound for‖ε‖∞ = ‖f − p‖∞ AND ‖ε‖∞ = ‖p/f − 1‖∞, over giveninterval [a, b]?Use multiprecision IA: MPFI 2
Use a higher degree approximation polynomial T for thefunction f
Compute (T,∆) s.t f(x)− T (x) ∈ ∆,∀x ∈ [a, b].Break the dependency problem in two:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸tightly bounding a polynomial
1 Automatic Di�erentiation/ Taylor Models
2 Tightly bounding the polynomial di�erence - Roots isolationand re�nement techniques
2http://gforge.inria.fr/projects/mp�/12 / 32
(1) Computing (T, ∆) - Automatic di�erentiation (AD)
Example:
f(x) = exp(x) over [0, 1]p(x) =
∑5i=0 cix
i
Introduce a higher degree polynomial T : Use AD
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i =
8∑i=0
exp(1/2)i! (x− 1/2)i
Compute an enclosure of the remainder: Use AD
∆9(x, ξ) =f (9)(ξ)
9!︸ ︷︷ ︸∈exp([0, 1])
9!
× (x− 1/2)9︸ ︷︷ ︸≤(1/2)(−9)
|∆9(x, ξ)| ∈ 1.4630578142[1; 2]× e− 8
13 / 32
(1) Computing (T, ∆) - Automatic di�erentiation (AD)
Example:
f(x) = exp(x) over [0, 1]p(x) =
∑5i=0 cix
i
Introduce a higher degree polynomial T : Use AD
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i =
8∑i=0
exp(1/2)i! (x− 1/2)i
Compute an enclosure of the remainder: Use AD
∆9(x, ξ) =f (9)(ξ)
9!︸ ︷︷ ︸∈exp([0, 1])
9!
× (x− 1/2)9︸ ︷︷ ︸≤(1/2)(−9)
|∆9(x, ξ)| ∈ 1.4630578142[1; 2]× e− 8
13 / 32
(1) Computing (T, ∆) - Automatic di�erentiation (AD)
Example:
f(x) = exp(x) over [0, 1]p(x) =
∑5i=0 cix
i
Introduce a higher degree polynomial T : Use AD
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i =
8∑i=0
exp(1/2)i! (x− 1/2)i
Compute an enclosure of the remainder: Use AD
∆9(x, ξ) =f (9)(ξ)
9!︸ ︷︷ ︸∈exp([0, 1])
9!
× (x− 1/2)9︸ ︷︷ ︸≤(1/2)(−9)
|∆9(x, ξ)| ∈ 1.4630578142[1; 2]× e− 813 / 32
(1) Computing (T, ∆) - Automatic di�erentiation (AD)
Main idea:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸bounding a remainder
+ ‖T − p‖∞︸ ︷︷ ︸bounding a polynomial
Achieved so far:
‖exp−p‖∞ ≤ ‖exp−T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸bounding a polynomial
14 / 32
(2)Bounding the polynomial di�erence
Purpose: Tightly bound ‖T − p‖∞ over the interval [a, b]
Example:
p(x) =∑5
i=0 cixi
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i
Tightly bound theroots of the derivative
Evaluate usinginterval arithmetic
15 / 32
(2)Bounding the polynomial di�erence
Purpose: Tightly bound ‖T − p‖∞ over the interval [a, b]
Example:
p(x) =∑5
i=0 cixi
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i
Tightly bound theroots of the derivative
Evaluate usinginterval arithmetic
15 / 32
(2)Bounding the polynomial di�erence
Purpose: Tightly bound ‖T − p‖∞ over the interval [a, b]
Example:
p(x) =∑5
i=0 cixi
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i
Tightly bound theroots of the derivative
Evaluate usinginterval arithmetic
15 / 32
(2)Bounding the polynomial di�erence
Purpose: Tightly bound ‖T − p‖∞ over the interval [a, b]
Example:
p(x) =∑5
i=0 cixi
T (x) =8∑
i=0
f (i)(1/2)i! (x− 1/2)i
Tightly bound theroots of the derivative
Evaluate usinginterval arithmetic
15 / 32
(3)Isolation and re�nement of roots of polynomials
Techniques based on counting the number of roots inside aninterval considered
Sturm TheoremDescartes' Rule of Signs
Use a bisection strategy for isolating the roots
Use dichotomy or Newton iteration process
16 / 32
(3)Isolation and re�nement of roots of polynomials
Techniques based on counting the number of roots inside aninterval considered
Sturm TheoremDescartes' Rule of Signs
Use a bisection strategy for isolating the roots
Use dichotomy or Newton iteration process
16 / 32
(3)Isolation and re�nement of roots of polynomials
Techniques based on counting the number of roots inside aninterval considered
Sturm TheoremDescartes' Rule of Signs
Use a bisection strategy for isolating the roots
Use dichotomy or Newton iteration process
16 / 32
(3)Isolation and re�nement of roots of polynomials
Techniques based on counting the number of roots inside aninterval considered
Sturm TheoremDescartes' Rule of Signs
Use a bisection strategy for isolating the roots
Use dichotomy or Newton iteration process
16 / 32
(3)Bounding the polynomial di�erence
r0 ∈ 0.0068[5; 6], r1 ∈ 0.2544[3; 4],r2 ∈ 0.5059[4; 5], r3 ∈ 0.7544[7; 8],r4 ∈ 0.9345[8; 9]
‖T − p‖∞ ∈ 1.11486943[1; 2]× e− 6
17 / 32
(3)Bounding the polynomial di�erence
r0 ∈ 0.0068[5; 6], r1 ∈ 0.2544[3; 4],r2 ∈ 0.5059[4; 5], r3 ∈ 0.7544[7; 8],r4 ∈ 0.9345[8; 9]‖T − p‖∞ ∈ 1.11486943[1; 2]× e− 6
17 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.
18 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.
18 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.
18 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.
18 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.
18 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.
18 / 32
Our Approach - Summary
1 Purpose: fast and safely compute the supremum norm‖f − p‖∞ over an interval [a, b]
2 Introduce a higher degree approximation polynomial:‖f − p‖∞ ≤ ‖f − T‖∞ + ‖T − p‖∞ (AD/TM)
3 Bound the remainder (AD/TM)
4 Bound the polynomial di�erence (polynomial roots isolation)
5 Add the two bounds to obtain a tight and safe bound of theapproximation error
Our example:
‖f − p‖∞ ≤ ‖f − T‖∞︸ ︷︷ ︸∈ 1.4630578142[1;2]×e−8
+ ‖T − p‖∞︸ ︷︷ ︸∈ 1.11486943[1;2]×e−6
‖f − p‖∞ ∈ 1.1295[0; 1]× e− 6
IA: ‖f − p‖∞ ≤ 0.4, sampling: ‖f − p‖∞ ' 1.1295e− 6.18 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work! WHY?
19 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work! WHY?
19 / 32
(1) Computing (T, ∆) - Taylor Models at a Glance
Couple of form (T,∆), s.t. f − T ∈ ∆.
Propagation of error bounds combined with AD
Start with trivial Taylor Model: (1, [0, 0]).Easily de�ne arithmetic operations: +,−, ∗. Eg.Multiplication:(T1,∆1) ∗ (T2,∆2) =((T1T2)0..n,∆1B(T2) + ∆2B(T1) +B((T1T2)n..2n))Note: The bound B(T ) computed is propagated in theremainder. It in�uences the quality of the remainder!
Usually the bounds on the occuring polynomials are rough!
Tight bounds imply a computational e�ort at each operation.
20 / 32
Relative Error Issues
Example:
f(x) = cos(x) over [−1, 1]p(x) =
∑5i=0 cix
i
ε(x) = p(x)/f(x)− 1
Compute the Taylor Model for ε(x), order 12.
TM for f(x), remainder: [−1.351322e− 10; 1.351322e− 10]TM for 1
f(x) , remainder: [−3.1634304e− 2; 0.258216]
Remainder bounds are unsatisfactory in our case.
21 / 32
Relative Error Issues
Example:
f(x) = cos(x) over [−1, 1]p(x) =
∑5i=0 cix
i
ε(x) = p(x)/f(x)− 1
Compute the Taylor Model for ε(x), order 12.
TM for f(x), remainder: [−1.351322e− 10; 1.351322e− 10]TM for 1
f(x) , remainder: [−3.1634304e− 2; 0.258216]
Remainder bounds are unsatisfactory in our case.
21 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0
HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0
HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.
Solve τ(x) = 0
HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0
HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0
HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0
HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Our Approach - Relative Error
Compute: ‖ε‖∞ = ‖p/f − 1‖∞ over I
Brutal application of the same principle does not work:
Tight bounds for the polynomials interfering in Taylor Modelshave to be computedSome compositions (inverse function, logarithm) generateintervals that are too wide in our case
Idea: Tightly enclose all the extrema of ε: they are among thezeros of the �rst derivative.
ε′ = (p′f − pf ′)/f2. Consider τ = p′f − pf ′.Solve τ(x) = 0 HOW?
Find small intervals that enclose each root of τ .
Evaluate ε on these intervals.
22 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!
Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ I
τ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
First idea: interval Newton Method BUT Dependencyphenomenon present in τ = p′f − pf ′ also!Better idea: Compute a Taylor Model (T, [−θ, θ]) s.t.τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]Compute a list Lu of intervals where T − θ ≤ 0 and Ll whereT + θ ≥ 0
23 / 32
Finding enclosures of roots of a function
Compute a list of intervals where T − θ ≤ 0, T is a polynomial
Compute enclosures of the roots of T − θCompute sign changes
Find suitable intervals
24 / 32
Finding enclosures of roots of a function
Compute a list of intervals where T − θ ≤ 0, T is a polynomial
Compute enclosures of the roots of T − θ
Compute sign changes
Find suitable intervals
24 / 32
Finding enclosures of roots of a function
Compute a list of intervals where T − θ ≤ 0, T is a polynomial
Compute enclosures of the roots of T − θCompute sign changes
Find suitable intervals
24 / 32
Finding enclosures of roots of a function
Compute a list of intervals where T − θ ≤ 0, T is a polynomial
Compute enclosures of the roots of T − θCompute sign changes
Find suitable intervals
24 / 32
Our example - Relative error
Example:
f(x) = cos(x) over [−1, 1], p(x) =∑5
i=0 cixi,
ε(x) = p(x)/f(x)− 1
25 / 32
Our example - Relative error
Example:
f(x) = cos(x) over [−1, 1], p(x) =∑5
i=0 cixi,
ε(x) = p(x)/f(x)− 1
Compute Taylor Model for τ = p′f − pf ′, order 12.
Remainder: [−1.67352e− 7; 1.67352e− 7]
25 / 32
Our example - Relative error
r0 ∈ −0.886[9; 8], r1 ∈ −0.536[6; 4],r2 ∈ [−0.000001; 0.000001], r3 ∈ 0.536[4; 6],r4 ∈ 0.886[8; 9]
25 / 32
Our example - Relative error
r0 ∈ −0.886[9; 8], r1 ∈ −0.536[6; 4],r2 ∈ [−0.000001; 0.000001], r3 ∈ 0.536[4; 6],r4 ∈ 0.886[8; 9]‖ε‖∞ ∈ 5.59[3/4]× e− 5.
25 / 32
Our Approach - Summary
Purpose: fast and safely compute the supremum norm‖f − p‖∞ and ‖p/f − 1‖∞ over an interval [a, b]Enclose all the zeros of the �rst derivative of the error
Evaluate the approximation error on these small intervals only,using IA
Use a Taylor Model based approach to overcome thedependency
Enclose the zeros of a function using our algorithm
26 / 32
Our example - Relative error - How small should theremainder be?
τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
27 / 32
Our example - Relative error - How small should theremainder be?
τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
Degree: 1227 / 32
Our example - Relative error - How small should theremainder be?
τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
27 / 32
Our example - Relative error - How small should theremainder be?
τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
Degree: 1427 / 32
Our example - Relative error - How small should theremainder be?
τ(x)− T (x) ∈ [−θ, θ],∀x ∈ Iτ(x) = 0 =⇒ T (x) ∈ [−θ, θ]
Degree: 827 / 32
Our example - Relative error - How small should theremainder be?
Heuristics
Start with a value close to the approx. max value of τ
In our example: θi = 6.44e− 4
θ degree T ‖ε‖∞θi 8 [5.59e− 5; 1.84]θi/2 9 [5.59e− 5; 1e− 3]θi/10 10 [5e− 5; 8e− 5]θi/102 11 0.5[5; 7]e− 4θi/103 12 0.559[3; 4]e− 4θi/104 13 0.55935[2; 4]e− 4θi/105 14 0.5593528[2; 4]e− 4
28 / 32
Results
Experiments were made on an Intel Pentium D 3.00GHz with a2GB RAM.
f [a, b] dp1 m2 acc3 time4
exp(x)− 1 [−0.25, 0.25] 5 r 37.6 412log2(1 + x) [−2−9, 2−9] 7 r 83.3 2, 186
cos(x) [−0.5, 0.25] 15 r 19.5 2, 235exp(x) [−0.125, 0.125] 25 r 42.3 7, 753sin(x) [−0.5, 0.5] 9 a 21.5 520
exp(cos(x)2 + 1) [1, 2] 15 r 25.5 10, 984tan(x) [0.25, 0.5] 10 r 26.0 1, 072x2.5 [1, 2] 7 r 15.5 1, 362
1Degree of p2Error mode considered: a=absolute, r=relative3Accuracy4Timings in ms
29 / 32
Conclusion
Safe and fast algorithm for bounding the supremum norm ofthe error functions
Combination and reusal of various techniques (TM, polynomialroots isolation, interval arith)
Absolute and Relative errors handled
Faster and more accurate than other current approaches
Future works:
Formal proof (AD, isolation of roots, multiple precision intervalarithmetic are needed in the proof checker)Generalization of the algorithm for multivariate functions.
30 / 32
Questions
Thank you for your attention!
Questions?
31 / 32
Results
Experiments were made on an Intel Pentium D 3.00GHz with a2GB RAM.
f [a, b] dp1 m2 dT
3 acc4 time5
exp(x)− 1 [−0.25, 0.25] 5 r 11 37.6 412log2(1 + x) [−2−9, 2−9] 7 r 23 83.3 2, 186
cos(x) [−0.5, 0.25] 15 r 28 19.5 2, 235exp(x) [−0.125, 0.125] 25 r 41 42.3 7, 753sin(x) [−0.5, 0.5] 9 a 14 21.5 520
exp(cos(x)2 + 1) [1, 2] 15 r 60 25.5 10, 984tan(x) [0.25, 0.5] 10 r 21 26.0 1, 072x2.5 [1, 2] 7 r 26 15.5 1, 362
1Degree of p2Error mode considered: a=absolute, r=relative3Degree of T4Accuracy5Timings in ms
32 / 32