a history of numerical analysis ideas
DESCRIPTION
A History of Numerical Analysis Ideas. Prepared for CS 378 History of Computing October 14, 2003. Alan Kaylor Cline Department of Computer Sciences The University of Texas at Austin. What is Different in Numerical Computing?. What is Different in Numerical Computing?. Well, it’s numbers. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/1.jpg)
A History of Numerical Analysis Ideas
Alan Kaylor ClineDepartment of Computer SciencesThe University of Texas at Austin
Prepared for CS 378History of Computing
October 14, 2003
![Page 2: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/2.jpg)
What is Different in Numerical Computing?
![Page 3: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/3.jpg)
What is Different in Numerical Computing?
Well, it’s numbers
![Page 4: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/4.jpg)
Scientific Computing vs.
Numerical Analysis
![Page 5: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/5.jpg)
Your Original CS Department
1966
![Page 6: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/6.jpg)
Your Original CS Department
196610 faculty with 2 numerical analysts
![Page 7: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/7.jpg)
Is there anything special with numerical computing?
![Page 8: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/8.jpg)
A Small Example
A computation of π
![Page 9: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/9.jpg)
Simple iteration:
PP
nn n
n FHGIKJ
FHGG
IKJJ1
2
2 2 1 12
n 2 3, ,...
2 2 2P
![Page 10: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/10.jpg)
2 2.828427124746190 3 3.061467458920719 4 3.121445152258053 5 3.136548490545941 6 3.140331156954739 7 3.141277250932757 8 3.141513801144145 9 3.141572940367883 10 3.141587725279961 11 3.141591421504635 12 3.141592345611077 13 3.141592576545004 14 3.141592633463248 15 3.141592654807589 16 3.141592645321215 17 3.141592607375720 18 3.141592910939673 19 3.141594125195191 20 3.141596553704820 21 3.141596553704820 22 3.141674265021758 23 3.141829681889202 24 3.142451272494134 25 3.142451272494134 26 3.162277660168380 27 3.162277660168380 28 3.464101615137754 29 4.000000000000000 30 0.000000000000000 31 0.000000000000000
Result of 15 digit computation
Red digits are correct
White and green digits are incorrect
![Page 11: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/11.jpg)
2 2.828427124746190 3 3.061467458920719 4 3.121445152258053 5 3.136548490545941 6 3.140331156954739 7 3.141277250932757 8 3.141513801144145 9 3.141572940367883 10 3.141587725279961 11 3.141591421504635 12 3.141592345611077 13 3.141592576545004 14 3.141592633463248 15 3.141592654807589 16 3.141592645321215 17 3.141592607375720 18 3.141592910939673 19 3.141594125195191 20 3.141596553704820 21 3.141596553704820 22 3.141674265021758 23 3.141829681889202 24 3.142451272494134 25 3.142451272494134 26 3.162277660168380 27 3.162277660168380 28 3.464101615137754 29 4.000000000000000 30 0.000000000000000 31 0.000000000000000 . . .
Result of 15 digit computation
Red digits are correct
White and green digits are incorrect
π = 0 ?
![Page 12: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/12.jpg)
Where’s the problem?
![Page 13: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/13.jpg)
Where’s the problem?
1 1 small
PP
nn n
n FHGIKJ
FHGG
IKJJ1
2
2 2 1 12
is calculated as zero
![Page 14: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/14.jpg)
1 1 small
Let’s replace
with the algebraically identical expression
1 1
small
small
![Page 15: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/15.jpg)
New iteration:
n 2 3, ,...
2 2 2P
12
2
1 1 ( )2
n n
nn
P PP
results in …
![Page 16: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/16.jpg)
2 2.828427124746190 3 3.061467458920719 4 3.121445152258053 5 3.136548490545941 6 3.140331156954739 7 3.141277250932757 8 3.141513801144145 9 3.141572940367883 10 3.141587725279961 11 3.141591421504635 12 3.141592345611077 13 3.141592576545004 14 3.141592633463248 15 3.141592654807589 16 3.141592645321215 17 3.141592607375720 18 3.141592910939673 19 3.141594125195191 20 3.141596553704820 21 3.141596553704820 22 3.141674265021758 23 3.141829681889202 24 3.142451272494134 25 3.142451272494134 26 3.162277660168380 27 3.162277660168380 28 3.464101615137754 29 4.000000000000000 30 0.000000000000000 31 0.000000000000000
2 2.828427124746190 3 3.061467458920719 4 3.121445152258053 5 3.136548490545940 6 3.140331156954753 7 3.141277250932773 8 3.141513801144301 9 3.141572940367091 10 3.141587725277160 11 3.141591421511200 12 3.141592345570118 13 3.141592576584872 14 3.141592634338563 15 3.141592648776985 16 3.141592652386591 17 3.141592653288992 18 3.141592653514593 19 3.141592653570993 20 3.141592653585093 21 3.141592653588618 22 3.141592653589499 23 3.141592653589719 24 3.141592653589774 25 3.141592653589788 26 3.141592653589792 27 3.141592653589793 28 3.141592653589793 29 3.141592653589793 30 3.141592653589793 31 3.141592653589793
π correct to all digits
![Page 17: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/17.jpg)
Boring…
Is that all there isto numerical analysis?
![Page 18: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/18.jpg)
Not so boring if the result of this
computation affects
![Page 19: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/19.jpg)
Not so boring if the result of this
computation affects• The ability of the next plane you fly to stay in the air
![Page 20: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/20.jpg)
Not so boring if the result of this
computation affects• The ability of the next plane you fly to stay in the air
• The integrity of the next bridge you cross
![Page 21: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/21.jpg)
Not so boring if the result of this
computation affects• The ability of the next plane you fly to stay in the air
• The integrity of the next bridge you cross
• The state of the economy on which you live
![Page 22: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/22.jpg)
Not so boring if the result of this
computation affects• The ability of the next plane you fly to stay in the air
• The integrity of the next bridge you cross
• The state of the economy on which you live
• The path of a missile that isn’t intended to strike you
![Page 23: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/23.jpg)
So what are the common problems of numerical
analysis?
![Page 24: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/24.jpg)
So what are the common problems of numerical
analysis?Application areas:
![Page 25: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/25.jpg)
So what are the common problems of numerical
analysis?Application areas:
• Petroleum modeling
![Page 26: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/26.jpg)
So what are the common problems of numerical
analysis?Application areas:
• Petroleum modeling• Atomic energy – including weapons
![Page 27: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/27.jpg)
So what are the common problems of numerical
analysis?Application areas:
• Petroleum modeling• Atomic energy – including weapons • Weather modeling
![Page 28: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/28.jpg)
So what are the common problems of numerical
analysis?Application areas:
• Petroleum modeling• Atomic energy – including weapons • Weather modeling • Other modeling such as aircraft and automobile
![Page 29: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/29.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
![Page 30: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/30.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
• Linear Equations
![Page 31: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/31.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
• Linear Equations • Nonlinear equations - single and systems
![Page 32: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/32.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
• Linear Equations • Nonlinear equations - single and systems• Optimization
![Page 33: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/33.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
• Linear Equations • Nonlinear equations - single and systems• Optimization • Data Fitting - interpolation and approximation
![Page 34: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/34.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
• Linear Equations • Nonlinear equations - single and systems• Optimization • Data Fitting - interpolation and approximation• Integration
![Page 35: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/35.jpg)
So what are the common problems of numerical
analysis?Algorithm areas:
• Linear Equations • Nonlinear equations - single and systems• Optimization • Data Fitting - interpolation and approximation• Integration • Differential Equations - ordinary and partial
![Page 36: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/36.jpg)
Didn’t we study that stuff in math
classes?
![Page 37: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/37.jpg)
Didn’t we study that stuff in math
classes?Yes, but as the Pi Example shows, math classes are
just the beginning
![Page 38: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/38.jpg)
Let’s get back to history
![Page 39: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/39.jpg)
Why were computers used primarily for numerical problems
initially?
![Page 40: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/40.jpg)
Why were computers used primarily for numerical problems
initially?
• Mathematicians and engineers designed them
![Page 41: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/41.jpg)
Why were computers used primarily for numerical problems
initially?
• Mathematicians and engineers designed them• A history of algorithms in that area
![Page 42: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/42.jpg)
Why were computers used primarily for numerical problems
initially?
• Mathematicians and engineers designed them• A history of algorithms in that area• Immediate war-time and post-war-time applications
![Page 43: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/43.jpg)
Why were computers used primarily for numerical problems
initially?
• Mathematicians and engineers designed them• A history of algorithms in that area• Immediate war-time and post-war-time applications• Applications did not depend upon having a large number of computers
![Page 44: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/44.jpg)
Why were computers used primarily for numerical problems
initially?
• Mathematicians and engineers designed them• A history of algorithms in that area• Immediate war-time and post-war-time applications• Applications did not depend upon having a large number of computers• However, there were non-numerical examples ENIGMA
![Page 45: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/45.jpg)
What were the major computing ideas that arose in
numerical analysis?
![Page 46: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/46.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
![Page 47: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/47.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
input output
![Page 48: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/48.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
input output
true operation
![Page 49: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/49.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
input output
true operation
approximate operation
![Page 50: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/50.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
input output
true operation
approximate operation
error
![Page 51: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/51.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
input output
true operation
approximate operation
![Page 52: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/52.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis
input output
true operation
approximate operation
backward error
![Page 53: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/53.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN
![Page 54: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/54.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN
•Mathematical based•Computationally Efficient•Portable•Standardized – 3 times
![Page 55: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/55.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages
![Page 56: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/56.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages•IMSL
•Eispack •Linpack
![Page 57: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/57.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages • NANET
![Page 58: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/58.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages • NANET•Weekly information about people, problems, and papers•Software repository
![Page 59: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/59.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages • NANET• Matlab, Mathematica
![Page 60: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/60.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages • NANET• Matlab, MathematicaScientific computing environments
![Page 61: A History of Numerical Analysis Ideas](https://reader038.vdocuments.site/reader038/viewer/2022102517/56812bad550346895d8fdd71/html5/thumbnails/61.jpg)
What were the major computing ideas that arose in
numerical analysis?
• Backward error analysis • FORTRAN • Mathematical software packages • NANET• Matlab, Mathematica•Super computers - Parallelism