ch17 curve fitting
TRANSCRIPT
-
8/12/2019 Ch17 Curve Fitting
1/44
Motilal Nehru National Institute of Technology
Civil Engineering Department
Computer Based Numerical Techniques
CE-401
Least Square Regression
-
8/12/2019 Ch17 Curve Fitting
2/44
CURVE FITTING
There are two general approaches for curve fitting: Least Squares regression:
Data exhibit a significant degree of scatter. The strategy isto derive a single curve that represents the general trend
of the data. Interpolation:
Data is very precise. The strategy is to pass a curve or aseries of curves through each of the points.
-
8/12/2019 Ch17 Curve Fitting
3/44
Introduction
In engineering, two types of applications areencountered:
Trend analysis . Predicting values of dependentvariable, may include extrapolation beyond datapoints or interpolation between data points.
Hypothesis testing . Comparing existingmathematical model with measured data.
-
8/12/2019 Ch17 Curve Fitting
4/44
-
8/12/2019 Ch17 Curve Fitting
5/44
Mathematical Background Arithmetic mean . The sum of the individual data
points (yi) divided by the number of points (n).
Standard deviation . The most common measure of aspread for a sample.
ni
n
y y i ,,1,
2)(,1
y yS n
S S it
t y
-
8/12/2019 Ch17 Curve Fitting
6/44
Mathematical Background (contd)
Variance . Representation of spread by the square ofthe standard deviation.
or
Coefficient of variation . Has the utility to quantify the
spread of data.
1 /22
2 n n y yS ii
y
1
)( 22 n
y yS i y
%100.. y
S vc y
-
8/12/2019 Ch17 Curve Fitting
7/44
Least Squares RegressionChapter 17
Linear RegressionFitting a straight line to a set of pairedobservations: (x 1, y 1 ), (x 2 , y 2 ),,(x n, y n ).y = a 0 + a 1 x + ea 1 - slopea 0 - intercepte - error, or residual, between the model andthe observations
-
8/12/2019 Ch17 Curve Fitting
8/44
Linear Regression: Residual
-
8/12/2019 Ch17 Curve Fitting
9/44
Linear Regression: Question
How to find a 0 and a 1 so that the error would beminimum?
-
8/12/2019 Ch17 Curve Fitting
10/44
Linear Regression: Criteria for a Best Fit
n
iii
n
ii xaa ye
110
1
)(min
e1 e2
e1= -e 2
-
8/12/2019 Ch17 Curve Fitting
11/44
Linear Regression: Criteria for a Best Fit
n
iii
n
ii xaa ye
110
1
||||min
-
8/12/2019 Ch17 Curve Fitting
12/44
Linear Regression: Criteria for a Best Fit
||||max min 10n
1i iii xaa ye
-
8/12/2019 Ch17 Curve Fitting
13/44
Linear Regression: Least Squares Fit
n
iii
n
iir xaa yeS
1
210
1
2 )(min
n
i
n
iiiii
n
iir xaa y y yeS
1 1
210
2
1
2 )()model,measured,(
Yields a unique line for a given set of data.
-
8/12/2019 Ch17 Curve Fitting
14/44
Linear Regression: Least Squares Fit
n
iii
n
iir xaa yeS
1
210
1
2 )(min
The coefficients a 0 and a 1 that minimize S r must satisfythe following conditions:
0
0
1
0
aS
aS
r
r
-
8/12/2019 Ch17 Curve Fitting
15/44
210
10
11
1
0
0
0)(2
0)(2
iiii
ii
iioi
r
ioio
r
xa xa x y
xaa y
x xaa ya
S
xaa yaS
Linear Regression:Determination of a o and a 1
210
10
00
iiii
ii
xa xa x y
ya xna
naa
2 equations with 2unknowns, can be solvedsimultaneously
-
8/12/2019 Ch17 Curve Fitting
16/44
Linear Regression:Determination of ao and a1
221
ii
iiii
x xn
y x y xna
xa ya 10
-
8/12/2019 Ch17 Curve Fitting
17/44
-
8/12/2019 Ch17 Curve Fitting
18/44
18
-
8/12/2019 Ch17 Curve Fitting
19/44
19
-
8/12/2019 Ch17 Curve Fitting
20/44
Error Quantification of Linear Regression
Total sum of the squares around the mean forthe dependent variable, y, is St
Sum of the squares of residuals around the
regression line is Sr
2it y yS )(
2n
1ii1oi
n
1i
2ir xaa yeS )(
-
8/12/2019 Ch17 Curve Fitting
21/44
Error Quantification of Linear Regression
St -Sr quantifies the improvement or errorreduction due to describing data in terms of astraight line rather than as an average value.
t
r t
S S S
r 2
r 2: coefficient of determinationr : correlation coefficient
-
8/12/2019 Ch17 Curve Fitting
22/44
Error Quantification of Linear Regression
For a perfect fit: Sr= 0 and r = r 2 =1, signifying that the line
explains 100 percent of the variability of thedata.
For r = r 2 = 0, Sr = St, the fit represents no
improvement.
-
8/12/2019 Ch17 Curve Fitting
23/44
Least Squares Fit of a Straight Line:Example
Fit a straight line to the x and y values in thefollowing Table:
5.119 ii y x
28 i x 0.24 i y
1402 i x
4728
x
428571.37
24 y
xi yi xiyi xi2
1 0.5 0.5 1
2 2.5 5 4
3 2 6 9
4 4 16 165 3.5 17.5 25
6 6 36 36
7 5.5 38.5 49
28 24 119.5 140
-
8/12/2019 Ch17 Curve Fitting
24/44
Least Squares Fit of a Straight Line: Example(contd)
07142857.048392857.0428571.3
8392857.0281407
24285.1197
)(
10
2
221
xa ya
x xn
y x y xna
ii
iiii
Y = 0.07142857 + 0.8392857 x
-
8/12/2019 Ch17 Curve Fitting
25/44
Least Squares Fit of a Straight Line: Example(Error Analysis)
9911.22
ir eS
932.0868.02 r r
xi y i 1 0.52 2.5
3 2.04 4.05 3.56 6.07 5.5
8.5765 0.16870.8622 0.5625
2.0408 0.34730.3265 0.32650.0051 0.58966.6122 0.79724.2908 0.1993
2^
22 )( y y e ) y(y iii
28 24.0 22.7143 2.9911
868.02
t
r t
S S S
r
7143.222 y yS it
-
8/12/2019 Ch17 Curve Fitting
26/44
Least Squares Fit of a Straight Line:Example (Error Analysis)
9457.117
7143.221n
S s t y
7735.0
27
9911.2
2/
n
S s r x y
y x y S S /
The standard deviation (quantifies the spread around the mean):
The standard error of estimate (quantifies the spread around theregression line)
Because , the linear regression model has good fitness
-
8/12/2019 Ch17 Curve Fitting
27/44
Algorithm for linear regression
-
8/12/2019 Ch17 Curve Fitting
28/44
Linearization of Nonlinear Relationships
The relationship between the dependent andindependent variables is linear. However, a few types of nonlinear functions
can be transformed into linear regressionproblems.The exponential equation.The power equation.The saturation-growth-rate equation.
-
8/12/2019 Ch17 Curve Fitting
29/44
-
8/12/2019 Ch17 Curve Fitting
30/44
Linearization of Nonlinear Relationships1. The exponential equation.
xbea y 11
xba y 11lnln
y* = a o + a 1 x
-
8/12/2019 Ch17 Curve Fitting
31/44
Linearization of Nonlinear Relationships2. The power equation
22
b xa y
xba y logloglog 22
y* = a o + a 1 x*
-
8/12/2019 Ch17 Curve Fitting
32/44
Linearization of Nonlinear Relationships3. The saturation-growth-rate equation
xb x
a y3
3
xa
ba y
111
3
3
3
y* = 1/yao = 1/a 3a1 = b3/a 3x* = 1/x
-
8/12/2019 Ch17 Curve Fitting
33/44
ExampleFit the following Equation:
22
b xa y
to the data in the following table:
xi y i 1 0.52 1.73 3.44 5.75 8.415 19.7
X*=log x i Y*=logy i 0 -0.3010.301 0.2260.477 0.5340.602 0.7530.699 0.9222.079 2.141
)log(log 22
b xa y
2120
**
log
logloglet
b , aaa
x, y, X Y xba y logloglog 22
*10* X aaY
-
8/12/2019 Ch17 Curve Fitting
34/44
Example
Xi Yi X*i=Log(X) Y*i=Log(Y) X*Y* X*^2
1 0.5 0.0000 -0.3010 0.0000 0.0000
2 1.7 0.3010 0.2304 0.0694 0.0906
3 3.4 0.4771 0.5315 0.2536 0.2276
4 5.7 0.6021 0.7559 0.4551 0.3625
5 8.4 0.6990 0.9243 0.6460 0.4886
Sum 15 19.700 2.079 2.141 1.424 1.169
1 2 22
0 1
5 1.424 2.079 2.141 1.755 1.169 2.079( )
0.4282 1.75 0.41584 0.334
i i i i
i i
n x y x yan x x
a y a x
-
8/12/2019 Ch17 Curve Fitting
35/44
Linearization of Nonlinear
Functions: Example
log y=-0.334+1.75log x
1.750.46 y x
-
8/12/2019 Ch17 Curve Fitting
36/44
Polynomial Regression
Some engineering data is poorly representedby a straight line.
For these cases a curve is better suited to fitthe data.
The least squares method can readily beextended to fit the data to higher orderpolynomials.
-
8/12/2019 Ch17 Curve Fitting
37/44
Polynomial Regression (contd)
A parabola is preferable
-
8/12/2019 Ch17 Curve Fitting
38/44
Polynomial Regression (contd)
A 2nd order polynomial (quadratic) is defined by:
The residuals between the model and the data:
The sum of squares of the residual:
e xa xaa y o 2
21
221 iioii xa xaa ye
22212 iioiir xa xaa yeS
-
8/12/2019 Ch17 Curve Fitting
39/44
Polynomial Regression (contd)
0 x xa xaa y2aS
0 x xa xaa y2aS
0 xa xaa y2aS
2i
2i2i1oi
2
r
i2i2i1oi
1
r
2i2i1oi
o
r
)(
)(
)(
4i2
3i1
2ioi
2i
3i2
2i1ioii
2i2i1oi
xa xa xa y x
xa xa xa y x
xa xaan y 3 linear equationswith 3 unknowns(a o,a 1,a 2), can besolved
-
8/12/2019 Ch17 Curve Fitting
40/44
Polynomial Regression (contd)
A system of 3x3 equations needs to be solved to determinethe coefficients of the polynomial.
The standard error & the coefficient of determination
3/ nS
s r x yt
r t
S S S
r 2
ii
ii
i
iii
iii
ii
y x y x
y
aa
a
x x x x x x
x xn
22
1
0
432
32
2
-
8/12/2019 Ch17 Curve Fitting
41/44
Polynomial Regression (contd)
General:The mth-order polynomial:
A system of (m+1)x(m+1) linear equations must be solved fordetermining the coefficients of the mth-order polynomial.
The standard error:
The coefficient of determination:
e xa xa xaa y mmo .....2
21
1/ mn S s r x y
t
r t
S S S
r 2
-
8/12/2019 Ch17 Curve Fitting
42/44
Polynomial Regression- Example
Fit a second order polynomial to data:
2253 i x
9794 i x
x i y i x i 2 x i
3 x i 4 x i y i x i
2 y i0 2.1 0 0 0 0 0
1 7.7 1 1 1 7.7 7.7
2 13.6 4 8 16 27.2 54.4
3 27.2 9 27 81 81.6 244.8
4 40.9 16 64 256 163.6 654.4
5 61.1 25 125 625 305.5 1527.5
15 152.6 55 225 979 585.6 2489
6.585 ii y x
15 i x
6.152 i y
552
i x
43.256
6.152 ,5.2
615
y x 8.24882 ii y x
-
8/12/2019 Ch17 Curve Fitting
43/44
Polynomial Regression- Example (contd)
The system of simultaneous linear equations:
2
210
86071.135929.247857.2
86071.1,35929.2,47857.2
x x y
aaa
8.2488
6.585
6.152
97922555
2255515
55156
2
1
0
a
a
a
74657.32
ir eS 39.25132 y yS it
-
8/12/2019 Ch17 Curve Fitting
44/44
Polynomial Regression- Example (contd) x i y i y model e i
2 (y i -y`) 2
0 2.1 2.4786 0.14332 544.42889
1 7.7 6.6986 1.00286 314.45929
2 13.6 14.64 1.08158 140.01989
3 27.2 26.303 0.80491 3.12229
4 40.9 41.687 0.61951 239.22809
5 61.1 60.793 0.09439 1272.13489
15 152.6 3.74657 2513.39333
The standard error of estimate:
The coefficient of determination:
12.136
74657.3/ x y s
99925.0,99851.074657.339.2513 22 r r r