che 555 curve fitting

by Lale Yurttas, Texas A&M University

Chapter 4 1

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Chapter 4CURVE FITTING


Chapter 4 2

CURVE FITTING• Describes techniques to fit curves (curve fitting) to

discrete data to obtain intermediate estimates.

• There are two general approaches for curve fitting:– Data exhibit a significant degree of scatter. The strategy is to

derive a single curve that represents the general trend of the data.

– Data is very precise. The strategy is to pass a curve or a series of curves through each of the points.

• In engineering two types of applications are encountered:– Trend analysis. Predicting values of dependent variable, may

include extrapolation beyond data points or interpolation between data points.

– Hypothesis testing. Comparing existing mathematical model with measured data.


Chapter 4 3

• Arithmetic mean. The sum of the individual data points (yi) divided by the number of points (n).

• Standard deviation. The most common measure of a spread for a sample.

or

nin

yy i

,,1

2)(1

yySnSS

it

ty

1/22

2

n

nyyS ii

y

MATHEMATICAL BACKGROUND


Chapter 4 4

• Variance. Representation of spread by the square of the standard deviation.

• Coefficient of variation. Has the utility to quantify the spread of data.

1)( 2

2

n

yyS i

y Degrees of freedom

%100..y

Svc y


Chapter 4 5

LINEAR REGRESSION• Fitting a straight line to a set of paired

observations: (x1, y1), (x2, y2),…,(xn, yn).

where: a1- slopea0- intercepte- error, or residual, between

the model and the observations

xaya

xxn

yxyxna

ii

iiii

10

221

Mean values

exaay 10


Chapter 4 6

Criteria for a “Best” Fit• Minimize the sum of the residual errors for

all available data:

n = total number of points• However, this is an inadequate criterion.• Minimize the sum of the absolute values:

n

iioi

n

ii xaaye

11

1

)(

n

iii

n

ii xaaye

110

1


Chapter 4 7

a) Minimizes the sum of residuals

b) Minimizes the sum of absolute values of residuals

c) Minimizes the max error of any individual point


Chapter 4 8

• Best strategy is to minimize the sum of the squares of the residuals between the measured y and the y calculated with the linear model:

• Yields a unique line for a given set of data.

• Standard error of the estimate, Sy/x:

n

i

n

iiiii

n

iir xaayyyeS

1 1

210

2

1

2 )()model,measured,(

2/

nSS r

xy


Chapter 4 9

Figure 17.3


Chapter 4 10

a) Linear regression with small residual errors

b) Linear regression with large residual errors


11

Estimation of Errors• St-Sr quantifies the improvement or error reduction due to

describing data in terms of a straight line rather than as an average value.

• For a perfect fit:Sr=0 and r=r2=1, signifying that the line explains 100 percent of the variability of the data.

• For r=r2=0, Sr=St, the fit represents no improvement.

t

rt

SSSr

2r2 -coefficient of determination

r – correlation coefficient

Chapter 4

Example 1

Fit a straight line to the x and y values in table 1. Then compute the total standard deviation, standard error of the estimate and the correlation coefficient for the data


Chapter 4 12

xi yi

1 0.52 2.53 2.04 4.05 3.56 6.07 5.5

Table 1


Chapter 4 13

POLYNOMIAL REGRESSION

• Some engineering data is poorly represented by a straight line.

• For these cases a curve is better suited to fit the data.

• The least squares method can readily be extended to fit the data to higher order polynomials.

• For example: 2nd order polynomialexaxaay 2210

• Set of normal equations:

• In matrix form:


Chapter 4 14

24

13

022

23

12

0

22

10

axaxaxxy

axaxaxxy

axaxnay

iii

iiiii

iii

ii

i

ii

i

i

i

yxyx

y

aaa

xxxxxxxxn

iiii

ii

i

22

1

0

432

32

2

• Standard error:

Where: m = m-th order polynomial


Chapter 4 15

)1(

)(

)1(1

22210

/

mn

xaxaay

mnSS

n

iiii

rxy

Example 2Fit a second-order polynomial to the x and y values in table 2. Then compute the total standard deviation, standard error of the estimate and the correlation coefficient for the data.


Chapter 4 16

xi yi

0 2.11 7.72 13.63 27.24 40.95 61.1

Table 2


Chapter 4 17

MULTIPLE LINEAR REGRESSION

• Useful extension of linear regression in case where y is a linear function of 2 or more independent variables.

• For example: exaxaay 22110

• Set of normal equations:

• In matrix form:


Chapter 4 18

222121022

221121011

22110

axaxxaxxy

axxaxaxxy

axaxnay

iiiiii

iiiiii

iii

i

ii

i

iiii

iiii

ii

yxyx

y

aaa

xxxxxxxx

xxn

i2

1

2

1

0

22212

21211

21

• Standard error:

Where: m = dimension


Chapter 4 19

)1(

)(

)1(1

222110

/

mn

xaxaay

mnSS

n

iiii

rxy

Example 3

The data from table 3 was calculated from the equationy = 5 + 4x1 – 3x2. Use multiple linear regression to fit this data.


Chapter 4 20

x1 x2 y0 0 52 1 10

2.5 2 91 3 04 6 37 2 27

Table 3


Chapter 4 21

GENERAL LINEAR LEAST SQUARES

EAZY

• General model:

• Express in matrix form:functions basis 1 are 10

221100

m, z, , zzezazazazay

m

mm

residualsE

tscoefficienunknown Avariabledependent theof valuedobservedY

t variableindependen theof valuesmeasured at the functions basis theof valuescalculated theofmatrix

Z

• Outcome / Solution to this model:

a)Equivalent to the normal equation developed previously

b)3 solution technique: (i) Gauss Elimination; (ii) Cholesky’s ; (iii) matrix inverse

• Sum squares of the residuals:


Chapter 4 22

2

1 0

n

i

m

jjijir zayS

Minimized by taking its partial derivative with respect to each of the coefficients and setting the resulting equation equal to zero

YZAZZ TT

NONLINEAR REGRESSION

• In engineering many cases where nonlinear model must be fit to data.

• General equation:

• For example:

• Gauss Newton method – algorithm for minimizing the sum of squares of residual between data and nonlinear equations.


Chapter 4 23

eeaxf xa )1()( 10

imii eaaaxfy ),...,,;( 10

Procedures1. Linearized the original model with respect

to the parameter (e.g. : 2 parameter)

In matrix form:


Chapter 4 24

ijiji

jii eaaxf

aaxf

xfy

11

00

)()()(

EAZD j ][

Where:


Chapter 4 25

nnn

nn

j

a

aa

A

xfy

xfyxfy

D

afaf

afafafaf

Z

.

.

.;

)(...

)()(

//......////

][

1

0

22

11

10

1202

1101

2. Applying linear least square theory; solve {∆A}

3. Improve values of parameters:

4. Repeated until the solution converges – falls below an acceptable stopping criterion.


Chapter 4 26

DZAZZ TT

1,11,10,01,0 ; aaaaaa jjjj

%1001,

,1,

jk

jkjkka a

aa

• Sum of squares of the residuals


Chapter 4 27

2

1

)(

n

iiir xfyS

Example 4Fit the function

to data in table 4. Use initial guesses of a0 = 1.0 and a1 = 1.0 for the parameters.


Chapter 4 28

xi yi

0.25 0.280.75 0.571.25 0.681.75 0.742.25 0.79

Table 4eeaxf xa )1()( 1

0

che 555 curve fitting

Documents