![Page 1: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/1.jpg)
Statistical Analysis related to the Inverse problem
Se Hee Kim and Lingsong Zhang
SAMSI/CRSC Undergraduate Workshop
![Page 2: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/2.jpg)
5/25/2006 SAMSI/CRSC 2006 2
Review of the Inverse problem
• What we get– Displacement yi and time ti
– Spring model
• Target:Estimate C and K based on the observed yi
![Page 3: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/3.jpg)
5/25/2006 SAMSI/CRSC 2006 3
• Estimation procedure– Minimize the cost function
– Guess initial values of C and K– Using optimization method and differential
equations to find the values of C and K which minimize the above cost function.
– inv_beam.m
Review of the Inverse Problem (cont.)
![Page 4: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/4.jpg)
5/25/2006 SAMSI/CRSC 2006 4
Underlying Statistical Models
• yi has measurement error
– The above model can be viewed as a regression model
where are iid (independent identically distributed) from N(0, 2).
– Estimating C and K leads to the procedure mentioned earlier (will show later)
![Page 5: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/5.jpg)
5/25/2006 SAMSI/CRSC 2006 5
Nonlinear regression
• Linear– Linear is for the parameter(s)
• Nonliner– y(ti, C, K ) are defined by an ODE, it is not from a
simple linear function of C and K– A regression model is called nonlinear, if the
derivatives of the model with respect to the model parameters depend on one or more parameters
![Page 6: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/6.jpg)
5/25/2006 SAMSI/CRSC 2006 6
Nonlinear regression
• A regression model is not necessarily nonlinear if the graphed regression trend is curved– Example: – Take derivatives of y with respect to the
parameters 0, 1 and 2:
– None of these derivatives depend on a model
parameters, thus the model is linear.
![Page 7: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/7.jpg)
5/25/2006 SAMSI/CRSC 2006 7
Nonlinear regression
• The general form of a nonlinear regression model is
• Where x is a vector of explanatory variables, is a vector of unknown parameters and is a N(0, 2) error term
• To estimate unknown parameters,
![Page 8: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/8.jpg)
5/25/2006 SAMSI/CRSC 2006 8
Statistical problems
• How to evaluate the estimation of C and K?– Estimation of .
– Variation of the and .
• Are those assumptions correct?– Measurement errors are truly from iid Normal
distribution?
– Are there better models?
![Page 9: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/9.jpg)
5/25/2006 SAMSI/CRSC 2006 9
Estimation of
• If all model assumptions hold– we have
Use these to estimate
Our data set gives
YOURS? estimateofsigma.m
![Page 10: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/10.jpg)
5/25/2006 SAMSI/CRSC 2006 10
Evaluate the Estimation of C and K
• Method 1- repeating experiments– Independent experiments under the same
conditions• How many experiments required?
we will use 8 sets of data to illustrate this method
– Use simple univariate statistics of C’s and K’s to evaluate the performance of estimation.
![Page 11: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/11.jpg)
5/25/2006 SAMSI/CRSC 2006 11
Examples
• 8 sets of data from the same conditions– Get the estimations of C
and K
– Variations of C and K
sd(C)= .2675
sd(K)=4.3437
C K
0.89710
1.04680
0.90619
0.89321
1.63440
0.86776
0.78324
1.03730
1526.9
1526.8
1528.3
1530.2
1518.7
1527.0
1531.4
1533.0
1.00820 1527.8
![Page 12: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/12.jpg)
5/25/2006 SAMSI/CRSC 2006 12
![Page 13: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/13.jpg)
5/25/2006 SAMSI/CRSC 2006 13
![Page 14: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/14.jpg)
5/25/2006 SAMSI/CRSC 2006 14
Examples
• 8 sets of data from the same conditions– Get the estimations of C
and K
– Variations of C and K
sd(C)= .2675
sd(K)=4.3437
After removing the “outlier”
sd(C)=.0937, sd(K)=2.4828
C K
0.89710
1.04680
0.90619
0.89321
1.63440
0.86776
0.78324
1.03730
1526.9
1526.8
1528.3
1530.2
1518.7
1527.0
1531.4
1533.0
1.00820 1527.8
![Page 15: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/15.jpg)
5/25/2006 SAMSI/CRSC 2006 15
Exercise
• To get the standard deviations from your 10 data sets (std)– If you do not have the record of estimations,
• Use inv_beam_all.m to get Cvec, Kvec,
(it will take a long time, do it during the break)
– If you have the record of the estimations– Input them into Cvec and Kvec,
• Bar-chart?– Try barCvec.m and barKvec.m (need to adjust some
numbers)
![Page 16: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/16.jpg)
5/25/2006 SAMSI/CRSC 2006 16
• Method 2 - Using the nonlinear regression model to evaluate the estimation of C and K.– How to get the standard deviation (or variance)
of and ?
Use the covariance matrix of
Evaluate the Estimation (cont.)
![Page 17: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/17.jpg)
5/25/2006 SAMSI/CRSC 2006 17
Recall: Linear Regression
• Simple Linear Model:– The model
we estimate the covariance matrix of 0 and 1
using
where
![Page 18: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/18.jpg)
5/25/2006 SAMSI/CRSC 2006 18
Apply to the inverse problem
Our model
we will have similar result
where
the standard errors of and are square roots of the diagonal elements of
![Page 19: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/19.jpg)
5/25/2006 SAMSI/CRSC 2006 19
Checking the Assumptions
• Check whether the residuals are iid Normal noise– Independent?
• Residual plot vs Time
– Variance are constant?• Residual plot vs. fitted values
– Residuals are Normally distributed?• Normal Quantile-Quantile plot
![Page 20: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/20.jpg)
5/25/2006 SAMSI/CRSC 2006 20
Checking dependency
![Page 21: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/21.jpg)
5/25/2006 SAMSI/CRSC 2006 21
Checking Constant Variance
![Page 22: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/22.jpg)
5/25/2006 SAMSI/CRSC 2006 22
Checking Normality
![Page 23: Statistical Analysis related to the Inverse problem](https://reader035.vdocuments.site/reader035/viewer/2022062423/568144b2550346895db17b7f/html5/thumbnails/23.jpg)
5/25/2006 SAMSI/CRSC 2006 23
Other models?
• Is the spring model appropriate for our data?– Residual shows dependent structure– It seems that variances are not equal– Normal assumption might not hold
• Other statistical inference method?– Same underlying model, but different assumptions– Other statistical models to fit the data
• Some alternative physical models for our data?