![Page 1: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/1.jpg)
Adavanced Numerical Computation 2008, AM NDHU
1
•Function approximation•Linear models•Line fitting•Hyper-plane fitting
•Discriminate analysis
Least Square Method
![Page 2: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/2.jpg)
Advanced Numerical Computation 2008, AM NDHU
2
Unconstrained optimization
A target function, y = g(x), where x Rd
A sample from the surface of f paired data S={(xi ,yi)}i
yi =g(xi )+i
Let G(x; ) be an approximating function to g collects built-in parameters Minimizing the mean square approximating
error induces an unconstrained optimization
![Page 3: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/3.jpg)
Adavanced Numerical Computation 2008, AM NDHU
3
Data driven function approximation
![Page 4: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/4.jpg)
Adavanced Numerical Computation 2008, AM NDHU
4
Data driven function approximation
![Page 5: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/5.jpg)
Advanced Numerical Computation 2008, AM NDHU
5
Mean square approximating error
)(minarg
));(()( 2
Sopt
iiiS
E
xGyE
![Page 6: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/6.jpg)
Advanced Numerical Computation 2008, AM NDHU
6
Nonlinear System
i k
ii
i k
ii
kk
Tn
i
ii
i
ii
i
iii
iiiS
d
xGdxG
d
xdGy
d
dEf
ffF
d
xGdxG
d
xdGy
d
xdGxGy
d
dE
xGyE
);();(
);()(
0)](),...,([)(
);();(
);(
);());((
));((2
1)(
1
2
![Page 7: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/7.jpg)
Advanced Numerical Computation 2008, AM NDHU
7
Minima
dE/d = 0Local minima : unsatisfied approximationGlobal minima : reliable and effective
approximation
![Page 8: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/8.jpg)
Adavanced Numerical Computation 2008, AM NDHU
8
Nonlinear system
The severe local minimum problem needs to be overcome
0)(
)(
d
dEF S
![Page 9: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/9.jpg)
Adavanced Numerical Computation 2008, AM NDHU
9
Approximating function
DimensionalityOne dimensional functionsHigh dimensional functionsExtremely high dimensional functions
Linear functions, quadratic functions and nonlinear functions
![Page 10: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/10.jpg)
Adavanced Numerical Computation 2008, AM NDHU
10
Line fitting
Given paired data, (xi, yi ), minimize
n
iii yaxaaa
1
20101 )(),E(
one dimensional linear function
![Page 11: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/11.jpg)
Adavanced Numerical Computation 2008, AM NDHU
11
Paired data
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 11.8
2
2.2
2.4
2.6
2.8
3
3.2
3.4
3.6
n=100; x=rand(1, n);y=1.5*x+2+rand(1, n)*0.1-0.05;plot(x,y,'.')
2
5.1
0
1
a
a
![Page 12: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/12.jpg)
Adavanced Numerical Computation 2008, AM NDHU
12
Fitting criteria
n1,...,ifor ,01 axay ii
n1,...,ifor ,01 ii yaax
![Page 13: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/13.jpg)
Adavanced Numerical Computation 2008, AM NDHU
13
Pseudo Inverse
n1,...,ifor ,01 ii yaax
nn y
y
y
a
a
x
x
x
2
1
0
12
1
1
1
1
1
![Page 14: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/14.jpg)
Adavanced Numerical Computation 2008, AM NDHU
14
Form X and b
X=[x' ones(n,1)];b=y';
![Page 15: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/15.jpg)
Adavanced Numerical Computation 2008, AM NDHU
15
Line fitting
ba TT X)XX( 1
>> a=inv(X'*X)*X'*b
a =
1.4816 2.0113
![Page 16: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/16.jpg)
Adavanced Numerical Computation 2008, AM NDHU
16
Strategy I: Line fitting
ba )X(pinv
>> a=pinv(X)*b
a =
1.4816 2.0113
Matlab built-in function
![Page 17: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/17.jpg)
Adavanced Numerical Computation 2008, AM NDHU
17
Demo_line_fitting
demo_line_fitting.m
![Page 18: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/18.jpg)
Adavanced Numerical Computation 2008, AM NDHU
18
Stand alone executable file
mcc -m demo_line_fitting.m
demo_line_fitting.exedemo_line_fitting.ctf
![Page 19: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/19.jpg)
Adavanced Numerical Computation 2008, AM NDHU
19
Strategy II
Conjugate gradient method
bTT XXX
![Page 20: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/20.jpg)
Adavanced Numerical Computation 2008, AM NDHU
20
Comparison
pinv versus conjugate gradient methodd=1,n=100
d=1;n=100; x=rand(d, n); y=rand(1,d)*x+2+rand(1, n)*0.1-0.05; plot(x,y,'.'); X=[x' ones(n,1)]; b=y'; tstart = tic; a=pinv(X)*b telapsed = toc(tstart) tstart = tic; x0=rand(1,d+1)'; a2 = conjugate(X'*X,X'*b,x0) telapsed = toc(tstart)
![Page 21: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/21.jpg)
Adavanced Numerical Computation 2008, AM NDHU
21
Hyper-plane fitting
bXa
nmnnn
m
m
m
mn
xxxx
xxxx
xxxx
xxxx
321
3333231
2232221
1131211
X
ma
a
a
a
3
2
1
a
nb
b
b
b
3
2
1
b
![Page 22: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/22.jpg)
Adavanced Numerical Computation 2008, AM NDHU
22
Sampling
A sample from a hyper-plane or a cloud
A mapping from R2 to RPaired data:
iiiii sss )},,({ 321s
iiiii Ryxx }),,(|)y,{( 21ii xx
plane-hyper a from sampling as
mappinglinear of data pairedConsider
),,(
),,(
321
21
iii
iiii
sss
yxx
s
![Page 23: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/23.jpg)
Adavanced Numerical Computation 2008, AM NDHU
23
n=100, d=2General coordinate of points (s1,s2,s3) or
(x1,x2,y)Linea relation
s1a1+s2a2+a3=s3
equivalently
x1a1+x2a2+a3=y
![Page 24: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/24.jpg)
Adavanced Numerical Computation 2008, AM NDHU
24
d < n
The number of unknowns is less than the constraint numberOne hundred data points in R3 space
Data point on a hyper-planesi1a1+si2a2 +a3 - si3 =0
xi1a1+xi2a2 +a3 - yi =
Minimization of the mean square error ke
2e
![Page 25: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/25.jpg)
Adavanced Numerical Computation 2008, AM NDHU
baX
n1,...,ifor bi axTi
nmnnn
m
m
m
mn
xxxx
xxxx
xxxx
xxxx
321
3333231
2232221
1131211
X
ma
a
a
a
3
2
1
a
nn y
y
y
y
b
b
b
b
3
2
1
3
2
1
b
m=d+1 and 1imx
![Page 26: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/26.jpg)
Adavanced Numerical Computation 2008, AM NDHU
26
Strategy I : Pseudo Inverse
baX
ba X)(pinv
![Page 27: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/27.jpg)
Adavanced Numerical Computation 2008, AM NDHU
27
Strategy II: minimizing mean square errors
n
ii
Ti
n
ii b
ne
n
eE
Minimize
1
2
1
2
2
)(11
)(
ax
a
![Page 28: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/28.jpg)
Adavanced Numerical Computation 2008, AM NDHU
28
Minimization
m1,...,jfor 0)(
jda
dE a
![Page 29: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/29.jpg)
Adavanced Numerical Computation 2008, AM NDHU
29
Derivative
0)(2)(
1
n
iiji
Ti
j
xbnda
dEax
a
n
ii
Ti b
nE
Minimize
1
2)(1
)( axa
![Page 30: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/30.jpg)
Adavanced Numerical Computation 2008, AM NDHU
30
Vector Form
0)(2)(
1
m
iii
Ti b
nd
dExax
a
a
n
ii
Ti b
nE
Minimize
1
2)(1
)( axa
![Page 31: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/31.jpg)
Adavanced Numerical Computation 2008, AM NDHU
31
Linear system: normal equations
n
iii
n
ii
Ti b
11
xaxx
0)(2)(
1
n
iii
Ti b
nd
dExax
a
a
![Page 32: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/32.jpg)
Adavanced Numerical Computation 2008, AM NDHU
32
n
iii
n
i
Tii
n
iii
n
ii
Ti
b
b
11
11
xaxx
xaxx
![Page 33: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/33.jpg)
Adavanced Numerical Computation 2008, AM NDHU
33
Tn
T
T
x
x
x
2
1
X nT xxx 21X
![Page 34: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/34.jpg)
Adavanced Numerical Computation 2008, AM NDHU
34
Tj
jj
Tn
T
T
nT
xx
x
x
x
xxx
2
1
21XX
![Page 35: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/35.jpg)
Adavanced Numerical Computation 2008, AM NDHU
35
n
iii
n
i
Tii b
11
xaxx
bXXaXT T
![Page 36: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/36.jpg)
Adavanced Numerical Computation 2008, AM NDHU
36
bXXXa TT 1)(
bXXaXT T
![Page 37: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/37.jpg)
Adavanced Numerical Computation 2008, AM NDHU
37
Comparison
pinv versus conjugate gradient methodd=2,n=100
d=2;n=100; x=rand(d, n); y=rand(1,d)*x+2+rand(1, n)*0.1-0.05; plot(x,y,'.'); X=[x' ones(n,1)]; b=y'; tstart = tic; a=pinv(X)*b telapsed = toc(tstart) tstart = tic; x0=rand(1,d+1)'; a2 = conjugate(X'*X,X'*b,x0) telapsed = toc(tstart)
![Page 38: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/38.jpg)
Adavanced Numerical Computation 2008, AM NDHU
38
Hyper-plane fitting
Step 1. Input paired data, (xi , yi), i=1…n Step 2. Form matrix X and vector b Step 3. Set a to pinv(X)*b Step 4. Set c to bTT XXX 1)(
Hyper-plane fitting ax y
![Page 39: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/39.jpg)
Adavanced Numerical Computation 2008, AM NDHU
39
>> n=30;S=rand(n,2);y=S*[1 2]'+1;>> b=y;>> X=[S ones(n,1)];>> a=pinv(X)*b; c=inv(X'*X)*(X'*b);>> sum(abs(a-c))
ans =
1.0547e-015
![Page 40: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/40.jpg)
Adavanced Numerical Computation 2008, AM NDHU
40
Generate training dataGet S and y
Performance evaluation
Hyper-plane fitting
S,ya
Generate testing dataGet S_test and y_test
Generate y_hat
S_test
y_test
-
Error rate for testing
TRAINING TESTING
![Page 41: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/41.jpg)
Adavanced Numerical Computation 2008, AM NDHU
41
Generate y_hat
>> n=10;S_test=rand(n,2);y_test=S_test*[1 2]'+1;>> X_test = [S_test ones(n,1)];>> y_hat = X_test * a;
![Page 42: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/42.jpg)
Adavanced Numerical Computation 2008, AM NDHU
42
Testing error
>> error_rate= mean((y_test-y_hat).^2)
![Page 43: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/43.jpg)
Adavanced Numerical Computation 2008, AM NDHU
43
demo_hp_fitting>> demo_hp_fittinga1:1a2:2a3:3
a =
0.9959 2.0035 3.0141
![Page 44: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/44.jpg)
Adavanced Numerical Computation 2008, AM NDHU
44
HP Tool
![Page 45: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/45.jpg)
Adavanced Numerical Computation 2008, AM NDHU
45
HP Tool
MLP_Tool.mMLP_Tool.fig
![Page 46: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/46.jpg)
Adavanced Numerical Computation 2008, AM NDHU
46
Mesh
fstr=input('input a 2D function: x1.^2+x2.^2+cos(x1) :','s');fx=inline(fstr);range=2*pi;x1=-range:0.1:range;x2=x1;for i=1:length(x1) C(i,:)=fx(x1(i),x2);endmesh(x1,x2,C);
![Page 47: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/47.jpg)
Adavanced Numerical Computation 2008, AM NDHU
47
Nonlinear function approximation
![Page 48: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/48.jpg)
Adavanced Numerical Computation 2008, AM NDHU
48
Nonlinear function approximation
Target function & sample Unfaithful approximationby hyper-plane fitting
![Page 49: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/49.jpg)
Adavanced Numerical Computation 2008, AM NDHU
49
Linear projection
132121111 axaxay
132 211 xxy
![Page 50: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/50.jpg)
Adavanced Numerical Computation 2008, AM NDHU
50
Two linear projections
Add two linear projections
132)2,1( 211 xxyxxf
232)2,1( 212 xxyxxf
34)2,1( 121 xyyxxf
![Page 51: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/51.jpg)
Adavanced Numerical Computation 2008, AM NDHU
51
132)2,1( 211 xxyxxf 232)2,1( 212 xxyxxf
34)2,1( 1 xxxf
![Page 52: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/52.jpg)
Adavanced Numerical Computation 2008, AM NDHU
52
Post-nonlinear Projection
)tanh( 32211 axaxahy
tanh32211 axaxah y
![Page 53: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/53.jpg)
Adavanced Numerical Computation 2008, AM NDHU
53
)132tanh()2,1( 211 xxyxxf
)232tanh()2,1( 212 xxyxxf
![Page 54: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/54.jpg)
Adavanced Numerical Computation 2008, AM NDHU
54
Two post-nonlinear projections
)232tanh()132tanh()2,1( 2121 xxxxxxf
![Page 55: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/55.jpg)
Adavanced Numerical Computation 2008, AM NDHU
55
Data driven function approximation
![Page 56: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/56.jpg)
Adavanced Numerical Computation 2008, AM NDHU
56
Data driven function approximation
![Page 57: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/57.jpg)
Adavanced Numerical Computation 2008, AM NDHU
57
Linearly non-separable
• Classify blue and red dots to two categories• Linearly non-separable by hyper-plane fitting
![Page 58: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/58.jpg)
Adavanced Numerical Computation 2008, AM NDHU
58
Error rate
22.48 %
![Page 59: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/59.jpg)
Adavanced Numerical Computation 2008, AM NDHU
59
Classification
Discriminate analysisLinear discriminate analysis
win.rar178 paired data (s,y)s{R13} : predictor or features, the last 13 columny { 1,2,3} : three categories, the first column
![Page 60: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/60.jpg)
Adavanced Numerical Computation 2008, AM NDHU
60
Training & Testing data
win.dat
load win.datind=randperm(178)
win_train=win(ind(1:140),:);win_test=win(ind(141:178),:);
save win_train.mat win_train;save win_test.mat win_test;
win_train.matwin_test.mat
![Page 61: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/61.jpg)
Adavanced Numerical Computation 2008, AM NDHU
61
Load training dataGet S and y
Discriminate Analysis
Hyper-plane fitting
S,y
a
Load testing dataGet S_test and y_test
Generate y_hat
S_test
y_test
compare
Error rate for testing
![Page 62: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/62.jpg)
Adavanced Numerical Computation 2008, AM NDHU
62
![Page 63: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/63.jpg)
Adavanced Numerical Computation 2008, AM NDHU
63
Linear assumption
Predictor x=[x1,…,x13]T
y = a1*x1+a2*x2+…+a13*x13
Find a to
n
ii
Ti y
nE
Minimize
1
2)(1
)( axa
![Page 64: Adavanced Numerical Computation 2008, AM NDHU1 Function approximation Linear models Line fitting Hyper-plane fitting Discriminate analysis Least Square](https://reader035.vdocuments.site/reader035/viewer/2022070412/5697bf7c1a28abf838c8425f/html5/thumbnails/64.jpg)
Adavanced Numerical Computation 2008, AM NDHU
64
Demo_wine_fitting
Error Rate : 3.93%