![Page 1: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/1.jpg)
Regularized Least-Squares
Regularized Least-Squares
![Page 2: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/2.jpg)
Regularized Least-Squares
Outline
• Why regularization?• Truncated Singular Value Decomposition• Damped least-squares• Quadratic constraints
![Page 3: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/3.jpg)
Regularized Least-Squares
Why regularization?
• We have seen that
systemt independen an form of columns the
solution unique a has
A
Axbx
2
min
![Page 4: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/4.jpg)
Regularized Least-Squares
Why regularization?
• We have seen that
• But what happens if the system is almost dependent?– The solution becomes very sensitive to the data– Poor conditioning
systemt independen an form of columns the
solution unique a has
A
Axbx
2
min
![Page 5: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/5.jpg)
Regularized Least-Squares
The 1-dimensional case
• The 1-dimensional normal equation
baaa. TT x ˆ
![Page 6: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/6.jpg)
Regularized Least-Squares
The 1-dimensional case
• The 1-dimensional normal equation
aa
ba0 a
baaa.
T
T
TT
x
x
ˆ
ˆ
.if
![Page 7: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/7.jpg)
Regularized Least-Squares
The 1-dimensional case
• The 1-dimensional normal equation
troubleif
if
.
.
ˆ
ˆ
0 a
aa
ba0 a
baaa.
T
T
TT
x
x
![Page 8: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/8.jpg)
Regularized Least-Squares
Why regularization
• Contradiction between data and model
![Page 9: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/9.jpg)
Regularized Least-Squares
A more interesting example:scattered data interpolation
)( ipf
ip
![Page 10: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/10.jpg)
Regularized Least-Squares
“True” curve
![Page 11: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/11.jpg)
Regularized Least-Squares
Radial basis functions
![Page 12: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/12.jpg)
Regularized Least-Squares
Radial basis functions
i
n
ii p-pλpf
1
)(~
![Page 13: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/13.jpg)
Regularized Least-Squares
Rbf are popular
• Modeling– J. C. Carr, R. K. Beatson, J. B. Cherrie, T. J. Mitchell,W. R. Fright, B. C. McCallum,
and T. R. Evans. Reconstruction and representation of 3d objects with radial basis functions. In Proceedings of ACM SIGGRAPH 2001, Computer Graphics Proceedings, Annual Conference Series, pages 67–76, August 2001.
– G. Turk and J. F. O’Brien. Modelling with implicit surfaces that interpolate. ACM Transactions on Graphics, 21(4):855–873, October 2002.
• Animation– J. Noh and U. Neumann. Expression cloning. In Proceedings of ACMSIGGRAPH
2001, Computer Graphics Proceedings, Annual Conference Series, pages 277–288, August 2001.
– F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin. Synthesizing realistic facial expressions from photographs. In Proceedings of SIGGRAPH 98, Computer Graphics Proceedings, Annual Conference Series, pages 75–84, July 1998.
![Page 14: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/14.jpg)
Regularized Least-Squares
Radial basis functions• At every point )()(
~ii pfpf
![Page 15: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/15.jpg)
Regularized Least-Squares
Radial basis functions• At every point
• Solve the least-squares problem
)()(~
ii pfpf
2
1 1
2
1
)(min)(~
)(min
n
j
n
iiij
n
jjj ii
p-pλpfpfpf λλ
![Page 16: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/16.jpg)
Regularized Least-Squares
Radial basis functions• At every point
• Solve the least-squares problem
)()(~
ii pfpf
2
1 1
2
1
)(min)(~
)(min
n
j
n
iiij
n
jjj ii
p-pλpfpfpf λλ
nnnn
n
n λ
λ
pppp
pppp
pf
pf
:.
..
::
..
:1
1
1111
Axb
![Page 17: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/17.jpg)
Regularized Least-Squares
Rbf results
i
n
ii p-pλpf
1
)(~
![Page 18: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/18.jpg)
Regularized Least-Squares
pi0 close to
pi1
![Page 19: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/19.jpg)
Regularized Least-Squares
Radial basis functions• At every point
• Solve the least-squares problem
)()(~
ii pfpf
2
1 1
2
1
)(min)(~
)(min
n
j
n
iiij
n
jjj ii
p-pλpfpfpf λλ
nnnn
n
n λ
λ
pppp
pppp
pf
pf
:.
..
::
..
:1
1
1111
Axb
![Page 20: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/20.jpg)
Regularized Least-Squares
Radial basis functions• At every point
• Solve the least-squares problem
• If pi0 close to
pi1
, A is near singular
)()(~
ii pfpf
2
1 1
2
1
)(min)(~
)(min
n
j
n
iiij
n
jjj ii
p-pλpfpfpf λλ
nnnn
n
n λ
λ
pppp
pppp
pf
pf
:.
..
::
..
:1
1
1111
Axb
![Page 21: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/21.jpg)
Regularized Least-Squares
pi0 close to
pi1
![Page 22: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/22.jpg)
Regularized Least-Squares
pi0 close to
pi1
![Page 23: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/23.jpg)
Regularized Least-Squares
Rbf results with noise
![Page 24: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/24.jpg)
Regularized Least-Squares
Rbf results with noise
![Page 25: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/25.jpg)
Regularized Least-Squares
The Singular Value Decomposition• Every matrix A (nxm) can be decomposed into:
– where•U is an nxn orthogonal matrix•V is an mxm orthogonal matrix•D is an nxm diagonal matrix
TUDVA
![Page 26: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/26.jpg)
Regularized Least-Squares
The Singular Value Decomposition• Every matrix A (nxm) can be decomposed into:
– where•U is an nxn orthogonal matrix•V is an mxm orthogonal matrix•D is an nxm diagonal matrix
TUDVA
T
mmm
m
m
nnn
n
vv
vv
uu
uu
,1,
,11,1
1
,1,
,11,1
..
::
..
.
0..0
::
..0
::
0..
.
..
::
..
A
![Page 27: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/27.jpg)
Regularized Least-Squares
Geometric interpretation
T
mmm
m
m
nnn
n
vv
vv
uu
uu
,1,
,11,1
1
,1,
,11,1
..
::
..
.
0..0
::
..0
::
0..
.
..
::
..
A
![Page 28: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/28.jpg)
Regularized Least-Squares
Solving with the SVDTTT UDVAbAxAA.. and ˆ
![Page 29: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/29.jpg)
Regularized Least-Squares
Solving with the SVD
bUVDx
UDVAbAxAA..T
TTT
ˆ
ˆ and
![Page 30: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/30.jpg)
Regularized Least-Squares
Solving with the SVD
bUVDx
UDVAbAxAA..T
TTT
ˆ
ˆ and
bx
T
nnn
n
mmmm
m
uu
uu
vv
vv
,1,
,11,11
,1,
,11,1
..
::
..
.
0..0
::
1..0
::
0..1
.
..
::
..ˆ
![Page 31: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/31.jpg)
Regularized Least-Squares
Solving with the SVD
b
u
u
vvx
nm
m :.
0..0
::
1..0
::
0..1
...ˆ1
1
1
bUVDx
UDVAbAxAA..T
TTT
ˆ
ˆ and
![Page 32: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/32.jpg)
Regularized Least-Squares
Solving with the SVD
m
i
Tii
iσ1
1ˆ buvx
b
u
u
vvx
nm
m :.
0..0
::
1..0
::
0..1
...ˆ1
1
1
bUVDx
UDVAbAxAA..T
TTT
ˆ
ˆ and
![Page 33: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/33.jpg)
Regularized Least-Squares
A is nearly rank defficient
![Page 34: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/34.jpg)
Regularized Least-Squares
A is nearly rank defficient
T
mmm
m
m
nnn
n
vv
vv
uu
uu
,1,
,11,1
1
,1,
,11,1
..
::
..
.
0..0
::
..0
::
0..
.
..
::
..
A
![Page 35: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/35.jpg)
Regularized Least-Squares
A is nearly rank defficient
T
mmm
m
nnn
n
vv
vv
uu
uu
,1,
,11,1
1
,1,
,11,1
..
::
..
.
0..0
::
0..0
::
0..
.
..
::
..
A
![Page 36: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/36.jpg)
Regularized Least-Squares
A is nearly rank defficient
• A is nearly rank defficient =>some of the are close to 0iσ
![Page 37: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/37.jpg)
Regularized Least-Squares
A is nearly rank defficient
• A is nearly rank defficient =>some of the are close to 0=>some of the are close to
iσ
iσ
1
![Page 38: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/38.jpg)
Regularized Least-Squares
A is nearly rank defficient
• A is nearly rank defficient =>some of the are close to 0=>some of the are close to
• Problem with
m
i
Tii
iσ1
1ˆ buvx
iσ
iσ
1
![Page 39: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/39.jpg)
Regularized Least-Squares
A is nearly rank defficient
• A is nearly rank defficient =>some of the are close to 0=>some of the are close to
• Problem with
• Truncate the SVD
n
i
Tii
iσ1
1ˆ buvx
iσ
k
i
Tii
im σ
mkσσ1
1
1ˆ1 buvx , and with
iσ
1
![Page 40: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/40.jpg)
Regularized Least-Squares
pi0 close to
pi1
![Page 41: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/41.jpg)
Regularized Least-Squares
Rbf fit with truncated SVD
![Page 42: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/42.jpg)
Regularized Least-Squares
Rbf results with noise
![Page 43: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/43.jpg)
Regularized Least-Squares
Rbf fit with truncated SVD
![Page 44: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/44.jpg)
Regularized Least-Squares
Choosing cutoff value k
• The first k such as kσ
![Page 45: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/45.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
![Page 46: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/46.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
?
![Page 47: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/47.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
)(tivbT )()( , i
Bbbbii wt vTv
![Page 48: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/48.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
• Inverse skinning– Let be a set of pairs of geometry and skeleton
configurations
)()( , iBb
bbii wt vTv
)(tiv
bT
si
sb vT ,
![Page 49: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/49.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
• Inverse skinning– Let be a set of pairs of geometry and skeleton
configurations
)()( , iBb
bbii wt vTv
)(tiv
bT
si
sb vT ,
2
,sii
Bb
sbbiw vvT
![Page 50: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/50.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
• Inverse skinning– Let be a set of pairs of geometry and skeleton
configurations
)()( , iBb
bbii wt vTv
)(tiv
bT
si
sb vT ,
s
sii
Bb
sbbiw
2
, vvT
![Page 51: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/51.jpg)
Regularized Least-Squares
Example: inverse skinning“Skinning Mesh Animations”, James and Twigg, siggraph
• Skinning
• Inverse skinning– Let be a set of pairs of geometry and skeleton
configurations
)()( , iBb
bbii wt vTv
)(tiv
bT
s
sii
Bb
sbbiw w
bi
2
,,min vvT
si
sb vT ,
![Page 52: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/52.jpg)
Regularized Least-Squares
“Skinning Mesh Animations”, James and Twigg, siggraph
![Page 53: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/53.jpg)
Regularized Least-Squares
Problem with the TSVD
• We have to compute the SVD of A, and O() process:impractical for large marices
• Little control over regularization
![Page 54: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/54.jpg)
Regularized Least-Squares
Damped least-squares
• Replace
by
where is a scalar and L is a matrix
2min Axbx
222min Lx Axbx
![Page 55: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/55.jpg)
Regularized Least-Squares
Damped least-squares
• Replace
by
where is a scalar and L is a matrix
The solution verifies
2min Axbx
222min Lx Axbx
x̂
bAxLLAA TTT ˆ2
![Page 56: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/56.jpg)
Regularized Least-Squares
Examples of L
nl
l
0
01
L
11
11
L
Diagonal Differential
Limit scale Enforce smoothness
n
i
Tii
ii
i
lσ
σ
122
ˆ buvx
![Page 57: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/57.jpg)
Regularized Least-Squares
Rbf results with noise
![Page 58: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/58.jpg)
Regularized Least-Squares
810
![Page 59: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/59.jpg)
Regularized Least-Squares
710
![Page 60: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/60.jpg)
Regularized Least-Squares
410
![Page 61: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/61.jpg)
Regularized Least-Squares
Example: “Least-Squares Meshes”, Sorkin and Cohen-Or, siggaph
• Reconstruct a mesh given– Control points– Connectivity (planar mesh)
![Page 62: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/62.jpg)
Regularized Least-Squares
Example: “Least-Squares Meshes”, Sorkin and Cohen-Or, siggaph
• Reconstruct a mesh given– Control points– Connectivity (planar mesh)
• Smooth reconstruction
Eji
jiid,
vv
![Page 63: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/63.jpg)
Regularized Least-Squares
Example: “Least-Squares Meshes”, Sorkin and Cohen-Or, siggaph
• Reconstruct a mesh given– Control points– Connectivity (planar mesh)
• Smooth reconstruction
• In matrix form
Eji
jiid,
vv
0
/1
1
, iji dL
otherwise
if
if
E(i,j)
ji
0LzLyLx
x
x
n
i
.
.
v
v
x and
![Page 64: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/64.jpg)
Regularized Least-Squares
Reconstruction
• Minimize reconstruction error
where
Cs
ss xx 22min Lxx
point control the of position desired the is
mesh tedreconstruc
the inpoint control the of position the is
indicespoint control ofset the is
s
s
x
x
C
![Page 65: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/65.jpg)
Regularized Least-Squares
“Least-Squares Meshes”, Sorkine and Cohen-Or, siggraph
![Page 66: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/66.jpg)
Regularized Least-Squares
Quadratic constraints
• Solve
or
Lx min2
tosubject Axbx
d-Lx min2
tosubject Axbx
![Page 67: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/67.jpg)
Regularized Least-Squares
Quadratic constraints
• Solve
or
Lx min2
tosubject Axbx
d-Lx min2
tosubject Axbx
d-xL
d-Lx
ˆ
2
,particular In minimized. be to is
which inset feasible a defines Axb
![Page 68: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/68.jpg)
Regularized Least-Squares
Example
1
0
0
000
010
001
,3 dLx and R
![Page 69: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/69.jpg)
Regularized Least-Squares
Example
1
0
0
000
010
001
,3 dLx and R
222
21
21 xxd-Lx
![Page 70: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/70.jpg)
Regularized Least-Squares
Example
1
0
0
000
010
001
,3 dLx and R
222
21
21 xxd-Lx
12
1x2x
![Page 71: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/71.jpg)
Regularized Least-Squares
Discussion
• If , there is no solution (since there is no x for which )
d-Lxxmin d-Lx
![Page 72: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/72.jpg)
Regularized Least-Squares
Discussion
• If , there is no solution (since there is no x for which )
• If , the solution exists and is unique
d-Lxxmin d-Lx
d-Lxxmin
![Page 73: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/73.jpg)
Regularized Least-Squares
Discussion
• If , there is no solution (since there is no x for which )
• If , the solution exists and is unique
– Either the solution of is in the feasible set
d-Lxxmin d-Lx
d-Lxxmin
2min Axb x
![Page 74: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/74.jpg)
Regularized Least-Squares
Discussion
• If , there is no solution (since there is no x for which )
• If , the solution exists and is unique
– Either the solution of is in the feasible set
– or the solution is at the boundarySolve
d-Lxxmin d-Lx
d-Lxxmin
2min Axb x
d-Lx
d-LxAxb min2
tosubject x
![Page 75: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/75.jpg)
Regularized Least-Squares
Discussion
Solve
where is a Lagrange multiplier
dLbAxLLAA TTTT ˆ
![Page 76: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/76.jpg)
Regularized Least-Squares
Conclusion
• TSVD really useful if you need an SVD
![Page 77: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/77.jpg)
Regularized Least-Squares
Conclusion
• TSVD really useful if you need an SVD
• Regularization constrains the solution:– Value, differential operator, other properties– Soft (damping) or hard constraint (quadratic)– Linear or non-linear
![Page 78: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/78.jpg)
Regularized Least-Squares
Conclusion
• TSVD really useful if you need an SVD
• Regularization constrains the solution:– Value, differential operator, other properties– Soft (damping) or hard constraint (quadratic)– Linear or non-linear
• Danger of over-damping or constraining
![Page 79: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/79.jpg)
Regularized Least-Squares
Example: inverse kinematic
• Problem: solve for joint angles given end-effector positions
![Page 80: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/80.jpg)
Regularized Least-Squares
Example: inverse kinematic
• Problem: solve for joint angles given end-effector positions
?
![Page 81: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/81.jpg)
Regularized Least-Squares
Example: inverse kinematic
• Problem: solve for joint angles given end-effector positions
![Page 82: Regularized Least-Squares. Outline Why regularization? Truncated Singular Value Decomposition Damped least-squares Quadratic constraints](https://reader038.vdocuments.site/reader038/viewer/2022102923/551a9d8e5503466b3a8b5546/html5/thumbnails/82.jpg)
Regularized Least-Squares
Example: inverse kinematic
• Problem: solve for joint angles given end-effector positions