geometric computer vision review and outlook. what is computer vision? three related fieldsthree...
TRANSCRIPT
![Page 1: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/1.jpg)
Geometric Computer VisionReview and Outlook
![Page 2: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/2.jpg)
What is Computer Vision?
• Three Related fieldsThree Related fields– Image Processing: Changes 2D images into Image Processing: Changes 2D images into
other 2D imagesother 2D images– Computer Graphics: Takes 3D models, renders Computer Graphics: Takes 3D models, renders
2D images2D images– Computer vision: Extracts scene information Computer vision: Extracts scene information
from 2D images and videofrom 2D images and video– e.g. Geometry, “Where” something is in 3D, e.g. Geometry, “Where” something is in 3D, – Objects “What” something is”Objects “What” something is”
• What information is in a 2D image?What information is in a 2D image?• What information do we need for 3D analysis?What information do we need for 3D analysis?
• CV hard. Only special cases can be solved. CV hard. Only special cases can be solved. •
90
horizon
Image ProcessingComputer Graphics
Computer Vision
Computer Vision
![Page 3: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/3.jpg)
The equation of projection
Mathematically:Mathematically:• Cartesian coordinates:Cartesian coordinates:(x, y, z) ( f
x
z, f
y
z)
Intuitively:Intuitively:
43: PPXx
33: KRK Xtx Camera matrix (internal params)
Rotation, translation (ext. params)t,R
Projection matrix
Projectively:Projectively:
![Page 4: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/4.jpg)
X
Y
Z
Homogeneous coordinates
X
Y
Z
s
s 0
X∞
X
Y
0
X∞ =
• Perspective imaging models 2d projective space
• Each 3D ray is a point in P2 : homogeneous coords.
• Ideal points
• P2 is R2 plus a “line at infinity” l∞
The 2D projective plane
Projective point
l∞
x
yX
Y1Z= Inhomogeneou
s equivalent
![Page 5: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/5.jpg)
Lines
HZ • Ideal line ~ the plane parallel to the image
A
B
C
l =X=lTX = XTl = AX + BY + CZ = 0
l∞ =
0
0
1
• Projective line ~ a plane through the origin
For any 2d projective property, a dual property holds when the role of points and lines are interchanged.
Duality:
X
Y
0
X∞ =
l1 l2X =X1 X2=l
The line joining two points The point joining two lines
X
Y
Z
X
l“line at infinity
”
![Page 6: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/6.jpg)
Projective transformations
• Homographies, collineations, projectivitiesHomographies, collineations, projectivities
• 3x3 nonsingular H3x3 nonsingular H
3
2
1
333231
232221
131211
3
2
1
'
'
'
x
x
x
hhh
hhh
hhh
x
x
x
l0= Hà Tl
xTl = 0 x0Tl0= 0
maps P2 to P2
8 degrees of freedomdetermined by 4 corresponding points
x0= Hx• Transforming Lines?subspaces preserved
xTHTl0= 0substitution
dual transformation
![Page 7: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/7.jpg)
Planar Projective Warping
HZA novel view rendered via
four points with known structurexi
0= Hxii = 1. . .4
xi xi0
![Page 8: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/8.jpg)
GroupGroup TransformationTransformation InvariantsInvariants DistortionDistortion
ProjectiveProjective
8 DOF8 DOF
• Cross ratioCross ratio
• IntersectionIntersection
• TangencyTangency
AffineAffine
6 DOF6 DOF
• ParallelismParallelism
• Relative dist in 1dRelative dist in 1d
• Line at infinityLine at infinity
MetricMetric
4 DOF4 DOF
• Relative distancesRelative distances
• AnglesAngles
• Dual conicDual conic
EuclideanEuclidean
3 DOF3 DOF
• LengthsLengths
• AreasAreas
Geometric strata: 2d overview
1TS
sRH
O
t
1TA
AH
O
t
1TE
RH
O
t
v
AH TP v
t
C*
2 dofl
2 dof
l
C*
![Page 9: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/9.jpg)
Projective 3D space
0),,,,( 44321 xxxxxX
),,()1,,,(
)/,/,/(),,,( 4342414321
ZYXZYX
xxxxxxxxxx
)0,,,( 321 xxx
•Projective transformation:Projective transformation: –4x4 nonsingular matrix 4x4 nonsingular matrix HH–Point transformationPoint transformation–Plane transformationPlane transformation
•Quadrics:Quadrics: QQ Dual: Dual: Q*Q*–4x4 symmetric matrix Q4x4 symmetric matrix Q
–9 DOF (defined by 9 points in general pose)9 DOF (defined by 9 points in general pose)
•PointsPoints
•PlanesPlanes0
),,,( 4321
Xπ
πT
Points and planes are dual in 3d projective
space.
ππ
XXTH
H
'
'
0XX QT 0* ππ QT
•Lines: 5DOF, various parameterizationsLines: 5DOF, various parameterizations
![Page 10: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/10.jpg)
GroupGroup TransformationTransformation InvariantsInvariants DistortionDistortion
ProjectiveProjective
15 DOF15 DOF
• Cross ratioCross ratio
• IntersectionIntersection
• TangencyTangency
AffineAffine
12 DOF12 DOF
• ParallelismParallelism
• Relative dist in 1dRelative dist in 1d
• Plane at infinityPlane at infinity
MetricMetric
7 DOF7 DOF
• Relative distancesRelative distances
• AnglesAngles
• Absolute conicAbsolute conic
EuclideanEuclidean
6 DOF6 DOF
• LengthsLengths
• AreasAreas
• VolumesVolumes
Geometric strata: 3d overview
1TS
sRH
O
t
1TA
AH
O
t
1TE
RH
O
t
v
AH TP v
t
π
3DOFπ
5DOF
Faugeras ‘95
![Page 11: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/11.jpg)
Tasks in Geometric Vision
•Tracking:Tracking: Often formulated in Often formulated in 2D, but tracks 3D objects/scene2D, but tracks 3D objects/scene
•Visual control:Visual control: Explicit 2D Explicit 2D geometric constraints implicitly geometric constraints implicitly fulfills a 3D motion alignmentfulfills a 3D motion alignment
•3D reconstruction3D reconstruction: Computes : Computes an explicit 3D model from 2D an explicit 3D model from 2D imagesimages
y31
y21
y11
![Page 12: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/12.jpg)
Camera model: Relates 2D – 3D
43: PPXx
33: KRK Xtx Camera matrix (internal params)
Rotation, translation (ext. params)t,R
[Dürer]
Projection matrix
![Page 13: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/13.jpg)
• Projection equationProjection equation
xxii=P=PiiXX
• Resection:Resection:– xxii,X P,X Pii
2D-3D geometry - resection
Given image points and 3D points calculate camera projection matrix.
![Page 14: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/14.jpg)
• Projection equationProjection equation
xxii=P=PiiXX
• Intersection:Intersection:– xxii,P,Pi i XX
2D-3D geometry - intersection
Given image points and camera projections in at least 2 views calculate the 3D points (structure)
![Page 15: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/15.jpg)
• Projection Projection equationequation
xxii=P=PiiXX
• Structure from Structure from motion (SFM)motion (SFM)– xxii P Pii,, XX
2D-3D geometry - SFM
Given image points in at least 2 views calculate the 3D points (structure) and camera projection matrices (motion)
•Estimate projective structure
•Rectify the reconstruction to metric (autocalibration)
![Page 16: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/16.jpg)
Reconstructing scenes‘Small’ scenes (one, few buildings)
SFM + multi view stereo
man made scenes: prior on architectural elements
interactive systems
City scenes (several streets, large area)
aerial images
ground plane, multi cameras
SFM + stereo [+ GPS]
depth map fusions
![Page 17: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/17.jpg)
time
Modeling dynamic scenes
Large scale (city) modeling
![Page 18: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/18.jpg)
Modeling (large scale) scenes
[Adam
Rachmielowski ]
![Page 19: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/19.jpg)
SFM + stereo
[Dellaert et al 3DPVT06 ]
[Zisserman, Werner ECCV02 ]
Man-made environments : straight edges family of lines vanishing points
![Page 20: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/20.jpg)
SFM + stereo dominant planes plane sweep – homog between 3D pl. and camera pl. one parameter search – voting for a plane
[Zisserman, Werner ECCV02 ] [Bischof et al 3DPVT06 ]
![Page 21: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/21.jpg)
SFM + stereo
[Zisserman, Werner ECCV02 … ]
refinement – architectural primitives
![Page 22: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/22.jpg)
SFM+stereo
• Refinement – dense stereoRefinement – dense stereo
[Pollefeys, Van Gool 98,00,01]
www.arc3d.be
![Page 23: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/23.jpg)
Façade – first system
[Debevec, Taylor et al. Siggraph 96]
Based on SFM (points, lines, stereo)Some manual modelingView dependent texture
![Page 24: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/24.jpg)
Priors on architectural primitives
θ – parameters for architectural priors
type, shape, texture
M – model
D – data (images)
I – reconstructed structures (planes, lines …)
[Cipolla, Torr, … ICCV01]
prior
Occluded windows
![Page 25: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/25.jpg)
Interactive systems
[Torr et al. Eurogr.06, Siggraph07]
Video, sparse 3D points, user input
M – model primitives
D- data
I – reconstructed geometry
Solved with graph cut
![Page 26: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/26.jpg)
City modeling – aerial images
[Heiko Hirschmuller et al - DLR]Airborne pushbroom camera
Semi-global stereo matching (based on mutual information)
![Page 27: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/27.jpg)
City modeling – ground plane
Camera clustercar + GPS
Calibrated cameras – relative poseGPS – car position - 3D tracking
2D feature tracker
3D pointsDense stereo+fusionTexture
3D MODEL
[Nister, Pollefeys et al
3DPVT06, ICCV07]
[Cornelis, Van Gool CVPR06…]
Video: Cannot do frame-frame
correspondences
SFM
![Page 28: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/28.jpg)
City modeling - example
[Cornelis, Van Gool CVPR06…]
1. feature matching = tracking
2. SFM – camera pose + sparse 3D points
3. Façade reconstruction
– rectification of the stereo images
- vertical line correlation
4. Topological map generation
- orthogonal proj. in the horiz. plane
- voting based carving
5. Texture generation
- each line segment – column in texture space VIDEO
![Page 29: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/29.jpg)
On-line scene modeling : Adam’s projectOn-line modeling from video Model not perfect but enough for scene visualization Application predictive display
Tracking and ModelingNew imageDetect fast corners (similar to Harris)SLAM (mono SLAM [Davison ICCV03])
Estimate camera poseUpdate visible structure
Partial bundle adjustment – update all pointsSave image if keyframe (new view – for texture)
VisualizationNew visual pose
Compute closet viewTriangulate Project images from closest views onto surface
SLAMCamera pose3D structure Noise modelExtended Kalman Filter
![Page 30: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/30.jpg)
Model refinement
![Page 31: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/31.jpg)
![Page 32: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/32.jpg)
Modeling dynamic scenes
[Neil Birkbeck]
![Page 33: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/33.jpg)
Multi-camera systems
Several cameras mutually registered (precalibrated)
Video sequence in each camera
Moving object
time
![Page 34: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/34.jpg)
Techniques Naïve : reconstruct shape every frame
Integrate stereo and image motion cues
Extend stereo in temporal domain
Estimate scene flow in 3D from optic flow and stereo
Representations :
Disparity/depth
Voxels / level sets
Deformable mesh – hard to keep time consistency
Knowledge:
Camera positions
Scene correspondences (structured light)
![Page 35: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/35.jpg)
Spacetime stereo
[Zhang, Curless, Seitz: Spacetime stereo, CVPR 2003]Extends stereo in time domain: assumes intra-frame correspondences
)()(),,( 000 00yydxxddtyxd yx
Static scene: disparity Dynamic scene:
)(
)()(),,(
0
000
0
00
ttd
yydxxddtyxd
t
yx
![Page 36: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/36.jpg)
Spacetime stereo: Results
![Page 37: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/37.jpg)
Spacetime stereo:video
![Page 38: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/38.jpg)
Spacetime photometric stereo[Hernandez et al. ICCV 2007]
One color camera
projectors – 3 different positions
Calibrated w.r. camera
Each channel (R,G,B) – one colored light posePhotometric stereo
![Page 39: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/39.jpg)
Spacetime PS - Results
![Page 40: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/40.jpg)
3. Scene flow[Vedula, Baker, Rander, Collins, Kanade: Three dimensional scene flow, ICCV 99]
2D Optic flow
t
I
dt
dI
dt
d iii
uu:
3D Scene flow
iu
i
i
i
tdt
d
dt
d
tt
xu
u
xx
uxx ));((
Scene flow on tangent plane
Motion of x along a ray
![Page 41: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/41.jpg)
Scene flow: results
[Vedula, et al. ICCV 99]
![Page 42: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/42.jpg)
Scene flow: video
[Vedula, et al. ICCV 99]
![Page 43: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/43.jpg)
4. Carving in 6D[Vedula, Baker, Seitz, Kanade: Shape and motion carving in 6D]
Hexel:
),,(),,(),,(
),,,,,(
111222
111
zyxzyxzyx
zyxzyx
6D photo-consistency:
21
212121
2
)()(
))((;))((
nn
SSSSSSSS
PISSPISi
ti
ti
t
i
ti
ti
t
xx
![Page 44: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/44.jpg)
6D slab sweeping
Slab = thickened plane (thikness = upper bound on the flow magnitude) compute visibility for x1
determine search region compute all hexel photo-consistency carving hexels update visibility(Problem: visibility below the top layer in the slab before carving)
![Page 45: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/45.jpg)
Carving in 6D: results
![Page 46: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/46.jpg)
7. Surfel sampling[ Carceroni, Kutulakos: Multi-view scene capture by surfel samplig, ICCV01]
Surfel: dynamic surface element shape component : center, normal, curvature motion component: reflectance component: Phong parameters
knx ,, 00S
vtuttM xxx ~,~,~
},...,{,, 1 PkfR
![Page 47: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/47.jpg)
Reconstruction algorithm
lllii
predi
RS
i jiii
prediic
LrI
RSERS
IIvRSEi
)(),,,()(
)),(min(minarg,
)()()(),(
**
pplpcnpp
ppp
visibility
shadowPhong reflectance
ci – camera i
ll- light l
![Page 48: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/48.jpg)
Surfel sampling : results
![Page 49: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/49.jpg)
Modeling humans in motion
GRIMAGE platform- INRIA Grenoble
Multiple calibrated cameras
Human in motion
Goal: 3D model of the human
Instantaneous model that can be viewed from different poses (‘Matrix’) and inserted in an artificial scene (tele-conferences)
Our goal: 3D animated human model
capture model deformations and appearance change in motion
animated in a video game
![Page 50: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/50.jpg)
Articulated model Geometric Model
Skeleton + skinned mesh (bone weights )
50+ DOF (CMU mocap data)
Tracking
visual hull – bone weights by diffusion
refine mesh/weights
Components silhouette extraction tracking the course model learn deformations learn appearance change
[Neil Birkbeck]Model based approach
![Page 51: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/51.jpg)
Neil- tracking results
![Page 52: Geometric Computer Vision Review and Outlook. What is Computer Vision? Three Related fieldsThree Related fields –Image Processing: Changes 2D images into](https://reader035.vdocuments.site/reader035/viewer/2022062305/5697bf7c1a28abf838c84249/html5/thumbnails/52.jpg)
Computer Vision
Questions?Questions?