3d model-based human face modeling · 3d model-based human face modeling. ii-i - introduction ii-ii...
TRANSCRIPT
![Page 1: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/1.jpg)
André Gagalowicz
Projet MIRAGESINRIA - Rocquencourt - Domaine de Voluceau
78153 Le Chesnay CedexE-Mail : [email protected]
3D model-based human facemodeling
![Page 2: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/2.jpg)
IIII--II -- INTRODUCTIONINTRODUCTION
IIII--IIII FIRST STEP : DEFORMATION OF THEFIRST STEP : DEFORMATION OF THEGENERIC MODEL WITH THE AID OF FACEGENERIC MODEL WITH THE AID OF FACEFEATURE POINTSFEATURE POINTS
IIII--IIIIII SECOND STEP: USE OF LIMBS TO FINALIZESECOND STEP: USE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
IIII--IVIV RESULTSRESULTS
![Page 3: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/3.jpg)
No RealNo Real--timetime
ContextContext
QualityQuality (model must(model must bebe preciseprecise))
Goal: FaceGoal: Face TrackingTracking in ain a videovideo sequencesequence
Application: PostApplication: Post--productionproduction
Face pose estimation forFace pose estimation for virtualvirtual realityrealityLipsLips andand emotionemotion trackingtrackingMustMust bebe quick,quick, regularregular andand reusablereusable
RotoscopyRotoscopy
IIII--II -- INTRODUCTIONINTRODUCTION
![Page 4: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/4.jpg)
PreviousPrevious approachesapproachesHow toHow to obtainobtain aa goodgood 3D model of face3D model of face??
3D DEVICES3D DEVICES(control on the 3D model production REQUIRED(control on the 3D model production REQUIRED))
1.1. Laser captureLaser capture devicesdevices
2.2. StructuredStructured lightlight
![Page 5: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/5.jpg)
PrevioPrevioUUss ApproachesApproaches
How toHow to obtainobtain aa goodgood 3D model of face3D model of face??FromFrom IMAGESIMAGES
1.1. GraphicsGraphics designersdesigners
2.2. ShapeShape fromfrom shadingshading
![Page 6: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/6.jpg)
PreviousPrevious ApproachesApproachesHow toHow to obtainobtain aa goodgood 3D model of face3D model of face??
FromFrom IMAGESIMAGES
1.1. GraphicsGraphics designersdesigners
2.2. ShapeShape fromfrom shadingshading
3.3. GenericGeneric face modificationface modification
![Page 7: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/7.jpg)
ModelingModeling isis performedperformed inin twotwo stepssteps::
••FIRST STEP: DEFORMATION OF THE GENERICFIRST STEP: DEFORMATION OF THE GENERICMODEL WITH THE AID OF FACE FEATURE POINTSMODEL WITH THE AID OF FACE FEATURE POINTS
•• SECOND STEP: USE OF LIMBS TO FINALIZE THESECOND STEP: USE OF LIMBS TO FINALIZE THE3D FACE RECONSTRUCTION3D FACE RECONSTRUCTION
![Page 8: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/8.jpg)
CooperativeCooperativeCalibration /Calibration /
ReconstructionReconstruction
Calibration ofthe camerasusing feature
points
3Dreconstructionof face feature
points
Deformationof the whole
3D meshusing RBF
RadialRadialBasisBasis
FunctionFunction
DataData
GenericGeneric
faceface
ModelRefinment
usinglimbs
FinalizeFinalize thethedeformationdeformation
SpecificSpecific
faceface
ResultsResults
SecondSecond stepstep
GLOBAL SCHEMEGLOBAL SCHEME
![Page 9: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/9.jpg)
IIII--IIII --11-- ChoiceChoice ofof characteristiccharacteristic pointspoints
IIII--IIII --22-- Interactive calibration of the camerasInteractive calibration of the cameras
IIII--IIII --33-- AssigningAssigning interactivelyinteractively thethe characteristiccharacteristic points ofpoints ofthethe genericgeneric model to the face imagesmodel to the face images
IIII--IIII FIRST STEP : DEFORMATION OF THE GENERIC MODELFIRST STEP : DEFORMATION OF THE GENERIC MODELWITH THE AID OF CHARACTERISTIC POINTSWITH THE AID OF CHARACTERISTIC POINTS
IIII--IIII ––44 RealisationRealisation of the modelof the model deformationdeformation
![Page 10: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/10.jpg)
IIII--IIII –– 11 ChoiceChoice ofof characteristiccharacteristic pointspoints
a prioria priori KnowledgeKnowledge
GenericGeneric modelmodel3434 FeatureFeature pointspoints
![Page 11: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/11.jpg)
IIII--IIII –– 11 ChoiceChoice ofof characteristiccharacteristic pointspoints
IIII--IIII –– 22 -- Interactive Calibration of the camerasInteractive Calibration of the cameras
IIII--IIII -- 44-- RealisationRealisation of the modelof the model deformationdeformation
IIII--IIII -- 33-- AssigningAssigning interactivelyinteractively thethe characteristiccharacteristicpoints of thepoints of the genericgeneric model to the face imagesmodel to the face images
IIII FIRST STEP : DEFORMATION OF THE GENERIC MODELFIRST STEP : DEFORMATION OF THE GENERIC MODELWITH THE AID OF CHARACTERISTIC POINTSWITH THE AID OF CHARACTERISTIC POINTS
![Page 12: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/12.jpg)
IIII--IIII –– 22 -- Interactive Calibration of the camerasInteractive Calibration of the cameras
![Page 13: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/13.jpg)
IIII--IIII -- 11 ChoiceChoice ofof characteristiccharacteristic pointspoints
IIII--IIII -- 22 -- Interactive Calibration of the camerasInteractive Calibration of the cameras
IIII--IIII -- 33-- AssigningAssigning interactivelyinteractively thethe characteristiccharacteristicpoints of thepoints of the genericgeneric model to the face imagesmodel to the face images
IIII--IIII -- 44-- RealisationRealisation of the modelof the modeldeformationdeformation
![Page 14: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/14.jpg)
IIII--IIII --33-- AssigningAssigning interactivelyinteractively thethe characteristiccharacteristic points of thepoints of thegenericgeneric model to the face imagesmodel to the face images
![Page 15: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/15.jpg)
IIII--IIII -- 22 -- Interactive Calibration of the camerasInteractive Calibration of the cameras
IIII--IIII -- 33-- AssigningAssigning interactivelyinteractively thethe characteristiccharacteristicpoints of thepoints of the genericgeneric model to the face imagesmodel to the face images
IIII--IIII -- 44-- RealisationRealisation of the modelof the model deformationdeformation
IIII--IIII -- 11 ChoiceChoice ofof characteristiccharacteristic pointspoints
a)a) CooperativeCooperative Calibration/3D Reconstruction ofCalibration/3D Reconstruction of featurefeature pointspointsb) Extrapolation ofb) Extrapolation of deformationsdeformations to theto the completecomplete faceface meshmesh
![Page 16: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/16.jpg)
a)a) CooperativeCooperative Calibration/3D Reconstruction ofCalibration/3D Reconstruction of featurefeature pointspoints
Calibration ofCalibration ofthe camerasthe camerasusingusing featurefeature
pointspoints
3D3Dreconstructionreconstruction
ofof featurefeaturepointspoints
![Page 17: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/17.jpg)
1
24
3
51
24
3
5
6
CalibrationCalibrationCalibrationCalibration
Calibration ofCalibration of eacheach cameracamera
needsneeds::thethe intrinsicintrinsic parametersparameters (focal(focal lengthlength, pixel aspect ratio, pixel aspect ratio ……))
atat leastleast 55 3D points3D pointsthethe associatedassociated projectionsprojections
IterativeIterative::
MinimizesMinimizes the distancethe distance betweenbetween thethe estimatedestimated projection ofprojection ofeacheach 3D point and3D point and eacheach associatedassociated real projection in imagereal projection in image
1
2
4
3
5
6
3434 featurefeature pointspoints
a)a) CooperativeCooperative Calibration / 3D Reconstruction ofCalibration / 3D Reconstruction of featurefeature pointspoints
methodmethod::POSITPOSIT úû
ùêë
é10TR
![Page 18: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/18.jpg)
3D Reconstruction3D Reconstruction3D Reconstruction
Camera 1Camera 1Camera 2Camera 2
3D Reconstruction by3D Reconstruction by stereostereo CalibrationCalibration isis notnot perfectperfect
ReconstructedReconstructed point: Middle point onpoint: Middle point onthe segmentthe segment givinggiving the minimumthe minimum
distancedistance betweenbetween thethe twotwo lineslines ofof sightsight
a)a) CooperativeCooperative Calibration / 3D Reconstruction ofCalibration / 3D Reconstruction of featurefeature pointspoints
![Page 19: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/19.jpg)
Calibration / Reconstruction
Average error for all cameras during calibration /reconstruction
00,10,20,30,40,50,60,70,80,9
1
1 2 3 4 5 6 7 8 9 10 11
Iteration number
Err
or v
alue
![Page 20: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/20.jpg)
11
22
33 11
2244
3311
2244
33
CaseCase wherewhere a pointa point isis seenseen onlyonly onon oneone image: Useimage: Use AdaptiveAdaptive symmetrysymmetry
555555
3D Reconstruction3D Reconstruction3D Reconstruction
a)a) CooperativeCooperative Calibration/3D Reconstruction ofCalibration/3D Reconstruction of featurefeature pointspoints
HereHere point 4point 4 isis seenseen onlyonly on theon the leftleft imageimage
![Page 21: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/21.jpg)
PP11
Proj1(P1)
Proj1(P2)
Proj2(P1)
Camera 1
Image 1
V1R1
Camera 2
Image 2
PP22
a)a) CooperativeCooperative Calibration/3D Reconstruction ofCalibration/3D Reconstruction of featurefeature pointspoints
AdaptiveAdaptive symmetrysymmetry (cont.)(cont.)
![Page 22: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/22.jpg)
R1
Proj1(P2)
Camera 1
Camera 2
Image 1
PP11
PP22
R2
R’2
Image 2
a)a) CooperativeCooperative Calibration/3D Reconstruction ofCalibration/3D Reconstruction of featurefeature pointspoints
AdaptiveAdaptive symmetrysymmetry (cont.)(cont.)
![Page 23: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/23.jpg)
GivenGiven a set ofa set ofdeformationdeformation vectorsvectors
BasicBasic ideaidea::
11 ComputeCompute thethe deformationdeformation atat featurefeature points (points (donedone inin a )a )))
22 InterpolateInterpolate thethe deformationsdeformations on theon the wholewhole meshmesh
USEUSE Radial BasisRadial Basis FunctionsFunctions (RBF)(RBF)
b) Extrapolation ofb) Extrapolation of deformationsdeformations to theto the completecomplete faceface meshmesh
![Page 24: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/24.jpg)
å=
-=n
ii DiPAP
1))(.()f( s
For nFor n deformationdeformation vectorsvectors DD1..n1..n of points Pof points P1..n1..n ,,choosechoose aa deformationdeformation functionfunction f(P) as :f(P) as :
WhereWhere ss isis thethe Radial BasisRadial Basis FunctionFunction
TypicallyTypically:: rr =)(s
, for local, for local effecteffect dependingdepending on Con C)/()(2 Crer -=s
, for, for smoothsmooth resultresult
WeWe have:have: ••a set of na set of n unknowsunknows AA1..n1..n
••nn constraintsconstraints:: ii DP =)f( For i = 1 .. nFor i = 1 .. n
b) Extrapolation ofb) Extrapolation of deformationsdeformations to theto the completecomplete faceface meshmesh
![Page 25: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/25.jpg)
Calibration / Reconstruction - Problems
• Anti-parallel positions of the cameras
C1
I1
C2
I2
R
C1
I1
C2
I2
R
![Page 26: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/26.jpg)
Calibration / Reconstruction - Problems
• Solution:– Angle between C1, C2
– Symmetry plane
3D reconstruction without using symmetry plane 3D reconstruction with using symmetry plane
![Page 27: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/27.jpg)
ModelModel ResultingResulting fromfrom thethe deformationdeformation basedbased onon featurefeature points (inpoints (in GREENGREEN)) isis poorpoor!!
IdeaIdea::
deformdeform thethe currentcurrent modelmodel usingusing face silhouette (face silhouette (limbslimbs) in) in REDRED
IIII--IIIIII SECOND STEPSECOND STEP–– USE OF LIMBS TO FINALIZEUSE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
![Page 28: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/28.jpg)
ForFor eacheach cameracamera
ComputeCompute the modelthe model limblimbandand projectproject itit in image planein image plane
IIII--IIIIII SECOND STEPSECOND STEP–– USE OF LIMBS TO FINALIZEUSE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
![Page 29: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/29.jpg)
Automatic 3D model silhouette extraction
• Silhouette construction • Special processing ofintersections
• Enhancements:– Processing of only those edges which lie between the visible and
invisible facets– Computation of the intersections only for the currently chosen edge
• Result: 60 sec→ 1.2 sec of computation time per view
Initial vertexProjections ofvisible facets on theimage plane
![Page 30: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/30.jpg)
ForFor eacheach cameracamera
ComputeCompute thethe limblimb andand projectproject ititin image planein image plane
ConstructConstruct by hand the realby hand the realcontour in image planecontour in image plane
Captions:Captions:
Control Point of the BControl Point of the Béézierzier curvecurve
NextNext tangent,tangent, evolvingevolving alongalong thethe curvecurve
PrecedingPreceding tangent,tangent, evolvingevolving alongalong thethe curvecurve
IIIIII SECOND STEPSECOND STEP–– USE OF LIMBS TO FINALIZEUSE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
![Page 31: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/31.jpg)
ForFor eacheach cameracamera
ComputeCompute thethe limblimb andandprojectproject itit in image planein image plane
ConstructConstruct the realthe realcontour in image planecontour in image plane
ComputeCompute the 2Dthe 2D deformationsdeformations transformingtransforming thetheprojectedprojected modelmodel limblimb to the real contourto the real contour
IIII--IIIIII SECOND STEPSECOND STEP–– USE OF LIMBS TO FINALIZEUSE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
![Page 32: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/32.jpg)
22
2
11
1
77
7
44
4
88
8
55
5
99
9
66
6
33
3
Use local maxima (Use local maxima (improvesimproves computation)computation)andand propagatepropagate thethe deformationsdeformations fromfrom thethemaximamaxima alongalong thethe curvecurve usingusing 1D RBF1D RBF
PP’’88
RR’’88
PP’’99RR’’99
2D2D DeformationDeformation vectorsvectors
How toHow to obtainobtain the 2Dthe 2D DeformationsDeformations ??
![Page 33: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/33.jpg)
InferInfer the 3Dthe 3D deformationsdeformations fromfrom the 2Dthe 2D onesones
ComputeCompute the 2Dthe 2D deformationsdeformations transformingtransforming thetheprojectedprojected limblimb intointo the real contourthe real contour
ForFor eacheach cameracamera
ComputeCompute thethe limblimb andandprojectproject itit in image planein image plane
ConstructConstruct the realthe realcontour in image planecontour in image plane
IIIIII SECOND STEPSECOND STEP–– USE OF LIMBS TO FINALIZEUSE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
![Page 34: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/34.jpg)
2D2D àà 3D3D DeformationsDeformations2D2D àà 3D3D DeformationsDeformations
![Page 35: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/35.jpg)
InferInfer the 3Dthe 3D deformationsdeformations fromfrom the 2Dthe 2D onesones
CalculateCalculate the 2Dthe 2D deformationsdeformations transformingtransforming thetheprojectedprojected limblimb to the real contourto the real contour
ForFor eacheach cameracamera
CalculateCalculate thethe limblimb andandprojectproject itit in image planein image plane
DefineDefine the realthe realcontour in image planecontour in image plane
AddAdd thisthis set of 3Dset of 3D deformationsdeformations incrementallyincrementally totothethe previousprevious set ofset of featurefeature pointspoints
IIII--IIIIII SECOND STEPSECOND STEP–– USE OF LIMBS TO FINALIZEUSE OF LIMBS TO FINALIZETHE 3D FACE RECONSTRUCTIONTHE 3D FACE RECONSTRUCTION
![Page 36: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/36.jpg)
IIII--IVIV RESULTSRESULTS
![Page 37: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/37.jpg)
IIII--IVIV RESULTSRESULTS
![Page 38: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/38.jpg)
LambertianLambertiansynthesissynthesis
OriginalOriginal
IIII--IVIV RESULTSRESULTS
![Page 39: 3D model-based human face modeling · 3D model-based human face modeling. II-I - INTRODUCTION II-II FIRST STEP : DEFORMATION OF THE GENERIC MODEL WITH THE AID OF FACE FEATURE POINTS](https://reader033.vdocuments.site/reader033/viewer/2022042314/5f01b68f7e708231d400ae7a/html5/thumbnails/39.jpg)
VALIDATIONVALIDATION
TheThe precisionprecision of the 3D reconstruction of facesof the 3D reconstruction of faces fromfrom 44imagesimages whenwhen thethe geometrygeometry of facesof faces waswas knownknown beforehandbeforehandwaswas 2%2%
((similarsimilar to theto the precisionprecision of aof a conventionalconventional laser system)laser system)
TIMETIME
The timeThe time requiredrequired toto constructconstruct a 3D modela 3D model fromfrom scratchscratch
lieslies betweenbetween 20 min to20 min to ½½ hourhour!!