deep learning for graphics supervised...
TRANSCRIPT
![Page 1: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/1.jpg)
Niloy Mitra Iasonas Kokkinos Paul Guerrero Vladimir Kim Kostas Rematas Tobias Ritschel
UCL UCL/Facebook UCL Adobe Research U Washington UCL
Deep Learning for Graphics
Supervised Applications
![Page 2: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/2.jpg)
EG Course Deep Learning for Graphics
Timetable
2
Niloy Iasonas Paul Vova Kostas Tobias
Introduction X X X X
Theory X
NN Basics X
Supervised Applications X X X
Data X
Unsupervised Applications X
Beyond 2D X X X
Outlook X X X X X X
![Page 3: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/3.jpg)
EG Course Deep Learning for Graphics
Fast (shared convolutions) Simple (dense)
FCN
Fully-Convolutional Network (FCN)
![Page 4: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/4.jpg)
EG Course Deep Learning for Graphics
J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. CVPR, 2015
FCN-based semantic segmentation
![Page 5: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/5.jpg)
EG Course Deep Learning for Graphics
L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy and A. Yuille, Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs, PAMI 2016
FCN-CRFs: Deeplab
![Page 6: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/6.jpg)
EG Course Deep Learning for Graphics
Ground truth FCN FCN-DCRF
Deeplab v2 results
![Page 7: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/7.jpg)
EG Course Deep Learning for Graphics
Deeplab v2 results
Ground truth FCN FCN-DCRF
![Page 8: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/8.jpg)
EG Course Deep Learning for Graphics
Object Detection: Fast(er)-RCNN
• Fast/Faster R-CNNGood speed
Good accuracy
Intuitive
Easy to use
Ross Girshick. “Fast R-CNN”. ICCV 2015.Shaoqing Ren, Kaiming He, Ross Girshick, & Jian Sun. “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”. NIPS 2015.
![Page 9: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/9.jpg)
EG Course Deep Learning for Graphics
Mask R-CNN
• Mask R-CNN = Faster R-CNN with FCN on RoIs
Faster R-CNN
FCN on RoI
![Page 10: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/10.jpg)
EG Course “Deep Learning for Graphics”
Mask R-CNN results on COCO
![Page 11: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/11.jpg)
EG Course “Deep Learning for Graphics”
• 1 keypoint = 1-hot “mask”
• Human pose = 17 masks
• Softmax over spatial locations• e.g. 562-way softmax on 56x56
Mask R-CNN for Human Keypoint Detection
![Page 12: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/12.jpg)
EG Course Deep Learning for Graphics
Mask R-CNN frame-by-frame
![Page 13: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/13.jpg)
EG Course “Deep Learning for Graphics”
Mask R-CNN frame-by-frame
![Page 14: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/14.jpg)
EG Course Deep Learning for Graphics
I. Kokkinos, UberNet: Training a Universal CNN for Low- Mid- and High-Level Vision, CVPR 2017
https://github.com/jkokkin/UberNet
UberNet: a “universal” network for all tasks
![Page 15: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/15.jpg)
EG Course Deep Learning for Graphics
“Inverse graphics”: understand how an image was generated from a scene
If we focus on a single object category: surface-based models
UberNet: Universal Network
DensePose: Unified model
What is the ultimate vision task?
![Page 16: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/16.jpg)
EG Course Deep Learning for Graphics
R. A. Guler, G. Trigeorgis, E. Antonakos, P. Snape, S. Zafeiriou, I. Kokkinos,
DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild, CVPR 2017
DenseReg: dense image-to-face regression
![Page 17: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/17.jpg)
R. A. Guler, N. Neverova, I. Kokkinos “DensePose: Dense Human Pose Estimation In The Wild”, CVPR’18
DensePose-RCNN: ~25 FPS
DensePose: dense image-to-body correspondence
![Page 18: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/18.jpg)
![Page 19: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/19.jpg)
EG Course Deep Learning for Graphics
SFSNet: incorporating image formation in model
SfSNet: Learning Shape, Reflectance and Illuminance of Faces ‘in the wild' SoumyadipSengupta Angjoo Kanazawa Carlos D. Castillo David W. Jacobs, CVPR 2018
![Page 20: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/20.jpg)
EG Course Deep Learning for Graphics
Beyond single frames: end-to-end optical flow
![Page 21: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/21.jpg)
EG Course Deep Learning for Graphics
End-to-end Structure From Motion
• DeMoN: Depth and Motion Network for Learning Monocular Stereo, B. Ummenhofer, et al, CVPR 2017
• Unsupervised learning of depth and ego-motion from video, T Zhou, M Brown, N Snavely, DG Lowe, CVPR 2017
![Page 22: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/22.jpg)
EG Course Deep Learning for Graphics
Monocular depth & normal estimation
• D. Eigen and R. Fergus, Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture, ICCV 2015
![Page 23: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/23.jpg)
EG Course “Deep Learning for Graphics”
Graphics applications
![Page 24: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/24.jpg)
EG Course Deep Learning for Graphics
Sketch Simplification
• Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup, Simon-Serra et al., 2016
• Deep Extraction of Manga Structural Lines, Li et al., 2017
24
![Page 25: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/25.jpg)
EG Course Deep Learning for Graphics
Sketch Simplification: Learning to Simplify
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup, Simo-Serra et al.
25
![Page 26: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/26.jpg)
EG Course Deep Learning for Graphics
Sketch Simplification: Learning to Simplify
• Loss for thin edges saturates easily
• Authors take extra steps to align input and ground truth edges
Pencil: inputRed: ground truth
Learning to Simplify: Fully Convolutional Networks for Rough Sketch Cleanup, Simo-Serra et al.
26
![Page 27: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/27.jpg)
EG Course Deep Learning for Graphics
Image Decomposition
• A selection of methods:
• Direct Instrinsics, Narihira et al., 2015
• Learning Data-driven Reflectance Priors for Intrinsic Image Decomposition, Zhou et al., 2015
• Decomposing Single Images for Layered Photo Retouching, Innamorati et al. 2017
27
![Page 28: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/28.jpg)
EG Course Deep Learning for Graphics
Image Decomposition: DecomposingSingle Images for Layered Photo Retouching
28
![Page 29: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/29.jpg)
EG Course Deep Learning for Graphics
Colorization
• Concurrent methods:• Let there be Color!, Iizuka et al., 2016
• Colorful Image Colorization, Zhang et al. 2016
• Learning Representations for Automatic Colorization, Larsson et al., 2016
• Real-Time User-Guided Image Colorization with Learned Deep Priors, Zhang et al. 2017
29
![Page 30: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/30.jpg)
EG Course Deep Learning for Graphics
Colorization: Let There Be Color!
Let there be Color!: Iizuka et al.
30
![Page 31: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/31.jpg)
EG Course Deep Learning for Graphics
Colorization: Colorful Image Colorization
inputdirect regression probability distr.
output
Image Credit: Colorful Image Colorization, Zhang et al.
31
![Page 32: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/32.jpg)
EG Course Deep Learning for Graphics
Mesh Labeling / Segmentation
3D Mesh Labeling via Deep Convolutional Neural Networks, Guo et al. 2016
32
![Page 33: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/33.jpg)
EG Course Deep Learning for Graphics
Mesh Labeling / Segmentation
3D Mesh Labeling via Deep Convolutional Neural Networks, Guo et al.
33
![Page 34: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/34.jpg)
EG Course Deep Learning for Graphics
LDR to HDR Image Reconstruction:
• Concurrently:
• Deep Reverse Tone Mapping, Endo et al. 2017
• HDR image reconstruction from a single exposure using deep CNNs, Eilertsen et al. 2017
34
![Page 35: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/35.jpg)
EG Course Deep Learning for Graphics
Reflectance Maps
• Paint a sphere as if it is made of a material under a certain illumination of another object in a photo
Deep Reflectance Maps. Rematas et al. CVPR 2015
35
![Page 36: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/36.jpg)
EG Course Deep Learning for Graphics
DeLight
• Factor BRDF and (HDR) Illumination
Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning. Georgoulis et al. PAMI 2017
36
![Page 37: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/37.jpg)
EG Course Deep Learning for Graphics
3D volumes form Xrays
Single-Image Tomography: 3D Volumes from 2D Cranial X-Rays. Henzler et al. EG 2018
37
![Page 38: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/38.jpg)
EG Course Deep Learning for Graphics
Deep Shading
• Paint a z-buffer like a path tracer (AO, DOF, MB)
Deep Shading, Nalbach et al. EGSR 2017
38
![Page 39: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/39.jpg)
EG Course Deep Learning for Graphics
Rendering Atmospherics
Speed up approx. 24 x Speed up approx. 24 x
39
Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks, Kallweit et al. SIGGRAPH Asia 2017
![Page 40: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/40.jpg)
EG Course Deep Learning for Graphics
Rendering Atmospherics: RPNN
Deep Scattering: Rendering Atmospheric Clouds with Radiance-Predicting Neural Networks, Kallweit et al. SIGGRAPH Asia 2017
In: Hierarchical representation of a cloud patchOut: incoming indirect radiance at patch center(incoming direct radiance is computed directly)
40
![Page 41: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/41.jpg)
EG Course Deep Learning for Graphics
Denoising Renderings
• Concurrent:
• Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings, Bako et al. 2017
• Interactive Reconstruction of Monte Carlo Image Sequences using a Recurrent Denoising Autoencoder, Chaitanya et al. 2017 (more on Autoencoders later)
Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings, Bako et al.
41
![Page 42: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/42.jpg)
EG Course Deep Learning for Graphics
Denoising Renderings:
Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings, Bako et al. SIGGRAPH 2017
42
![Page 43: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/43.jpg)
EG Course Deep Learning for Graphics
Geometry Abstraction / Simplification
Learning Shape Abstractions by Assembling Volumetric Primitives, Tulsiani et al. 2016
43
![Page 44: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/44.jpg)
EG Course Deep Learning for Graphics
Geometry Abstraction / Simplification:
44
Learning Shape Abstractions by Assembling Volumetric Primitives, Tulsiani et al. 2016
![Page 45: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/45.jpg)
EG Course Deep Learning for Graphics
Procedural Parameter Estimation
Interactive Sketching of Urban Procedural Models, Nishida et al. 2016
45
![Page 46: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/46.jpg)
EG Course Deep Learning for Graphics
Procedural Parameter Estimation:Interactive Sketching of Urban Procedural Models
Interactive Sketching of Urban Procedural Models, Nishida et al.
46
![Page 47: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/47.jpg)
EG Course Deep Learning for Graphics
Audio-driven facial animation
Audio-Driven Facial Animation by Joint End-to-End Learning of Pose and Emotion, Karras et al. 2017
![Page 48: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/48.jpg)
EG Course Deep Learning for Graphics
3D Pose Estimation: VNECT
VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera, Mehta et al., SIGGRAPH 2017
skeleton joint heatmapand 3d positions50
![Page 49: Deep Learning for Graphics Supervised Applicationsgeometry.cs.ucl.ac.uk/dl4g/slides/part4_Supervised... · 2018. 4. 18. · EG Course Deep Learning for Graphics Object Detection:](https://reader035.vdocuments.site/reader035/viewer/2022081409/6081c24039417573dd4d1009/html5/thumbnails/49.jpg)
EG Course Deep Learning for Graphics
Thank you!
http://geometry.cs.ucl.ac.uk/dl4g/
52