learning to shade hand- drawn sketches39.96.165.147/seminar/yuhan_200405.pdf · learning to shade...
TRANSCRIPT
-
Learning to Shade Hand-drawn Sketches
CVPR 2020 Oral Presenter: Yu Han
2020/04/05
1
-
Outline • Authors • Background • Network • Experiment • Conclusion
2
-
Qingyuan Zheng* · University of Maryland
3
-
Zhuoru Li* · Project HAT
4
-
Adam Bargteil · Assistant professor at University of Maryland
· Phd at UCB · Postdoc at CMU Graphics Lab
Research Interest: · Computer graphics and animation, especially using physics-based animation
5
-
Outline • Authors • Background • Network • Experiment • Conclusion
6
-
Background• Pix2pix
7
Image-to-Image Translation with Conditional Adversarial Networks. In CVPR, 2017.
-
Background• DeepNormal
8
Deep Normal Estimation for Automatic Shading of Hand-Drawn Characters. In ECCV, 2018.
-
Background• Sketch2Normal
9
Interactive Sketch-Based Normal Map Generation with Deep Neural Networks. In SIGGRAPH, 2018.
-
Background• Squeeze and Excitation(SE) Block
10
SE Net:Squeeze-and-Excitation Networks. In CVPR, 2018.
-
Outline • Authors • Background • Network • Experiment • Conclusion
11
-
Network
12
-
Data
13
· 1160 paired hand-drawn line drawings and shadows · Tagged with a lighting direction · 8x3+2=26 directions
-
Network Architecture 1. Generative Network 2. Discriminator Network 3. Loss Function
14
-
Generative Network
15
Two parts: • Shape net: encodes the underlying 3D structure from 2D sketches • Render net: renders artistic shadows based on the encoded structure
-
Generative Network
16
Render net: • Self-attention: enhance the visual reasoning 3D structure from 2D sketches. • SE blocks: filter out unnecessary features • Extract two supervision side outputs, s1 and s2
-
Network Architecture 1. Generative Network 2. Discriminator Network 3. Loss Function
17
-
Discriminator Network
18
• Self-attention: make discriminator sensitive to the distant features
-
Network Architecture 1. Generative Network 2. Discriminator Network 3. Loss Function
19
-
Loss Function
20
-
Outline • Authors • Background • Network • Experiment • Conclusion
21
-
Experiment
22
• Shift, zoom in/out, and rotate to augment the dataset • Input: 320x320 • 80 000 iterations
-
Experiment
23
-
Experiment
24
-
Experiment
25
-
Experiment
26
-
User Study
27
All people.
People with drawing experience.
-
Ablation Study
28
-
Ablation Study
29
Calculate the first two orders of the distance between distributions in Gaussian space.
-
Outline • Authors • Background • Network • Experiment • Conclusion
30
-
Conclusion • New dataset
• Propose a network that “understands” the structure and 3D spatial
relationships implied by line drawings and produces highly-detailed
and accurate shadows
31
-
Thank you
32