the choices we make - mit csail · mit computer science & artificial intelligence laboratory...
TRANSCRIPT
![Page 1: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/1.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
The Choices We MakeFrédo DurandMIT CSAIL
![Page 2: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/2.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Introduction• My background
– Math/CS– I have read much perception litterature– Amateur photographer– No artistic training, terrible draftsman
• Two example projects– Photography tonal management
– Line drawing from 3D models
![Page 3: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/3.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Introduction• I don't build tools, I am an academic researcher, my
deliverables are articles, not software– But it's not a complete excuse to write about useless tools
• Two types of "consumers/users"– Computer scientists who implement/extend my techniques
*Do they understand choices I made, choices they can make?
– En-users who use these tools*Are my choices relevant to them?
![Page 4: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/4.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Tonal management• Over and under-exposure is the largest
cause of bad photographs– Here's a choice I make!
• Both for professional and consumers
![Page 5: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/5.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
High Dynamic Range
• Real-world contrast is high• Display contrast is low
10-6 106
10-6 106
Real world
Picture
Low contrast
High dynamic range
![Page 6: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/6.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Our approach• Non-linear two-scale decomposition• Reduce contrast of large scale; preserve local detail
OutputLarge-scale
Detail
Color
![Page 7: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/7.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Live demo
![Page 8: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/8.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Choices I made• Decomposition: my contribution• Compute in log space: I have good reasons• How to separate intensity/color - incidental• How to reduce large-scale layer – incidental• Parameters I expose• Default parameters – matter of taste• Maybe other choices I don't realize I made
OutputLarge-scale
Detail
Color
![Page 9: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/9.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Anecdote about tone mapping evaluation
• Recent work has performed user experiments to evaluate competing tone mapping operators– [Ledda et al. 2005, Kuang et al. 2004]
• Interestingly, the former concludes my method is the worst, the latter that my method is the best!– They choose to test a different criterion: fidelity vs.
preference• More importantly, they focus on algorithm and
ignore parameters
![Page 10: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/10.jpg)
A programmable approach to Line DrawingWith Stephane Grabli, Emmanuel Turquin & François Sillion
![Page 11: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/11.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Motivation: Style vs. Technique
• Non-Photorealistic Rendering– Imitate traditional media
• Each paper focuses on one particular style, which isusually hardcode with a only few availableparameters
• Stylistic choices mixed with technical ones
![Page 12: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/12.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Goal: Decouple style from technique
• First step: pure line drawingSystem for rendering line drawing from 3D scenes– Including a flexible style description tool
![Page 13: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/13.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
• First step: pure line drawingSystem for rendering line drawing from 3D scenes– Including a flexible style description tool
Goal: Decouple style from technique
![Page 14: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/14.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Goal: Decouple style from technique
• First step: pure line drawingSystem for rendering line drawing from 3D scenes– Including a flexible style description tool– Ensuring model independence
![Page 15: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/15.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
• First step: pure line drawingSystem for rendering line drawing from 3D scenes– Including a flexible style description tool– Ensuring model independence
Goal: Decouple style from technique
![Page 16: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/16.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Style in line drawing
© ITEDO www.itedo.com
• Occlusion and nature thickness
![Page 17: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/17.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Style in line drawing
Herdman
• Depth discontinuity thickness
![Page 18: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/18.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Hypothesis
• Drawing can be described as a process
• Stylistic decisions (line thickness, omission) are related to scene and image information
• These decision strategies can be embedded in procedures
![Page 19: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/19.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Some relevant information
• Geometry (2D, 3D coordinates, normals, …)• Differential geometry (2D, 3D curvatures, …)• Line adjacency• Line nature (silhouette, crease, contour, …)• Occluding information
(visibility, occluder, depth discontinuity, …)• Material• Object id• Drawing density
![Page 20: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/20.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Approach
3D « View Map »Style
Drawing
+information
![Page 21: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/21.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Style: code
Operators.select(QuantitativeInvisibilityUP1D(0))Operators.bidirectionalChain(ChainSilhouetteIterator())Operators.recursiveSplit( Curvature2DF0D(),
pyParameterUP0D(0.2,0.8), NotUP1D(LengthHigherUP1D(75)), 2)
shaders_list = [ StrokeTextureShader("pencil.jpg", Stroke.DRY_MEDIUM, 1),ConstantColorShader(0,0,0,1),ConstantThicknessShader(2.0),pyGuidingLineShader(),pyBackboneStretcherShader(0.2)]
Operators.create(TrueUP1D(),shaders_list)
class pyGuidingLineShader(StrokeShader):def shade(self, stroke):
it = stroke.strokeVerticesBegin() itlast = stroke.strokeVerticesEnd()itlast.decrement() t = itlast.getObject().getPoint() - it.getObject().getPoint()itmiddle = StrokeVertexIterator(it)while(itmiddle.getObject().u()<0.5):
itmiddle.increment()while(it.isEnd() == 0):
it.getObject().SetPoint(itmiddle.getObject().getPoint() \+t*(it.getObject().u()-itmiddle.getObject().u()))it.increment()
![Page 22: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/22.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Style operators
• shade
• select
• chain
• split
![Page 23: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/23.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Style operators
• shade
• select
• chain
• split
![Page 24: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/24.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Shading Thickness Geometry
Color Information dependent
Plain strokes
![Page 25: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/25.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Shading
Depth discontinuity thickness
![Page 26: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/26.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Style operators
• shade
• select
• chain
• split
![Page 27: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/27.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Splitting
Split at points of highest 2D curvature
![Page 28: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/28.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Splitting
Split at points of highest 2D curvature
![Page 29: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/29.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Results
![Page 30: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/30.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Oriental style
![Page 31: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/31.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Density-based emphasis
![Page 32: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/32.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Technical illustration style
![Page 33: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/33.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Recap• Procedural description for style in line drawing
– Line drawing from 3D models– Control topology, geometry & attributes of strokes
• Choices– Automatic picture generation from 3D model– Restrict to pure line drawing– Describe style using procedures– View map, types of lines– Information we provide– Types of operators
![Page 34: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/34.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Choices We Make
• Model• Algorithms• Parameters• User Interface
• Problems we choose• Evaluation criteria
• In articles, we must explain the respective importance of choices
![Page 35: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/35.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
It's the question, stupid!
• The important is not the answer to a choice, it's the choice of the question
• And even more importantly, the implicit choices we make without asking the question
![Page 36: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/36.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
![Page 37: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/37.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Other choices
• Do we target pro of casual users?• How automatic should things be?
![Page 38: The Choices We Make - MIT CSAIL · MIT Computer Science & Artificial Intelligence Laboratory Anecdote about tone mapping evaluation • Recent work has performed user experiments](https://reader036.vdocuments.site/reader036/viewer/2022090604/60589969f5c5d027dd11c227/html5/thumbnails/38.jpg)
MIT Computer Science & Artificial Intelligence Laboratory
Bad choice consequences
• No uses our technique: we don't address any problem, or give the wrong solution
• People are frustrated by our technique• We make something too easy, becomes uniform
• Ethical problems