relighting with 4d incident light fields vincent masselus pieter peers philip dutré yves d. willems
TRANSCRIPT
![Page 1: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/1.jpg)
Relighting with 4D Incident Light FieldsRelighting with 4D Incident Light Fields
Vincent MasselusVincent MasselusPieter PeersPieter PeersPhilip DutréPhilip DutréYves D. WillemsYves D. Willems
![Page 2: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/2.jpg)
Relighting: apply incident illumination to a real object Relighting: apply incident illumination to a real object
![Page 3: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/3.jpg)
Reflectance fieldReflectance field
• Transfer of light through the sceneTransfer of light through the scene
4D exitant
4D incident
![Page 4: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/4.jpg)
Previous workPrevious work
• Debevec et al. SIGGRAPH2000Debevec et al. SIGGRAPH2000
2D exitant2D incident
Images ACM SIGGRAPH
(Environment map)
(Image)
![Page 5: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/5.jpg)
Previous workPrevious work
• Matusik et al. SIGGRAPH2002Matusik et al. SIGGRAPH2002
4D exitant
Images ACM SIGGRAPH
2D incident(Environment map)
(Light field)
![Page 6: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/6.jpg)
Incident IlluminationIncident Illumination
• Usually represented by an environment mapUsually represented by an environment map
2D incident
![Page 7: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/7.jpg)
Spatial varying incident illuminationSpatial varying incident illumination
• Environment Map: incident illumination at one pointEnvironment Map: incident illumination at one point
![Page 8: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/8.jpg)
• New effects:New effects:– Spot light Spot light – ShadowsShadows– ……
• New effects:New effects:– Spot light Spot light – ShadowsShadows– ……
Goal: relighting with angular andspatially varying illuminationGoal: relighting with angular andspatially varying illumination
![Page 9: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/9.jpg)
• One view pointOne view point• Use artificial incident light fields or captured Use artificial incident light fields or captured
from real environment from real environment
Goal: relighting with angular andspatially varying illuminationGoal: relighting with angular andspatially varying illumination
2D exitant(Image)
4D incident(Light Field)
![Page 10: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/10.jpg)
Acquire the reflectance fieldAcquire the reflectance field
• Use camera to capture exitant light fieldUse camera to capture exitant light field• Use projector to create incident light fieldUse projector to create incident light field
Projector
Camera
2D exitant(Image)
4D incident(Light Field)
![Page 11: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/11.jpg)
Data acquisitionData acquisition
![Page 12: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/12.jpg)
Data acquisitionData acquisition
![Page 13: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/13.jpg)
Data acquisitionData acquisition
![Page 14: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/14.jpg)
Data acquisitionData acquisition
![Page 15: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/15.jpg)
Data acquisitionData acquisition
![Page 16: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/16.jpg)
Data Acquisition: setupData Acquisition: setup
ProjectorCamera
Object
![Page 17: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/17.jpg)
Data Acquisition: setupData Acquisition: setup
![Page 18: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/18.jpg)
RelightingRelighting
Incident Light Field
Scene
x W1
W1
Apply
![Page 19: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/19.jpg)
RelightingRelighting
Incident Light Field
Scene
x W1
x W2
W2
![Page 20: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/20.jpg)
RelightingRelighting
Incident Light Field
Scene
x W1
x W2
x W3
W3
![Page 21: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/21.jpg)
RelightingRelighting
Incident Light Field
Scene
x W1
x W2
x W3
…
x WnWn
![Page 22: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/22.jpg)
RelightingRelightingx W1
x W2
x W3
…
x Wn
+
+
+
![Page 23: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/23.jpg)
NxN light bundles NxN light bundles
![Page 24: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/24.jpg)
• #images per projector position = N#images per projector position = N22
• Reduce #images per projector position: O(N)Reduce #images per projector position: O(N)
Speedup Speedup
…
…
…
…
…
N
N
![Page 25: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/25.jpg)
O(N) PatternsO(N) Patterns
![Page 26: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/26.jpg)
Reconstruct basis imageReconstruct basis image
![Page 27: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/27.jpg)
Reconstruct basis imageReconstruct basis image
• Take the minimum value per pixelTake the minimum value per pixel
![Page 28: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/28.jpg)
• #Patterns per projector position = O(N)#Patterns per projector position = O(N)
• Assumption: limited local influence Assumption: limited local influence • Total complexity:Total complexity:
O(PxNO(PxN22) to O(PxN) with P = #projector positions) to O(PxN) with P = #projector positions
Speedup Speedup
… …
4
N
![Page 29: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/29.jpg)
ResultsResults
• 32x7 projector positions32x7 projector positions• 16x16 light bundles16x16 light bundles
• 57344 basis images57344 basis images
reconstructed fromreconstructed from
14336 photographs14336 photographs
![Page 30: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/30.jpg)
![Page 31: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/31.jpg)
Relighting with a captured 4D Incident Light FieldRelighting with a captured 4D Incident Light Field
• Unger Unger et al.et al. EGSR2003 EGSR2003
![Page 32: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/32.jpg)
4D Incident light field vs. 2D Environment map4D Incident light field vs. 2D Environment map
4D 2D
![Page 33: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/33.jpg)
Comparison 4D vs. 2D IlluminationComparison 4D vs. 2D Illumination
2D4D
![Page 34: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/34.jpg)
ResultsResults
• 32x7 projector positions32x7 projector positions• 32x32 light bundles32x32 light bundles
• 229376 basis images229376 basis images
reconstructed fromreconstructed from
28672 photographs28672 photographs
![Page 35: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/35.jpg)
ResultsResults
![Page 36: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/36.jpg)
ResultsResults
![Page 37: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/37.jpg)
Conclusion Conclusion • Acquire reflectance field to relight with 4D Acquire reflectance field to relight with 4D
incident light fieldsincident light fields• Speedup: O(PxNSpeedup: O(PxN22) to O(PxN)) to O(PxN)
• Use other illumination basis functionsUse other illumination basis functions– Gaussians, Wavelets,…Gaussians, Wavelets,…
• Smarter acquisitionSmarter acquisition– What images do we really need?What images do we really need?
• Move the cameraMove the camera– Capture the complete 8D reflection fieldCapture the complete 8D reflection field
Future workFuture work
![Page 38: Relighting with 4D Incident Light Fields Vincent Masselus Pieter Peers Philip Dutré Yves D. Willems](https://reader036.vdocuments.site/reader036/viewer/2022062803/56649f455503460f94c67462/html5/thumbnails/38.jpg)
AcknowledgementsAcknowledgements
• Graphics group from K.U.LeuvenGraphics group from K.U.Leuven• All reviewersAll reviewers• Captured ILF by Jonas Unger Captured ILF by Jonas Unger