computational photography light field rendering jinxiang chai

Download Computational Photography Light Field Rendering Jinxiang Chai

Post on 21-Dec-2015

215 views

Category:

Documents

1 download

Embed Size (px)

TRANSCRIPT

  • Slide 1
  • Computational Photography Light Field Rendering Jinxiang Chai
  • Slide 2
  • Image-based Modeling: Challenging Scenes Why will they produce poor results? - lack of discernible features - occlusions - difficult to capture high-level structure - illumination changes - specular surfaces
  • Slide 3
  • Some Solutions - Use priors to constrain the modeling space - Aid modeling process with minimal user interaction - Combine image-based modeling with other modeling approaches
  • Slide 4
  • Videos Morphable face (click here)here Image-based tree modeling (click here)here Video trace (click here)here 3D modeling by ortho-images (Click here)here
  • Slide 5
  • Spectrum of IBMR Images user input range scans Model Images Image based modeling Image- based rendering Geometry+ Images Light field Images + Depth Geometry+ Materials Panoroma Kinematics Dynamics Etc. Camera + geometry
  • Slide 6
  • Outline Light field rendering [Levoy and Hanranhan SIG96] 3D light field (concentric mosaics) [Shum and He Sig99]
  • Slide 7
  • Plenoptic Function Can reconstruct every possible view, at every moment, from every position, at every wavelength Contains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! An image is a 2D sample of plenoptic function! P(x,y,z,,,,t)
  • Slide 8
  • Ray Lets not worry about time and color: 5D 3D position 2D direction P(x,y,z, )
  • Slide 9
  • Static objectCamera No Change in Radiance Static Lighting How can we use this?
  • Slide 10
  • Static objectCamera No Change in Radiance Static Lighting How can we use this?
  • Slide 11
  • Ray Reuse Infinite line Assume light is constant (vacuum) 4D 2D direction 2D position non-dispersive medium Slide by Rick Szeliski and Michael Cohen
  • Slide 12
  • Only need plenoptic surface
  • Slide 13
  • Synthesizing novel views Assume we capture every ray in 3D space!
  • Slide 14
  • Synthesizing novel views
  • Slide 15
  • Light field / Lumigraph Outside convex space 4D Stuff Empty
  • Slide 16
  • Light Field How to represent rays? How to capture rays? How to use captured rays for rendering
  • Slide 17
  • Light Field How to represent rays? How to capture rays? How to use captured rays for rendering
  • Slide 18
  • Light field - Organization 2D position 2D direction s
  • Slide 19
  • Light field - Organization 2D position 2 plane parameterization s u
  • Slide 20
  • Light field - Organization 2D position 2 plane parameterization u s t s,t u,v v s,t u,v
  • Slide 21
  • Light field - Organization Hold u,v constant Let s,t vary What do we get? s,tu,v
  • Slide 22
  • Lumigraph - Organization Hold s,t constant Let u,v vary An image s,tu,v
  • Slide 23
  • Lightfield / Lumigraph
  • Slide 24
  • Light field/lumigraph - Capture Idea 1 Move camera carefully over u,v plane Gantry >see Light field paper s,tu,v
  • Slide 25
  • Stanford multi-camera array 640 480 pixels 30 fps 128 cameras synchronized timing continuous streaming flexible arrangement
  • Slide 26
  • q For each output pixel determine s,t,u,v either use closest discrete RGB interpolate near values s u Light field/lumigraph - rendering
  • Slide 27
  • Nearest closest s closest u draw it Blend 16 nearest quadrilinear interpolation s u
  • Slide 28
  • Ray interpolation s u Nearest neighbor Linear interpolation in S-T Quadrilinear interpolation
  • Slide 29
  • Image Plane Camera Plane Light Field Capture Rendering Light Field/Lumigraph Rendering
  • Slide 30
  • Light fields Advantages: No geometry needed Simpler computation vs. traditional CG Cost independent of scene complexity Cost independent of material properties and other optical effects Disadvantages: Static geometry Fixed lighting High storage cost
  • Slide 31
  • 3D plenoptic function Image is 2D Light field/lumigraph is 4D What happens to 3D? - 3D light field subset - Concentric mosaic [Shum and He]
  • Slide 32
  • 3D light field One row of s,t plane i.e., hold t constant s,t u,v
  • Slide 33
  • 3D light field One row of s,t plane i.e., hold t constant thus s,u,v a row of images s u,v
  • Slide 34
  • Concentric mosaics [Shum and He] Polar coordinate system: - hold r constant - thus (,u,v)
  • Slide 35
  • Concentric mosaics Why concentric mosaic? - easy to capture - relatively small in storage size
  • Slide 36
  • Concentric mosaics From above How to captured images?
  • Slide 37
  • Concentric mosaics From above How to render a new image?
  • Slide 38
  • Concentric mosaics From above How to render a new image? - for each ray, retrieval the closest captured rays
  • Slide 39
  • Concentric mosaics From above How to render a new image? - for each ray, retrieval the closest captured rays
  • Slide 40
  • Concentric mosaics From above How to render a new image? - for each ray, retrieval the closest captured rays
  • Slide 41
  • Concentric mosaics From above object How to retrieval the closest rays?
  • Slide 42
  • Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?
  • Slide 43
  • Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?
  • Slide 44
  • Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?
  • Slide 45
  • Concentric mosaics From above object (s,t) interpolation plane How to retrieve the closest rays?
  • Slide 46
  • Concentric mosaics From above object (s,t) interpolation plane How to synthesize the color of rays?
  • Slide 47
  • Concentric mosaics From above object (s,t) interpolation plane How to synthesize the color of rays? - bilinear interpolation
  • Slide 48
  • Concentric mosaics From above
  • Slide 49
  • Concentric mosaics From above
  • Slide 50
  • Concentric mosaics What are limitations?
  • Slide 51
  • Concentric mosaics What are limitations? - limited rendering region? - large vertical distortion