# modeling light cs129: computational photography james hays, brown, spring 2011 slides from alexei a....

Post on 04-Jan-2016

217 views

Category:

## Documents

Embed Size (px)

TRANSCRIPT

• Modeling Lightcs129: Computational PhotographyJames Hays, Brown, Spring 2011Slides from Alexei A. Efros and others

• What is light?Electromagnetic radiation (EMR) moving along rays in spaceR(l) is EMR, measured in units of power (watts) l is wavelengthUseful things:Light travels in straight linesIn vacuum, radiance emitted = radiance arriving i.e. there is no transmission loss

• Figures Stephen E. Palmer, 2002What do we see?3D world2D image

• What do we see?3D world2D imagePainted backdrop

• On Simulating the Visual ExperienceJust feed the eyes the right dataNo one will know the difference!Philosophy:Ancient question: Does the world really exist?Science fiction:Many, many, many books on the subject, e.g. slowglass from Light of Other DaysLatest take: The MatrixPhysics:Slowglass might be possible?Computer Science:Virtual Reality

To simulate we need to know:What does a person see?

• The Plenoptic FunctionQ: What is the set of all things that we can ever see?A: The Plenoptic Function (Adelson & Bergen)

Lets start with a stationary person and try to parameterize everything that he can seeFigure by Leonard McMillan

• Grayscale snapshotis intensity of light Seen from a single view pointAt a single timeAveraged over the wavelengths of the visible spectrum(can also do P(x,y), but spherical coordinate are nicer)P(q,f)

• Color snapshotis intensity of light Seen from a single view pointAt a single timeAs a function of wavelength

P(q,f,l)

• A movieis intensity of light Seen from a single view pointOver timeAs a function of wavelengthP(q,f,l,t)

• Holographic movieis intensity of light Seen from ANY viewpointOver timeAs a function of wavelengthP(q,f,l,t,VX,VY,VZ)

• The Plenoptic FunctionCan reconstruct every possible view, at every moment, from every position, at every wavelengthContains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! Not bad for a functionP(q,f,l,t,VX,VY,VZ)

• Sampling Plenoptic Function (top view)Just lookup -- Quicktime VR

• Model geometry or just capture images?

• RayLets not worry about time and color:

5D3D position2D directionP(q,f,VX,VY,VZ)Slide by Rick Szeliski and Michael Cohen

• SurfaceCamera No Change in

• Ray ReuseInfinite lineAssume light is constant (vacuum)

4D2D direction2D positionnon-dispersive mediumSlide by Rick Szeliski and Michael Cohen

• Only need plenoptic surface

• Synthesizing novel viewsSlide by Rick Szeliski and Michael Cohen

• Lumigraph / LightfieldOutside convex space

4DStuffEmptySlide by Rick Szeliski and Michael Cohen

• Lumigraph - Organization 2D position2D directionsqSlide by Rick Szeliski and Michael Cohen

• Lumigraph - Organization 2D position2D position

2 plane parameterizationsuSlide by Rick Szeliski and Michael Cohen

• Lumigraph - Organization 2D position2D position

2 plane parameterizationustvSlide by Rick Szeliski and Michael Cohen

• Lumigraph - OrganizationHold s,t constantLet u,v varyAn image

s,tu,vSlide by Rick Szeliski and Michael Cohen

• Lumigraph / Lightfield

• Lumigraph - Capture Idea 1Move camera carefully over s,t planeGantrysee Lightfield paper

s,tu,vSlide by Rick Szeliski and Michael Cohen

• Lumigraph - Capture Idea 2Move camera anywhereRebinningsee Lumigraph paper

s,tu,vSlide by Rick Szeliski and Michael Cohen

• Lumigraph - RenderingFor each output pixeldetermine s,t,u,v

eitheruse closest discrete RGBinterpolate near valuesSlide by Rick Szeliski and Michael Cohen

• Lumigraph - RenderingNearestclosest sclosest udraw it

Blend 16 nearestquadrilinear interpolationSlide by Rick Szeliski and Michael Cohen

2005 Marc Levoy

Stanford multi-camera array640 480 pixels 30 fps 128 cameras

synchronized timingcontinuous streamingflexible arrangement

2005 Marc Levoy

Light field photography using a handheld plenoptic cameraRen Ng, Marc Levoy, Mathieu Brdif,Gene Duval, Mark Horowitz and Pat Hanrahan

2005 Marc Levoy

Conventional versus light field camera

2005 Marc Levoy

Conventional versus light field camera

2005 Marc Levoy

Prototype camera4000 4000 pixels 292 292 lenses = 14 14 pixels per lensContax medium format cameraKodak 16-megapixel sensor

2005 Marc Levoy

2005 Marc Levoy

Digitally stopping-downstopping down = summing only the central portion of each microlens

2005 Marc Levoy

Digital refocusingrefocusing = summing windows extracted from several microlenses

2005 Marc Levoy

Example of digital refocusing

2005 Marc Levoy

Digitally moving the observermoving the observer = moving the window we extract from the microlenses

2005 Marc Levoy

Example of moving the observer

• 3D LumigraphOne row of s,t planei.e., hold t constants,tu,v

• 3D LumigraphOne row of s,t planei.e., hold t constantthus s,u,va row of imagessu,v

• by David DeweyP(x,t)

• 2D: ImageWhat is an image?

All rays through a pointPanorama?Slide by Rick Szeliski and Michael Cohen

• ImageImage plane

2Dposition

• Spherical PanoramaAll light rays through a point form a ponoramaTotally captured in a 2D array -- P(q,f)Where is the geometry???See also: 2003 New Years Evehttp://www.panoramas.dk/fullscreen3/f1.html

• Other ways to sample Plenoptic FunctionMoving in time: Spatio-temporal volume: P(q,f,t)Useful to study temporal changesLong an interest of artists:

Claude Monet, Haystacks studies

• Space-time imagesOther ways to slice theplenoptic functionxyt

• The Theatre Workshop Metaphordesired image(Adelson & Pentland,1996)PainterLighting DesignerSheet-metalworker

• Painter (images)

• Lighting Designer (environment maps)Show Naimark SF MOMA videohttp://www.debevec.org/Naimark/naimark-displacements.mov

• Sheet-metal Worker (geometry)Let surface normals do all the work!

• working togetherWant to minimize costEach one does whats easiest for himGeometry big thingsImages detailLighting illumination effectsclever Italians

How would you recreate this scene?**The stanford and washington groups reverse their notation with regard to u,v and s,t******medium format cameras sensor twice as large as 35mm full frame, quite expensive (10k+)******

Recommended