modeling light cs129: computational photography james hays, brown, spring 2011 slides from alexei a....

Download Modeling Light cs129: Computational Photography James Hays, Brown, Spring 2011 Slides from Alexei A. Efros and others

Post on 04-Jan-2016

217 views

Category:

Documents

3 download

Embed Size (px)

TRANSCRIPT

  • Modeling Lightcs129: Computational PhotographyJames Hays, Brown, Spring 2011Slides from Alexei A. Efros and others

  • What is light?Electromagnetic radiation (EMR) moving along rays in spaceR(l) is EMR, measured in units of power (watts) l is wavelengthUseful things:Light travels in straight linesIn vacuum, radiance emitted = radiance arriving i.e. there is no transmission loss

  • Figures Stephen E. Palmer, 2002What do we see?3D world2D image

  • What do we see?3D world2D imagePainted backdrop

  • On Simulating the Visual ExperienceJust feed the eyes the right dataNo one will know the difference!Philosophy:Ancient question: Does the world really exist?Science fiction:Many, many, many books on the subject, e.g. slowglass from Light of Other DaysLatest take: The MatrixPhysics:Slowglass might be possible?Computer Science:Virtual Reality

    To simulate we need to know:What does a person see?

  • The Plenoptic FunctionQ: What is the set of all things that we can ever see?A: The Plenoptic Function (Adelson & Bergen)

    Lets start with a stationary person and try to parameterize everything that he can seeFigure by Leonard McMillan

  • Grayscale snapshotis intensity of light Seen from a single view pointAt a single timeAveraged over the wavelengths of the visible spectrum(can also do P(x,y), but spherical coordinate are nicer)P(q,f)

  • Color snapshotis intensity of light Seen from a single view pointAt a single timeAs a function of wavelength

    P(q,f,l)

  • A movieis intensity of light Seen from a single view pointOver timeAs a function of wavelengthP(q,f,l,t)

  • Holographic movieis intensity of light Seen from ANY viewpointOver timeAs a function of wavelengthP(q,f,l,t,VX,VY,VZ)

  • The Plenoptic FunctionCan reconstruct every possible view, at every moment, from every position, at every wavelengthContains every photograph, every movie, everything that anyone has ever seen! it completely captures our visual reality! Not bad for a functionP(q,f,l,t,VX,VY,VZ)

  • Sampling Plenoptic Function (top view)Just lookup -- Quicktime VR

  • Model geometry or just capture images?

  • RayLets not worry about time and color:

    5D3D position2D directionP(q,f,VX,VY,VZ)Slide by Rick Szeliski and Michael Cohen

  • SurfaceCamera No Change in

    RadianceLightingHow can we use this?

  • Ray ReuseInfinite lineAssume light is constant (vacuum)

    4D2D direction2D positionnon-dispersive mediumSlide by Rick Szeliski and Michael Cohen

  • Only need plenoptic surface

  • Synthesizing novel viewsSlide by Rick Szeliski and Michael Cohen

  • Lumigraph / LightfieldOutside convex space

    4DStuffEmptySlide by Rick Szeliski and Michael Cohen

  • Lumigraph - Organization 2D position2D directionsqSlide by Rick Szeliski and Michael Cohen

  • Lumigraph - Organization 2D position2D position

    2 plane parameterizationsuSlide by Rick Szeliski and Michael Cohen

  • Lumigraph - Organization 2D position2D position

    2 plane parameterizationustvSlide by Rick Szeliski and Michael Cohen

  • Lumigraph - OrganizationHold s,t constantLet u,v varyAn image

    s,tu,vSlide by Rick Szeliski and Michael Cohen

  • Lumigraph / Lightfield

  • Lumigraph - Capture Idea 1Move camera carefully over s,t planeGantrysee Lightfield paper

    s,tu,vSlide by Rick Szeliski and Michael Cohen

  • Lumigraph - Capture Idea 2Move camera anywhereRebinningsee Lumigraph paper

    s,tu,vSlide by Rick Szeliski and Michael Cohen

  • Lumigraph - RenderingFor each output pixeldetermine s,t,u,v

    eitheruse closest discrete RGBinterpolate near valuesSlide by Rick Szeliski and Michael Cohen

  • Lumigraph - RenderingNearestclosest sclosest udraw it

    Blend 16 nearestquadrilinear interpolationSlide by Rick Szeliski and Michael Cohen

    2005 Marc Levoy

    Stanford multi-camera array640 480 pixels 30 fps 128 cameras

    synchronized timingcontinuous streamingflexible arrangement

    2005 Marc Levoy

    Light field photography using a handheld plenoptic cameraRen Ng, Marc Levoy, Mathieu Brdif,Gene Duval, Mark Horowitz and Pat Hanrahan

    2005 Marc Levoy

    Conventional versus light field camera

    2005 Marc Levoy

    Conventional versus light field camera

    2005 Marc Levoy

    Prototype camera4000 4000 pixels 292 292 lenses = 14 14 pixels per lensContax medium format cameraKodak 16-megapixel sensor

    2005 Marc Levoy

    2005 Marc Levoy

    Digitally stopping-downstopping down = summing only the central portion of each microlens

    2005 Marc Levoy

    Digital refocusingrefocusing = summing windows extracted from several microlenses

    2005 Marc Levoy

    Example of digital refocusing

    2005 Marc Levoy

    Digitally moving the observermoving the observer = moving the window we extract from the microlenses

    2005 Marc Levoy

    Example of moving the observer

  • 3D LumigraphOne row of s,t planei.e., hold t constants,tu,v

  • 3D LumigraphOne row of s,t planei.e., hold t constantthus s,u,va row of imagessu,v

  • by David DeweyP(x,t)

  • 2D: ImageWhat is an image?

    All rays through a pointPanorama?Slide by Rick Szeliski and Michael Cohen

  • ImageImage plane

    2Dposition

  • Spherical PanoramaAll light rays through a point form a ponoramaTotally captured in a 2D array -- P(q,f)Where is the geometry???See also: 2003 New Years Evehttp://www.panoramas.dk/fullscreen3/f1.html

  • Other ways to sample Plenoptic FunctionMoving in time: Spatio-temporal volume: P(q,f,t)Useful to study temporal changesLong an interest of artists:

    Claude Monet, Haystacks studies

  • Space-time imagesOther ways to slice theplenoptic functionxyt

  • The Theatre Workshop Metaphordesired image(Adelson & Pentland,1996)PainterLighting DesignerSheet-metalworker

  • Painter (images)

  • Lighting Designer (environment maps)Show Naimark SF MOMA videohttp://www.debevec.org/Naimark/naimark-displacements.mov

  • Sheet-metal Worker (geometry)Let surface normals do all the work!

  • working togetherWant to minimize costEach one does whats easiest for himGeometry big thingsImages detailLighting illumination effectsclever Italians

    How would you recreate this scene?**The stanford and washington groups reverse their notation with regard to u,v and s,t******medium format cameras sensor twice as large as 35mm full frame, quite expensive (10k+)******

Recommended

View more >