and Computational Photography - ? What is Computational Photography? Computational photography is

Download and Computational Photography - ? What is Computational Photography? Computational photography is

Post on 25-Aug-2018

221 views

Category:

Documents

9 download

TRANSCRIPT

  • Pixels,Numbers,andProgramsComputational PhotographyComputationalPhotography

    StevenL.Tanimoto

    Pixels,Numbers,andPrograms;S.Tanimoto 1ComputationalPhotography

  • Outline

    Wh t i t ti l h t h ?Whatiscomputationalphotography?AutofocustechniquesHi h d i (HDR) h hHighdynamicrange(HDR)photographyCatadioptric camerasSeparatingimagesduetodirectandindirectlightingOtherdevelopments

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 2

  • WhatisComputationalPhotography?

    Computational photography is an emerging technology forComputationalphotographyisanemergingtechnologyforacquiringimagesthroughacombinationofoptics,sensors,andcomputers.

    Itincludesautomaticcameracontrol,computationofpixelvaluesusingmultiplesourcesofinformation,aswellasg p ,computingalternativerepresentationsofvisualinformationto2Dimages(e.g.,3Dmodels).

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 3

  • Autofocustechniques

    Focusing a camera lens was once a tedious chore.Focusingacameralenswasonceatediouschore.Instant(film)camerasusedfixedfocus,limitingphotographytowelllitscenesandfast(highASA)p g p y ( g )films.Acousticrangefinding wasusedonsomecamerastog f gdeterminethedistancefromthecameratothefirstsurfaceinthemiddleofthefieldofview.Digitalcameraschangetheeconomicsofautofocus.Now,techniquesbasedonsharpnessmeasurements

    ff tiPixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography

    areeffective.4

  • Autofocustechniques(cont.)

    Brennersharpness:

    Tenengrad sharpness:

    HereGx andGy arethehorizontalandverticalgradientoperators

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography

    x y g pusedintheSobel edgedetector.

    5

  • Autofocustechniques(cont.)

    x

    Frame#1 Frame#10

    Sharpnessasafunctionoffocussetting.Bestfocus

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 6

    gisatthe10th frame.

  • HighDynamicRangePhotography

    The dynamic range of a photo or camera is the rangeThedynamicrangeofaphotoorcameraistherangeofbrightnesslevelsthatitcanaccuratelycapture.Historically,dynamicrangewaslimitedby(a)filmy, y g y ( )technology,and(b)digitallightsensortechnology.

    Thephotographerschallenge:takeapictureofascenethatcontainsbothbrightanddarkregions,withoutlosingthedetailsinoneoftheseregions.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 7

  • HDR(cont.)

    The brightly lit area is capturedThebrightlylitareaiscapturedeffectivelyinthisphoto(takenwitha1/60sec.exposure).

    Butthelowerhalfofthephotop(theindoorpartofthescene)istoodarkandlackingincontrast.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 8

  • HDR(cont.)

    Here, the darkest areas areHere,thedarkestareasarecapturedeffectively(takenwitha15sec.exposure).

    Buttheupperhalfofthephotopp p(theoutdoorpartofthescene)istoobrightandlackingincontrast.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 9

  • HDR(cont.)

    Now, both areas have someNow,bothareashavesomevisiblestructure(takenwitha1/2sec.exposure).

    Butneithershowsdetailswell.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 10

  • HDR(cont.)

    HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherangerange.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 11

  • HDR(cont.)

    HDR to the rescue!HDRtotherescue!Takingafullsequenceofexposures,webuildahighp , gdynamicrangeimageinsidethecomputer.Fromit,weapplytonemappingtogetanimagethathasgoodcontrastacrosstherange Th id l i i i frange. Theidea:getluminosityinfoabouteachpixelfromeachimage,butweighttheevidenceaccordingtowhateachimageisgood at

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography

    goodat.

    12

  • Catadioptric Imaging

    Using optical systems that combine lenses andUsingopticalsystemsthatcombinelensesandmirrors,anamorphicimagescanbecapturedthatcontainentirepanoramas.Computerscaninverttheanamorphicdistortion.

    CourtesyofProf.ShreeNayar ofColumbiaUniversity.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 13

  • Catadioptric Imaging

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 14

  • Catadioptric Cameras

    TheOneshot360(RemoteReality,Inc.).Threeothercatadioptric cameras.http://www.cs.columbia.edu/CAVE/projects/cat_cam_360/

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 15

  • SeparatingDirect&IndirectLighting

    Thecolorofanobject(specificallyatapointPontheobject)inascenedepends on two different physical effects:dependsontwodifferentphysicaleffects:a. Directreflectionoflightfromthelightsourceoffthesurfaceofthe

    object(atpointP),andb Light coming from point P on the object that did not come directlyb. LightcomingfrompointPontheobjectthatdidnotcomedirectly

    fromthelightsource:i.lightthathascomefromthelightsourcebutbouncedoffother

    surfaces before getting to PsurfacesbeforegettingtoP.ii.lightthathasenteredthesurfacematerial(e.g.,thepaintlayer,

    whichistypicallyadielectricmaterialandispartiallytranslucent)elsewhere from point P but emerges from point PelsewherefrompointPbutemergesfrompointP.

    Weareoftenunawareof(ii).However,usingcomputationalphotography,we can separate the effects of (i) and (ii) The results are amazing

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography

    wecanseparatetheeffectsof(i)and(ii).Theresultsareamazing.

    16

  • Direct&Indirect(cont.)

    indirectlightsource surfacesin

    thescene

    P

    camera

    Rt

    RadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 17

  • Direct&Indirect(cont.)

    indirectlightsource

    BarriertocastashadowonP

    P

    camera

    Rg

    RadianceatPtowardsthecamera:Rt =Rd +RgRg istheglobalcomponentofradiance(basedonallindirectilluminationofP).Rt isthetotalradiance.By measuring R and R then we can compute R as R R

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 18

    BymeasuringRt andRg thenwecancomputeRd asRt Rg

  • ResultsofSeparation

    Byusingstructuredlight(projectionofcheckerboards),itisnotnecessaryto have a separate shadow image for each pixel. In theory, only 2 imagestohaveaseparateshadowimageforeachpixel.Intheory,only2imagesareneeded,butinpracticeitsbesttohave10to20images,accordingtoKrishnanandNayar.

    Rt Rd Rg

    In this example most of the radiance from global illumination is due toInthisexample,mostoftheradiancefromglobalilluminationisduetointerreflectance amongtheeggs.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 19

  • ResultsofSeparation

    AscenewithgrapesandcheeseMuch of the natural color (as seen in R ) comes from R and not RMuchofthenaturalcolor(asseeninRt )comesfromRg andnotRd .

    Rt Rd Rg

    Here,muchoftheradiancefromglobalilluminationisduetolightpassingthroughthetranslucentgrapesorcheese.Thecheese,shownwithdirectilluminationonly,issomewhatunappetizing.Thegrapesalsoappearblue.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography

    Applications:imageenhancement,imageanalysis.20

  • OtherDevelopments

    3Dcameras cameratakestwoormoreshotsofsamescene,then builds 3D model of scenethenbuilds3Dmodelofscene.Redeyereduction.Alreadystandardistheautomaticuseofapreflashtoconditionthepupilsofphotosubjectstoclosedown.Redeyecanalsobereducedusingfacedetectionmethodsandspecializedalgorithms.Light field cameras By capturing a sample of the complete lightLightfieldcameras.Bycapturingasampleofthecompletelightfieldattheviewpoint,operationssuchasfocusingonspecificobjectscanbeperformedlater.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 21

  • OtherDevelopments(cont.)

    IncameraHDR.Automaticexposurebracketingisleadingtheway to built in HDR capturewaytobuiltinHDRcapture.Motionblurinversion.Thecameracanestimateitsmotionusingacombinationofaccelerometerdataandvideodata.Themotioninfocanthenbeusedtopartiallyinvertthemotionblur.Allsmilessnapshots.Usingfacedetection,smiledetectionalgorithms and inpainting techniques it is possibly to synthesizealgorithms,andinpainting techniques,itispossiblytosynthesizeanimageofagroupofpeopleinwhicheveryoneissmiling,eventhoughineachcomponentshot,therewassomeonenotsmiling.

    Pixels,Numbers,andPrograms;S.Tanimoto ComputationalPhotography 22

Recommended

View more >