cs380: introduction to computer graphics sampling chapter 16...

18/05/08

1

MinH.Kim(KAIST) Foundationsof3DComputerGraphics,S.Gortler,MITPress,2012

CS380:IntroductiontoComputerGraphicsSampling

Chapter16

MinH.KimKAISTSchoolofComputing


SUMMARYTexturemapping

2

18/05/08

2


Texturemapping•  Wehavealreadyseenandusedtexture

mapping•  Inbasictexturing,wesimply‘glue’partofan

imageontoatrianglebyspecifyingtexturecoordinatesatthethreevertices.

3


Normalmapping

4

Originalmesh4Mtriangles

Simplifiedmesh500triangles

Simplifiedmeshandnormalmapping500triangles

18/05/08

3


Environmentcubemaps•  Duringtheshadingofapoint,wecantreatthe

materialatthatpointasaperfectmirrorandfetchtheenvironmentdatafromtheappropriateincomingdirection.

5


Projectortexturemapping

•  Therearetimeswhenwewishtoglueourtextureontoourtrianglesusingaprojectormodel,insteadoftheaffinegluingmodel.

6

18/05/08

4


Shadowmapping

7

•  Two-passrendering

2ndpassrendering1stpassrendering


SAMPLINGChapter16

8

Aliasing

Anti-aliasing(multi-sampling)

Anti-aliasing(super-sampling)

18/05/08

5


Sampling•  Pointsampling:

continuousvectoràdiscretepixel•  Ourscenesaredescribedwithtrianglesgivinga

continuous2Dcolorfield.•  Ourimagesaredigital/discretemadeupofagridof

dots.•  Needtomakeabridge

betweenthesetwoworlds(continuousvs.discrete).

•  Elsewewillgetsomeunnecessaryartifactscalled“aliasing”artifacts.–  Jaggies,moirepatterns,

flickering9


Sampling•  Theseoccurwhenthereistoomuchdetailtofit

inonepixel.•  Wecanmitigatetheseartifactsbyaveragingup

thecolorswithinapixel’ssquare.•  Thisiscalledanti-aliasing.

10

18/05/08

6


Twomodels•  Acontinuousimage,,isabivariate

function.–  rangeisalinearcolorspace.

•  Adiscreteimage,I[i][j],isatwo-dimensionalarrayofcolorvalues.

•  Weassociateeachpairofintegersi,j,withthecontinuousimagecoordinates

11

I(xw , yw )

xw = i and yw = j


Aliasing•  Thesimplestandmostobviousmethodtogo

fromacontinuoustoadiscreteimageisbypointsampling.

•  Toobtainthevalueofapixeli,j,wesamplethecontinuousimagefunctionatasingleintegervalueddomainlocation:

•  Thiscanresultsinunwantedartifacts.

12

I[i][j]← I(i , j)

18/05/08

7


Aliasing•  Scenemadeupofblackandwhitetriangles:jaggies

atboundaries–  Jaggieswillcrawlduringmotion

•  Iftrianglesaresmallenoughthenwegetrandomvaluesorweirdpatterns– Willflickerduringmotion

•  Theheartoftheproblem:toomuchinformationinonepixel

13


Anti-aliasing

14

•  Intuitively:thesinglesampleisabadvalue,wewouldbebetteroffsettingthepixelvalueusingsomekindofaveragevalueoversomeappropriateregion.

•  Intheaboveexamples,perhapssomegrayvalue.

18/05/08

8


Anti-aliasingsolutions•  Forexample,mathematicallythiscanbe

modeledusingFourieranalysis.– Breaksupthedataby“frequencies”andfiguresout

whattodowiththeun-representablehighfrequencies.

15


NyquistSamplingTheorem •  Itstatesthefollowing:

– “Thesamplingfrequencyshouldbeatleasttwicethehighestfrequencycontainedinthesignal.”

16

cs ff 2≥

18/05/08

9


Example •  Anoriginal,continuoussignalwithf=1Hz

17


Example •  Ifwesamplebymorethan2Hz,wecan

reconstructthesignalfunctioncorrectly

18

2Hz 3Hz

18/05/08

10


Example •  Ifwesampleby1.5Hz(<=2Hz),theremightbe

anambiguityaboutthesignalshape.

19

1.5Hz Groundtruth

≠


Anti-aliasing•  Wecanalsomodelthisasanoptimizationproblem.•  Theseapproachesleadto:

•  whereissomefunctionthattellsushowstronglythecontinuousimagevalueatshouldinfluencethepixelvalue

20

I[i][j]← I(x , y)Ω∫∫ Fi , j(x , y)dxdy

Fi , j(x , y)

[x , y]t i,j

I[i][j]

I(xw , yw )

I(x , y)←

ΩFi , jxw = i , yw = j

Fi , j(x , y)

I(x , y)

Verticalsectionview

18/05/08

11


Anti-aliasing•  Inthissetting,thefunctioniscalledafilter.–  Inotherwords,thebestpixelvalueisdeterminedby

performingsomecontinuousweightedaveragingnearthepixel’slocation.

– Effectively,thisislikeblurringthecontinuousimagebeforepointsamplingit.

– Forexamples,boxfilter,Gaussianfilter,Lanczosfilter,bilaterialfilter

21

Fi, j (x, y)


Boxfilter•  Weoftenchoosethefilterstobe

somethingnon-optimal,butthatcanmoreeasilycomputedwith.

•  Thesimplestsuchchoiceisaboxfilter,whereiszeroeverywhereexceptoverthe1-by-1squarecenterat.

•  CallingthissquarethatcoverstheareaI[i][j],wearriveat:

22

Fi, j (x, y)

Fi, j (x, y)x = i, y = j

Ωi , j

Ωi, j

I[i][j]← I(x , y)

Ωi , j∫∫ dxdy

x ⊂ xw = i , y ⊂ yw = j

I[i][j] I(xw , yw )

I(x , y)←

Ωi , j Ωi , j

18/05/08

12


Boxfilter•  Inthiscase,thedesiredpixelvalueissimplythe

averageofthecontinuousimageoverthepixel’ssquaredomain.

23


•  Eventhatintegralisnotreallyreasonabletocompute•  Instead,itisapproximatedbysomesumoftheform:

wherekindexessomesetoflocationscalledthesamplelocations,so-calledover-sampling.

•  Therendererfirstproducesa“highresolution”colorandz-buffer“image”,– wherewewillusethetermsampletorefertoeachof

thesehighresolutionpixels.

Over-sampling

24

I[i][j]← 1

nI(xk , yk )

k=1

n

∑

(xk , yk )

apixel

18/05/08

13


Over-sampling•  Then,oncerasterizationiscomplete,groupsof

thesesamplesareaveragedtogether,tocreatethefinallowerresolutionimage.

25


Super-sampling•  Ifthesamplelocationsforthehighresolution

imageformaregular,highresolutiongrid,thenthisiscalledsupersampling.

•  Wecanalsochooseothersamplingpatternsforthehighresolution“image”,– Suchlessregularpatterns(so-calledjittering)can

helpusavoidsystematicerrorsthatcanarisewhenusingthesumtoreplacetheintegral.

26

apixel

18/05/08

14


Multi-sampling•  Rendertoa“highresolution”colorandz-buffer•  Duringtherasterizationofeachtriangle,

“coverage”andz-valuesarecomputedatthissamplelevelofhighresolution.

•  Butforefficiency,thefragmentshaderisonlycalledonlyonceperfinalresolutionpixel.– Thiscolordataissharedbetweenallofthesamples

hitbythetriangleinasingle(finalresolution)pixel.

•  Oncerasterizationiscomplete,groupsofthesehighresolutionsamplesareaveragedtogether.

27


Multi-sampling•  Multisamplingcanbeaneffectiveanti-aliasing

method•  Withouttexturemapping,colorstendtovaryquite

slowlyovereachtriangle,andthustheydonotneedtobecomputedathighspatialresolution.

•  Todealwithaliasingthatoccursduringtexturemapping,wehavetheadvantageofpossessingthetextureimageinhandattheoutsetoftherenderingprocess.

•  Thisleadstospecializedtechniquessuchasmipmapping.

28

18/05/08

15


MIPmapping•  AnacronymoftheLatinphrase,multumin

parvo(muchinlittle).•  Mipmapsareprecalculatedscaledversionsofan

originaltexture.

29


MIPmapping•  Observethepatternofeachmipmapasit

decreasesinsize;itishalfthedimensionofthepreviousone.

•  Thispatternisrepeateduntilthelastmipmap’sdimensionis1×1.

•  Interpolatemulti-resolutiontexturesviatrilinearinterpolation

30

18/05/08

16


Aliasingvs.anti-aliasing

31

Aliasing

Anti-aliasing(multi-sampling)

Anti-aliasing(super-sampling)


Camera•  Indigitalcameras,anti-aliasingisaccomplished

byacombinationofthespatialintegrationthathappensovertheextentofeachpixelsensor,

•  aswellasbytheopticalblurring(similartofilter),so-calleddiffraction,thathappensatduetothelensàpoint-spreadfunction(PSF)

•  Somecamerasalsoincludeadditionalopticalelementsspecificallytoblurthecontinuousimagedatabeforeitissampledatthesensors.

32

18/05/08

17


PrincipleofSolid-StateSensor

•  Howtocollectrainoveralargeareawithmanybucketsoffixedsize?

Electrons

Jane

sick

p.2

3

Electrons

Cells Photons


Camera:Demosaicingproblem

•  Imagineablack-on-whitecorner•  Coloraliasinghappensduetothedemosaicing

processoftheBayer-patternfilteredinputBayer-patterndemosaicing

18/05/08

18


Humanvision•  Inhumanvision,aliasingartifactsarenot

typicallyencountered.– Mostoftheanti-aliasing,atleastinthefoveal

(central)regionofvision,isduetotheopticalblurringoflight(Airydisk),whichhappenswellbeforeithitsthereceptorcells.

– Theirregularspatiallayoutofthesensorcellsintheretinaalsohelpsbyeffectivelyprovidingspatialjitter(randomness)whichturnsnoticeablealiasingintolessconspicuousnoise.

35


Imagecompositing(layering)•  Giventwodiscreteimages,aforeground,,

andbackground,,thatwewanttocombineintooneimage.

•  Simple:incomposite,useforegroundpixelswheretheyaredefined.Elseusebackgroundpixels.

•  Thiswillgiveusajaggedboundary.•  Realimagewouldhave“boundary”pixelswith

blendedcolors.•  Butthisrequiresusing“sub-pixel”information.

36

If

Ib

Ic

18/05/08

19


Imagecompositing

37


Alphablending(akamatting)•  Associatewitheachpixelineachimagelayer,a

value,,thatdescribestheoverallopacityorcoverageoftheimagelayeratthatpixel.– Analphavalueof1representsafullyopaque/

occupiedpixel,whileavalueof0representsafullytransparent/emptyone.

– Afractionalvaluerepresentsapartiallytransparent(partiallyoccupied)pixel.

•  Alphawillbeusedduringcompositing.

38

α[i][j]

18/05/08

20


Alphadefinition•  Morespecifically,letbeacontinuous

image,andletbeabinaryvaluedcoveragefunctionoverthecontinuousdomain,withavalueof1atanypointwheretheimageis“occupied”and0whereitisnot.

•  Letusstoreinourdiscreteimagethevalues:

39

I(x, y)C(x, y)

(x, y)

I[i][j]← I(x , y)

Ωi , j∫∫ C(x , y)dxdy

α[i][j]← C(x , y)dxdy

Ωi , j∫∫

x ⊂ xw = i , y ⊂ yw = j

I[i][j] I(xw , yw )

I(x , y)←

Ωi , j Ωi , j


Overoperation•  Tocompose,wecompute

thecompositeimagecolors,,using

•  Thatis,theamountofobservedbackgroundcoloratapixelisproportionaltothetransparencyoftheforegroundlayeratthatpixel.

•  Likewise,alpha(coverage)forthecompositeimagecanbecomputedas:

40

If[i][j] over Ib[i][j]Ic[i][j]

Ic[i][j]← If [i][j]+Ib[i][j](1-α f [i][j])

αc[i][j]← α f [i][j]+α b[i][j](1-α f [i][j])

18/05/08

21


Overoperation•  Ifbackgroundisopaque,sothecompositepixel

isopaque.•  Butwecanmodelmoregeneralcaseaspartof

blendingmultiplelayers.

41


Overproperties•  Thisprovidesareasonableapproximationtothe

correctlyrenderedimage.•  Onecaneasilyverifythattheoveroperationis

associativebutnotcommutative.Thatis,but,

•  NBpullingmattesfromrealimagesagainstcontrolledoresp.uncontrolledbackgroundsisanareaofresearch.àmattingalgorithmsinCV

42

Ia over ( Ib over Ic ) = (Ia over Ib ) over Ic

Ia overIb ≠ Ib overIa

18/05/08

22


•  Idealcontinuouscomposition

Alphablendingerror

43

Ic[i][j]← I f (x , y)C f (x , y)+ Ib(x , y)Cb(x , y)− Ib(x , y)Cb(x , y)C f (x , y)dxdy

Ωi , j∫∫

= I f (x , y)C f (x , y)dxdyΩi , j∫∫ + Ib(x , y)Cb(x , y)dxdy

Ωi , j∫∫

− Ib(x , y)Cb(x , y)C f (x , y)dxdyΩi , j∫∫

I[i][j]← I(x , y)

Ωi , j∫∫ C(x , y)dxdy

α[i][j]← C(x , y)dxdy

Ωi , j∫∫

x ⊂ xw = i , y ⊂ yw = j

I[i][j] I(xw , yw )

I(x , y)←

Ωi , j Ωi , j


•  Whatwedowithalphablendingis:

Alphablendingerror

44

Ic[i][j]← If [i][j]+Ib[i][j](1-α f [i][j])

= I f (x , y)C f (x , y)dxdyΩi , j∫∫

+ Ib(x , y)Cb(x , y)dxdyΩi , j∫∫⎛

⎝⎜

⎞⎠⎟ 1− C f (x , y)dxdy

Ωi , j∫∫⎛

⎝⎜

⎞⎠⎟

= I f (x , y)C f (x , y)dxdyΩi , j∫∫ + Ib(x , y)Cb(x , y)dxdy

Ωi , j∫∫

− Ib(x , y)Cb(x , y)dxdyΩi , j∫∫ ⋅ C f (x , y)dxdy

Ωi , j∫∫

18/05/08

23


Alphablendingerror•  Thedifferencebetweentwo

•  Thereforetheerroristhedifferencebetweentheintegralofaproductandaproductofintegrals.

•  Itistheamountof“correlation(covariance)”betweenthedistributionofforegroundcoverageinsomepixelandthedistributionofthebackgrounddatawithinthatpixel.

45

error= Ib(x , y)Cb(x , y)C f (x , y)dxdyΩi , j∫∫ −

Ib(x , y)Cb(x , y)dxdy× C f (x , y)dxdyΩi , j∫∫Ωi , j

∫∫

cov(X ,Y )=E[XY ]−E[X ]E[Y ]

Realalphablending

Idealcontinuouscomposition

cs380: introduction to computer graphics sampling chapter 16...

Documents