cs123 | introduction to computer...

CS123|INTRODUCTIONTOCOMPUTERGRAPHICS

Andries van Dam© 10/3/2017

Image Processing & AntialiasingPart IV (Scaling)

1/51



} Images&Hardware} ExampleApplications} Jaggies&Aliasing} Sampling&Duals} Convolution} Filtering} Scaling} Reconstruction} Scaling,continued} Implementation

Outline

2/51



} Theorytellsus:} Convolvingwithsinc inthespatialdomainisalow-passfilterthatoptimallyfiltersouthigh

frequenciestomeettheNyquistcriterion} Samplingbymultiplyingwithaunitcombdoesn’twork– needtoconvolveinspatialdomain

withDiracdeltacomb} ButsamplingwithDiraccombproducesinfinitelyrepeatingspectra} AndtheywilloverlapandcauseevenmorecorruptionifsamplingatorbelowNyquistlimit

} Assumingweuseanadequatesamplingrateandpre-filtertheunrepresentable frequencies,wecanuseareconstructionfiltertoeliminaterepeatingspectraandgiveusagoodapproximationtotheoriginalcontinuoussignal…Notethiswon’thelpuscompletelyiftherealworldhasedges(e.g.,blackbuildingagainstbluesky)} Whichwecanthenscaleorotherwisetransform,andthenre-samplecomputationally

} ActualsamplingwithaCCDarraycompoundsproblemsinthatitisunweightedareasampling(boxfiltering),whichproducesacorruptedspectrum– samplingreplicatesit

} Solutionisfilteringtodealaswellaspossiblewithalltheseproblems– filterscanbecombined,aswe’llsee

Review

3/51



} Onceagain,considerthefollowinglineofpixelvalues:

} Aswesaw,ifwewanttoscaleupbyanyrationalnumbera,wemustsampleevery1/a pixelintervalsinthesourceimage

} Havingshownqualitativelyhowvariousfilterfunctionshelpusresample,let’sgetmorequantitative:showhowonedoesconvolutioninpractice,using1Dimagescalingasdrivingexample

1D Image Filtering/Scaling Up Again

4/51



} Callcontinuousreconstructedimageintensityfunctionh(x).Ifreconstructedbytrianglefilter,itlooksasbefore:

} Togetintensityofintegerpixelkin1.5Xscaleddestinationimage,h’(x),samplereconstructed(i.e.,filteredsource)imageh(x) atpoint

} Therefore,intensityfunctiontransformedforscalingis:

Resampling for Scaling Up (1/2)

5/51

)(xh

5.1kx =

h )(= 5.1)( kkh’

Note:Herewestartsamplingatthefirstpixel– slide21showsaslightlymoreaccuratealgorithmwhichstartssamplingtotheleftofthefirstpixel



} Asbefore,tobuildtransformedfunctionh’(k),takesamplesofh(x) atnon-integerlocations.

Resampling for Scaling Up (2/2)

6/51

h’(x)

h(x)

h(k/1.5)

reconstructedwaveform

plotitontheintegergrid

sampleitat15realvalues



} Previousslideshowsscalingupbyfollowingconceptualprocess:} reconstruct(byfiltering)originalcontinuousintensityfunctionfromdiscretenumberofsamples

} resamplereconstructedfunctionathighersamplingrate} stretchourinter-pixelsamplesbackintointeger-pixel-spacedrange

} Alternateconceptual approach:wecanchangewhenwescaleandstillgetsameresultbyfirst stretchingoutreconstructedintensityfunction,thensamplingitatintegerpixelintervals

Resampling for Scaling Up: An Alternate Approach (1/3)

7/51



Resampling for Scaling Up : An Alternate Approach (2/3)

8/51

} Thisnewmethodperformsscalinginsecondstepratherthanthird:stretchesoutthereconstructedfunctionratherthanthe samplelocations} reconstructoriginalcontinuousintensityfunctionfromdiscretenumberofsamples

} scaleupreconstructedfunctionbydesiredscalefactor} samplereconstructedfunctionatintegerpixellocations



} Alternateconceptualapproach(comparetoslide6);practically,we’lldobothstepsatthesametimeanyhow

Resampling for Scaling Up : An Alternate Approach (3/3)

9/51

scaledupreconstructedwaveform

h’(x)

h(x/1.5)

h(x)reconstructedwaveform

plotitontheintegergrid



Whyscalingdownismorecomplexthanscalingup} Trysameapproachasscalingup

} reconstructoriginalcontinuousintensityfunctionfromdiscretenumberofsamples,e.g.,15samplesinsource

} scaledownreconstructedfunctionbydesiredscalefactor,e.g.,3} samplereconstructedfunction(now3timesnarrower),e.g.,5samplesforthiscase,at

integerpixellocations,e.g.,0,3,6,9,12} Unexpected,unwantedsideeffect:bycompressingwaveforminto1/3itsoriginalinterval,spatialfrequenciestripled,whichextends(somewhat)band-limitedcorruptedspectrum(boxfilteredduetoimagingprocess)byfactorof3infrequencydomain.Can’tdisplaythesehigherfrequencieswithoutevenworsealiasing!

} Backtolowpassfilteringagaintolimitfrequencyband} e.g.,beforescalingdownby3,low-passfiltertolimitfrequencybandto1/3itsnewwidth

Scaling Down (1/6)

10/51



} Simplesinewaveexample} Firstwestartwithsinewave:

} 1/3Compressionofsinewaveandexpansionoffrequencyband:

} Getridofnewhighfrequencies(onlyonehere)withlow-passfilterinfrequencydomain(ideallysinc,butevenconeorpyramidwilldo;trianglein1D)

} Onlylowfrequencieswillremain– herenosignalwouldremain

Scaling Down (2/6)

11/51

Asineinthespatialdomain…isaspikeinthefrequencydomain

Signalcompressioninthespatialdomain…equalsfrequencyexpansioninthefrequencydomain

cuts out high frequencies(approx.)box in thefrequencydomain



Scaling Down (3/6)

12/51

c) Low pass filtered again, band-limited reconstructed signal

e) Scaled-down signal convolved with Dirac comb – replicas overlap badly and low-pass filtering will still have bad aliases

d) Low-pass filtered, reconstructed, scaled-down signal before re-sampling

b) Resampled filtered signal – convolution of spectrum with delta comb produces replicas

a) Low-pass filtered, reconstructed , scaled-up signal before re-sampling

} Sameproblemforacomplexsignal(showninfrequencydomain)

} a)– c)upscaling,d)– e)downscaling

} Ifshrinksignalinspatialdomain,moreactivityinasmallerspace,whichincreasesspatialfrequencies,thuswideningfrequencydomainrepresentation



} Revised(conceptual)pipelineforscalingdownimage:} reconstruction filter:low-passfiltertoreconstructcontinuousintensityfunctionfromoldscanned(box-filteredandsampled)image,alsogetsridofreplicatedspectraduetosampling(convolutionofspectrumw/adeltacomb)– thisissameasforscalingup

} scaledownreconstructedfunction} scale-down filter:low-passfiltertogetridofhighfrequenciesthatwouldleadtooverlappingspectrainthescaled-downfunction

} samplescaledreconstructedfunctionatpixelintervals– againweintroducespectrumreplicas!!!} Hardware nowreconstructsthatsetofdiscretesampleswithacontinuumfunctiondefinedacrossthescreen,withdiscontinuitiesateachh/wpixel’sedges.Thisisequivalenttoconvolvingwithaboxfilter,whichismultiplyingbysinc inthefrequencydomainandthereforedoes(mostly)attenuateallthereplicas.TheHVSfortunatelydoesadditionalfiltering!

} Wemustfilteringexplicitly twice(afterscannerimplicitlyboxfiltered)} firsttoreconstructsignalandeliminatereplicasduetosampling(filterg1)} thentolow-passfilterhighfrequenciesinscaled-downversion(filterg2)

Scaling Down (4/6)

13/51



} Inactualimplementation,wecancombinereconstructionandfrequencyband-limitingintoonefilteringstep.Why?

} Associativityofconvolution:

} Convolveourreconstructionandlow-passfilterstogetherintoonecombinedfilter!} Resultissimple:convolutionoftwosincfunctionsisjustthelargersincfunction.Inourcase,approximatelargersincwithlargertriangle,andconvolveonlyoncewithit.

} Theoreticaloptimal supportforscalingupis2,butfordown-scalingbya is2/a,i.e.,>2} Whydoessupport>2fordown-scalingmakesensefromaninformation-preservingPoV?

Scaling Down (5/6)

14/51



Whydoescomplex-soundingconvolutionoftwodifferently-scaledsinc filtershavesuchsimplesolution?}Convolutionoftwosinc filtersinspatialdomainsoundscomplicated,butrememberthatconvolutioninspatialdomainmeansmultiplicationinfrequencydomain!}Dualofsinc inspatialdomainisboxinfrequencydomain.Multiplicationoftwoboxesiseasy— productisnarroweroftwopulses:

}Narrowerpulseinfrequencydomainiswidersinc inspatialdomain(lowerfrequencies)}Thus,insteadoffilteringtwice(onceforreconstruction,onceforlow-pass),justfilteroncewithwider oftwofilterstodothesamething

} Trueforsinc ortriangleapproximation—itisthewidth ofthesupportthatmatters

Scaling Down (6/6)

15/51



} Images&Hardware} ExampleApplications} Jaggies&Aliasing} Sampling&Duals} Convolution} Filtering} Scaling} Reconstruction} Scaling,continued} Implementation

Outline

16/51



} Sofartextual explanations;let’sgetalgebraic!} Letf’(x) bethetheoretical,continuous,box-filtered(thuscorrupted)versionoforiginalcontinuousimagefunctionf(x)} producedbyscannerjustpriortosampling

} Reconstructed,doubly-filteredimageintensityfunctionh(x) returnsimageintensityatsamplelocationx,wherex isreal(anddeterminedbybackmapping usingthescalingratio);itisconvolutionoff’(x)withfilterg(x)thatisthewiderofthereconstructionandscalingfilters, centeredatx :

} Butwedodiscreteconvolution,andregardlessofwhereback-mappedxis,onlylookatnearbyintegerlocationswherethereareactualpixelvalues

} Noteagainthescalefactordeterminesnumberandlocationsofsamplepointsinsourceimageviabackmapping,basedonnumberofpixelsineachdestinationimagescanline

Algebraic Reconstruction (1/2)

17/51



} Onlyneedtoevaluatethediscreteconvolutionatpixellocationssincethat'swherethefunction’svaluewillbedisplayed

} Replaceintegralwithfinitesumoverpixellocationscoveredbyfilterg(x) centeredatx} Thusconvolutionreducesto:

} h(x)isacontinuousfunction,forallrealx} Note:signofargumentofg doesnotmattersinceourfiltersaresymmetric,e.g.,triangle} E.g.,ifx =13.7,andatrianglefilterhasoptimalscale-upsupportof2,evaluateg(13.7-13) =0.3andg(13.7-14) =0.7andmultiplythoseweightsbythevaluesofpixels13and14respectively

Algebraic Reconstruction (2/2)

18/51

Filter value at pixel location i, iterate i around x until g is 0

Pixel value at iFor all pixels i falling under filter support centered at x

h(x) = Pig(x− i)i∑



} Scalinguphasconstantreconstructionfilter,support=2} Scalingdownhassupport2/awhere a isthescalefactor,afractionlike1/3} Canparameterizeimagefunctionswithscale:writeageneralizedformulaforscalingupanddown} g(x,a) isparameterizedfilterfunction;

} Thefilterfunctionnowdependsonthescalefactor,a,becausethesupportwidthandfilterweightschangewhendownscalingbydifferentamounts.(g doesnotdependonawhenup-scaling.)

} h(x,a)isreconstructed,filteredintensityfunction(eitheridealcontinuous,ordiscreteapproximation)

} h’(k,a)isscaledversionofh(x,a) dealingwithimagescaling,sampledatpixelvaluesofx=k

Unified Approach to Scaling Up and Down

19/51



} Inordertohandleedgecasesgracefully,weneedtohaveabitofinsightaboutimages,specificallytheintervalaroundthem.

} Supposewesampleateachintegermarkonthefunctionbelow.

} Considertheintervalaroundeachsamplepoint.i.e.,theintervalforwhichthesamplerepresentstheoriginalfunction.Whatshoulditbe?

} Eachsample’sintervalmusthavewidthone} Noticethatthisintervalextendspastthelowerandupperindices(0and4)} ForafunctionwithpixelvaluesP0,P1,…,P4,,thedomainisnot [0,4],but[-0.5,4.5].} Intuition:Eachpixel“owns”aunitintervalaroundit.PixelP1,owns[0.5,1.5]

Image intervals

20/51

01234-.5.51.52.53.54.5



} Whenweback-mapwewant:} startofthedestinationintervalà startofsourceinterval(easiesttoseeifP0 liesat(0,0))} endofdestinationintervalà endofthesourceinterval

} Thequestionthenis,wheredoweback-mappointswithinthedestinationimage?} wewanttheretobealinearrelationshipinourback-map:} havetwoboundaryconditions:

Correct back-mapping (1/2)

21/51

Source

Destination

-.5

-.5 k-1+.5

P0…

Pk-1

q0 qm-1…

P1 P2

q1 q2

k =sizeofsourceimagem =sizeofdestinationimage


Andries van Dam©

} Solvingthesystemofequationsgiveus…

Correct back-mapping (2/2)

22/51

f (−.5) = −.5 5.5.1)5.1( -=+-=+- kkmff (x) = bx + c

k =sizeofsourceimagem =sizeofdestinationimagea=scalingratio,e.g.,1/3,fordownscaling

k −.5= (m−.5)b+ c− −.5= −.5b+ c k =mb

b = km=

1a

c = .5b−.5= .5(1a−1) = 1− a

2a

f (x) = xa+

1− a2a

, where a = mk

Thisequationspecswheretocenterthefilterinsourcewhenscalinganimage.Notex/awastheoriginalresult,correctedbyatermduetotheedgeconditions.Doesnotsolvetheproblemofhavingtorenormalizetheweightswhenfiltergoesofftheedge.

substitutesecondboundaryconditionf(m-1+.5)

substitutefirstboundaryconditionf(-.5)

subtractthetwo

substitutefirstboundaryconditionf(-.5)

10/3/2017


Andries van Dam©

} Justasfilterisacontinuousfunctionofx anda (scalefactor),sotooisthefilteredimagefunctionh(x,a)

} Back-mapdestinationpixelatk to(non-integer)sourcelocation :

} Canalmostwritethissumoutascodebutstillneedtofigureoutsummationlimitsandfilterfunction

Reconstruction for Scaling

23/51

For all pixels i where i is in support of g Pixel at integer i

Filter g, centered at sample point x, evaluated at I where g != 0

å -=i

i axigPaxh ) ,() ,(

ka+1− a2a

10/3/2017



} Nomenclaturesummary:} f’(x)isoriginal,mostlyband-limited,continuousintensityfunction– neverproducedinpractice!} Pi issampled(box-filtered,unitcombmultiplied)f’(x) storedaspixelvalues} g(x,a) isparameterizedfilterfunction,widerofthereconstructionandscalingfilters,removingbothreplicasdueto

samplingandhigherfrequenciesduetofrequencymultiplicationifdownscaling} h(x,a) isreconstructed,filteredintensityfunction(eitheridealcontinuousordiscreteapproximate)} h’(k,a) isscaledversionofh(x,a) dealingwithimagescaling} aisscalefactor} kisindexofapixelinthedestinationimage

} Incode,youwillbestartingwithPi (inputimage)anddoingthefilteringandmappinginonesteptogeth’(x,a),theoutputimage

Nomenclature Summary

24/51

CCD Samplingf(x)

Store as discrete pixels

f’(x)

Filter with g(x,a) to remove replicas

and reconstruct

PiScale to desired

sizeh(x,a)

Resample scaled

function and output

h’(k,a)



} Trianglefilter,modifiedtobereconstructionforscalingbyfactorofa:} fora >1,looksjustliketheoldtrianglefunction.Supportis2andtheareais1– averageatmost2pixels

} Fora <1,it’sverticallysquashedandhorizontallystretched.Supportis2/a andtheareaagainis1.

} Careful…} thisfunctionwillbecalledalot.Canyouoptimizeit?

} remember:fabs() isjustfloatingpointversionofabs()

Two for the Price of One (1/2)

25/51

-Max(1/a,1)

Min(a,1)

Max(1/a,1)


Andries van Dam©

} Thepseudocodetellsussupportofg} a <1:(-1/a)≤x≤(1/a)} a ≥1:-1≤x≤1

} Candescribeleftmostandrightmostpixelsthatneedtobeexaminedforpixelk,thedestinationpixelindex,insourceimageasdelimitingawindowaroundthecenter,.Windowissize2forscalingupand2/a forscalingdown.

} 𝑐 = $%+'(%

)%isnot,ingeneral,aninteger.Yetneedintegerindexforthepixelarray.Usefloor()

andceil().

} Ifa >1(scaleup)

} Ifa <1(scaledown)

Two for the Price of One (2/2)

c+1/a

Scale down

c - 1/a

Scale up

c - 1 c+ 1c

left = ceil( c – 1)

right = floor( c + 1)

_

_left = ceil( c – )1a

right = floor(c + )1a

ka+1− a2a

10/3/2017 26/51



} Toponder:Whendon’t youneedtonormalizesum?Why?Howcanyouoptimizethiscode?

} Remembertoboundscheck!

Triangle Filter Pseudocode

27/51

double h-prime(int k, double a) {

double sum = 0, weights_sum = 0;

int left, right;

float support;

float center= k/a + (1-a)/(2*a);

support = (a > 1) ? 1 : 1/a; //ternary operator

left = ceil(center – support);

right = floor(center + support);

for (int i = left; i <= right, i++) {

sum += g(i – center, a) * orig_image.Pi;

weights_sum += g(i – center, a);

}

result = sum/weights_sum;

}



} Foreachpixelindestinationimage:} determinewhichpixelsinsourceimagearerelevant} byapplyingtechniquesdescribedabove,usefilter-weightedvaluesofsourceimagepixelstogeneratevalueofcurrentpixelindestinationimage

The Big Picture, Algorithmically Speaking

28/51



} Noticeinpseudocodethatwesumfilterweights,thennormalizesumofweightedpixelcontributionsbydividingbyfilterweightsum.Why?

} Becausenon-integerwidthfiltersproducesumsofweightswhichvaryasafunctionofsamplingposition.Whyisthisaproblem?} “Venetianblinds”– sumsofweightsincreaseanddecreaseawayfrom1.0regularlyacrossimage.

} These“bands”scaleimagewithregularlyspacedlighteranddarkerregions.} Firstwewillshowexampleofwhyfilterswithintegerradiidosumto1andthenwhyfilterswithrealradiimaynot

Normalizing Sum of Filter Weights (1/5)

29/51



} Verifythatinteger-widthfiltershaveweightsthatalwayssumtoone:noticethatasfiltershifts,oneweightmaybelowered,butithasacorrespondingweightonoppositesideoffilter,aradiusapart,thatincreasesbysameamount


30/51

Considerourfamiliartrianglefilter

Whenweplaceitdirectlyoverapixel,wehaveoneweight,anditisexactly1.0.Therefore,thesumofweights(bydefinition)is1.0

Whenweplacethefilterhalfwaybetweentwopixels,wegettwoweights,each0.5.Thesymmetryofpixelplacementensuresthatwewillgetidenticalvaluesoneachsideofthefilter.Thetwoweightsagainsumto1.0

Ifweslidethefilter0.25unitstotheright,wehaveeffectivelyslidthetwopixelsunderitby0.25unitstotheleftrelativetoit.Sincethepixelsmovebythesameamount,anincreaseononesideofthefilterwillbeperfectlycompensatedforbyadecreaseontheother.Ourweightsagainsumto1.0.



} Butwhenfilterradiusisnon-integer,whichcaneasilyhappenfordown-scaling,thesumofweightschangesfordifferentfilterpositions

} Inthisexample,firstpositionfilter(radius2.5)atlocationA.Intersectionofdottedlineatpixellocationwithfilterdeterminesweightatthatlocation.NowconsiderfilterplacedslightlyrightofA,atB.

} Differencesinnew/oldpixelweightsshownasadditionsorsubtractions.Becausefilterslopesareparallel,thesedifferencesareallsamesize.Butthereare3negativedifferencesand2positive,hencetwosumswilldiffer


31/51



} Whenradiusisaninteger,contributingpixelscanbepairedandcontributionfromeachpairisequal.Thetwopixelsofapairareataradiusdistancefromeachother

} Proof:seeequationforvalueoffilterwithradiusr centeredatnon-integerlocationc:

} Supposepairis(b,d)asinfiguretoright.Contributionsumbecomes:} (Note|c– d|=x and|c– b|=yandx+y=r)


32/51



} Sumoffilterweightsattwopixelsinapairdoesnotdependond (locationoffiltercenter),sincetheyareataradiusdistancefromeachother

} Forintegerwidthfilters,wedonot needtonormalize

} Whenscalingup,wealwayshaveinteger-widthfilter,sowedon’t needtonormalize!

} Whenscalingdown,ourfilterwidthisgenerallynon-integer,andwedoneedtonormalize.

} Canyourewritethepseudocodetotakeadvantageofthisknowledge?


33/51



} Weknowhowtodo1Dscaling,buthowdowegeneralizeto2D?

} Doitin2D“allatonce”withonegeneralizedfilter} Hardertoimplement} Moregeneral} Generallymore“correct”– dealswithhighfrequency“diagonal”information

} Doitin1Dtwice– oncetorows,oncetocolumns} Easytoimplement} Forcertainfilters,worksprettydecently} Requiresintermediatestorage

} What’sthedifference?1Diseasier,butisitalegitimatesolution?

Scaling in 2D – Two Methods

34/51



} The1Dtwo-passmethodandthe2Dmethodwillgivethesameresultifandonlyif thefilterkernel(pixelmask)isseparable

} Aseparablekernelisonethatcanberepresentedasaproductoftwovectors.Thosevectorswouldbeyour1Dkernels} Mathematically,amatrixisseparableifitsrank(numberoflinearlyindependentrows/columns)is1

} Examples:box,Gaussian,Sobel(edgedetection),butnot coneandpyramid

} Fornon-separablefilters,thereisnowaytosplita2Dfilterinto21Dfiltersthatwillgivethesameresult

Digression on Separable Kernels (1/2)

35/51



} FortheFilterassignment,the1Dtwo-passapproachsufficesandiseasiertoimplement.Itdoesnotmatterwhetheryouapplythefilterinthexorydirectionfirst.

} Recallthatideallyweuseasinc forthelowpassfilter,butcan’tinpractice,souse,say,pyramidorGaussian.} Pyramidisnotseparable,butGaussianis

} Two1Dtrianglekernelswillnotmakeasquare2Dpyramid,butitwillbeclose} Ifyoumultiply[0.25,0.5,0.25]T *[0.25,0.5,0.25],yougetthekernelonslide37,which

isnot apyramid– thepyramidwouldhaveidenticalweightsaroundtheborder!Seealsonextslide…

} Feelfreetouse1DtrianglesasanapproximationtoanapproximationinFilter

Digression on Separable Kernels (2/2)

36/51



} Notthesame,butcloseenoughforareasonableapproximation

Pyramid vs. Triangles

37/51

2DPyramidkernel 2Dkernelfromtwo1Dtriangles



BoxGaussian

http://www.dspguide.com/ch24/3.htmPSFisthePointSpreadResponse,sameasyourfilterkernel

Examples of Separable Kernels

38/51



} Whyisfilteringtwicewith1Dfiltersfasterthanoncewith2D?} ConsideryourimagewithsizeW xH anda1DfilterkernelofwidthF

} Yourequivalent2DfilterwillhaveasizeF2} Withyour1Dfilter,youwillneedtodoFmultiplicationsandaddsperpixelandrun

throughtheimagetwice(e.g.,firsthorizontally(savedinatemp)andthenvertically)} About2FWH calculations

} Withyour2Dfilter,youneedtodoF2 multiplicationsandaddsperpixelandgothroughtheimageonce} RoughlyF2WH calculations

} Usinga1dfilter,thedifferenceisabout2/Ftimesthecomputationtime} Asyourfilterkernelsizegetslarger,thegainsfromaseparablekernelbecomemoresignificant!(atthecostofthetemp,butthat’snotanissueformostsystemsthesedays…)

Why is Separable Faster?

39/51



} Certainmappingoperations(suchasimageblurring,sharpening,edgedetection,etc.)changevaluesofdestinationpixels,butdon’tremappixellocations,i.e.,don’tsamplebetweenpixellocations.Theirfilterscanbeprecomputedasa“kernel”(or“pixelmask”)

} Othermappings,suchasimagescaling,requiresamplingbetweenpixellocationsandthereforecalculatingactualfiltervaluesatthosearbitrarynon-integerlocations,possiblyrequiringre-normalizationofweights.Fortheseoperations,ofteneasiertoapproximatepyramidfilterbyapplyingtrianglefilterstwice,oncealongx-axisofsource,oncealongy-axis

Digression on Precomputed Kernels

40/51



} Filterkernelisanarrayoffiltervaluesprecomputedatpredefinedsamplepoints} Kernelsareusuallysquare,oddnumberbyoddnumbersizegrids(centerofkernelcanbeatpixelthat

youareworkingwith[e.g.3x3kernelshownhere]):

} Whydoesprecomputationonlyworkformappingswhichsampleonlyatintegerpixelintervalsinoriginalimage?} Iffilterlocationismovedbyfractionofapixelinsourceimage,pixelsfallunderdifferentlocationswithinfilter,

correspondtodifferentfiltervalues.} Can’tprecomputeforthissinceinfinitelymanynon-integervalues

} Sincescalingwillalmostalwaysrequirenon-integerpixelsampling,youcannotuseprecomputedkernels.However,theywillbeusefulforimageprocessingalgorithmssuchasedgedetection

Precomputed Filter Kernels (1/3)

41/51

1/16 2/16 1/16

2/16 4/16 2/16

1/16 2/16 1/16



Evaluatingthekernel} Toevaluate,placekernel’scenteroverintegerpixellocationtobesampled.Eachpixelcoveredbykernelismultipliedbycorrespondingkernelvalue;resultsaresummed

} Note:havenotdealtwithboundaryconditions.Onecommontacticistoactasifthereisabufferzonewheretheedgevaluesarerepeated


42/51



Filterkernelinoperation} Pixelindestinationimageisweightedsumofmultiplepixelsinsourceimage


43/51



or they can be taken at random locations

samples can be taken in grid around filter center…

Center of current pixel

} Anti-aliasingofprimitivesinpractice} BadOldDays:Generatelow-resimageandpost-filterthewholeimage,e.g.withpyramid– blursimage(withitsaliases– evenifsingleimagelooksok,getbad temporal crawlies)

} Alternative:super-sampleandpost-filter,toapproximatepre-filteringbeforesampling} Pixel’svaluecomputedbytakingweightedaverageofseveralpointsamplesaroundpixel’scenter.Again,

approximating(convolution)integralwithweightedsum} Stochastic(random)pointsamplingasanapproximationconvergesfasterandismorecorrect(onaverage)

thanequallyspacedgridsampling

Supersampling for Image Synthesis (1/2)

44/51

Pixel at row/column intersection



} Whydoessupersamplingwork?} Samplingahigherfrequencypushesthereplicasapart,andsinceamplitudeofspectrafalloffat

approximately1/fp for(1<p <2)(i.e.somewherebetweenlinearlyandquadratically),thetailsoverlapmuchless,causingmuchlesscorruptionbeforethelow-passfiltering

} Withfewerthan128distinguishablelevelsofintensity,beingoffbyonestepishardlynoticeable} Stochasticsamplingmayintroducesomerandomnoise,butmakingmultiplepassesandaveraging,itwilleventuallyconvergeoncorrectanswer

} Sinceneedtotakemultiplesamplesandfilterthem,thisprocessiscomputationallyexpensive(butworthit,especiallytoavoidtemporalcrawlies)

Supersampling for Image Synthesis (2/2)

45/51



} Ironically,currenttrendsingraphicsaremovingbacktowardanti-aliasingasapostprocessingstep

} AMD’sMLAA(MorphologicalAnti-Aliasing)andNVIDIA’sFXAA(FastApproximateAnti-Aliasing)plusmanymore

} Generalidea:findedges/silhouettesintheimage,slightlyblurthoseareas} Fasterandlowermemoryrequirementscomparedtosupersampling} Scalesbetterwithlargerresolutions} Comparedtojustplainblurfilteringwithanyapproximationtosinc,looksbetterduetointelligentlyfilteringalongcontoursintheimage.Thereismorefilteringinareasofbadaliasingwhilestillpreservingcrispness

Modern Anti-Aliasing Techniques – Postprocessing

46/51



} Howdowefindedgestoblur?} FXAAworksthroughmultiplescreen-spacepasses

} Everypixelrunsthroughacontrasttesttoseeifitisaliased} Ifpixelpasses,it’sassignedanedgedirectionbasedofdirectionofhighest

contrast} Lookingatneighboringpixelsorthogonaltothisdirection,selectthepixelwith

highestabsolutecontrast(i.e.,contrastwithaverageofneighbors)} Alongthisedgedirection,ifthehighestcontrastpairhasalargechangein

luminance(beyondathreshold),endofedgeisreached} Usingtheorthogonaldirectionatendofedge,shifttheimagetoreducesubpixelaliasing

} Finally,blurneighbors

Modern Anti-Aliasing Techniques – FXAA

47/51



MLAA Example

48/51



MLAA vs Blur

49/51

NoAnti-Aliasing PlainBlur MLAA



MLAA vs Blur

50/51

NoAnti-Aliasing PlainBlur MLAA


Andries van Dam©

}Givenamovingobject,ifthesamplingfrequency(i.e.FPS)istoolow,willexperiencetemporalcrawlies}Temporalfilteringandsupersampling canreducecrawliesinsceneswithmotion

} Alsoreducessmallartifacts(e.g.,linealiasing)instillframes} NVIDIA’sTXAAisahardwareacceleratedversionoftemporalantialiasing,oftenproviding

comparableresultstospatialantialiasingtechniques

Modern Anti-Aliasing Techniques - Temporal AA

51/51Courtesyofhttps://de45xmedrsdbp.cloudfront.net/Resources/files/TemporalAA_small-59732822.pdf[SIGGRAPH2014]

TemporalAnti-AliasinginUnrealEngine4

10/3/2017


Andries van Dam©

} nVidiaTXAA} TemporalAAinRayTracing} UnrealEngineAA} AlienIsolation/AliasIsolation} AmazonLumberyardAA

Temporal AA Examples

10/3/2017

cs123 | introduction to computer...

Documents