Fast Texture Synthesis using Tree-structured Vector Quantization

Li-Yi Wei MarcLevoy

StanfordUniversity�

Figure1: Our texture generation processtakesanexampletexture patch (left) anda randomnoise(middle)asinput,andmodifiesthis randomnoiseto makeit look like thegivenexampletexture. Thesynthesizedtexture (right) canbeof arbitrary size, andis perceivedasverysimilar to thegivenexample. Usingouralgorithm,texturescanbegeneratedwithin seconds,andthesynthesizedresultsarealwaystileable.

Abstract

Texture synthesisis importantfor many applicationsin computergraphics,vision, andimageprocessing.However, it remainsdiffi-cult to designanalgorithmthatis bothefficientandcapableof gen-eratinghigh quality results. In this paper, we presentan efficientalgorithmfor realistic texture synthesis.The algorithmis easytouseandrequiresonly a sampletexture as input. It generatestex-tureswith perceivedquality equalto or betterthanthoseproducedby previous techniques,but runs two ordersof magnitudefaster.Thispermitsusto applytexturesynthesisto problemswhereit hastraditionally beenconsideredimpractical. In particular, we haveappliedit to constrainedsynthesisfor imageeditingandtemporaltexturegeneration.Ouralgorithmis derivedfrom Markov RandomField texturemodelsandgeneratestexturesthrougha determinis-tic searchingprocess.We acceleratethis synthesisprocessusingtree-structuredvectorquantization.

�GatesComputerScienceBuilding, Stanford,CA 94305

Email:�liyiwei � levoy � @graphics.stanford.edu

WWW: http://graphics.stanford.edu/projects/texture/

Keywords: Texture Synthesis,CompressionAlgorithms, ImageProcessing

1 Introduction

Texture is a ubiquitousvisual experience. It candescribea widevariety of surfacecharacteristicssuchasterrain,plants,minerals,fur and skin. Sincereproducingthe visual realismof the physi-cal world is a majorgoal for computergraphics,texturesarecom-monly employedwhenrenderingsyntheticimages.Thesetexturescanbeobtainedfrom a varietyof sourcessuchashand-drawn pic-turesor scannedphotographs.Hand-drawn picturescanbeaesthet-ically pleasing,but it is hard to make themphoto-realistic.Mostscannedimages,however, areof inadequatesizeandcan lead tovisibleseamsor repetitionif they aredirectlyusedfor texturemap-ping.

Texture synthesisis an alternative way to createtextures. Be-causesynthetictexturescan be madeany size, visual repetitionis avoided. Texturesynthesiscanalsoproducetileableimagesbyproperlyhandlingthe boundaryconditions.Potentialapplicationsof texture synthesisarealsobroad;someexamplesareimagede-noising,occlusionfill-in, andcompression.

Thegoalof texturesynthesiscanbestatedasfollows: Given atexture sample,synthesizea new texture that, whenperceived bya humanobserver, appearsto begeneratedby thesameunderlyingstochasticprocess.Themajorchallengesare1) modeling-how toestimatethe stochasticprocessfrom a given finite texture sampleand2) sampling-how to developanefficientsamplingproceduretoproducenew texturesfrom a givenmodel. Both themodelingandsamplingpartsare essentialfor the successof texture synthesis:the visual fidelity of generatedtextureswill dependprimarily on

theaccuracy of themodeling,while theefficiency of thesamplingprocedure� will directlydeterminethecomputationalcostof texturegeneration.

In this paper, we presenta very simplealgorithm that canef-ficiently synthesizea wide variety of textures. The inputsconsistof an exampletexture patchanda randomnoiseimagewith sizespecifiedby the user(Figure1). The algorithmmodifiesthis ran-domnoiseto make it look likethegivenexample.Thistechniqueisflexible andeasyto use,sinceonly anexampletexturepatch(usu-ally aphotograph)is required.New texturescanbegeneratedwithlittle computationtime, andtheir tileability is guaranteed.Theal-gorithmis alsoeasyto implement;thetwo majorcomponentsareamultiresolutionpyramidandasimplesearchingalgorithm.

Thekey advantagesof this algorithmarequality andspeed:thequality of thesynthesizedtexturesareequalto or betterthanthosegeneratedby previous techniques,while the computationspeedistwo ordersof magnitudefasterthanthoseapproachesthatgeneratecomparableresultsto our algorithm. This permitsus to applyouralgorithm in areaswheretexture synthesishastraditionally beenconsideredtooexpensive. In particular, wehaveextendedthealgo-rithm to constrainedsynthesisfor imageeditingandmotiontexturesynthesis.

1.1 Previous Work

Numerousapproacheshave beenproposedfor textureanalysisandsynthesis,andanexhaustive survey is beyondthescopeof this pa-per. We briefly review somerecentandrepresentative works andreferthereaderto [8] and[12] for morecompletesurveys.

Physical Simulation: It is possibleto synthesizecertainsur-facetexturesby directly simulatingtheir physicalgenerationpro-cesses. Biological patternssuchas fur, scales,and skin can bemodeledusingreactiondiffusion [26] andcellular texturing [27].Someweatheringandmineralphenomenacanbe faithfully repro-ducedby detailedsimulations[5]. Thesetechniquescanproducetexturesdirectly on 3D meshesso the texture mappingdistortionproblemis avoided.However, differenttexturesareusuallygener-atedby very differentphysicalprocessesso theseapproachesareapplicableto only limited classesof textures.

Markov Random Field and Gibbs Sampling: Many algo-rithms model texturesby Markov RandomFields (or in a differ-entmathematicalform, GibbsSampling),andgeneratetexturesbyprobabilitysampling[6, 28,20, 18]. SinceMarkov RandomFieldshave beenproven to be a good approximationfor a broadrangeof textures, thesealgorithmsare generaland someof them pro-ducegoodresults.A drawbackof Markov RandomFieldsampling,though,is that it is computationallyexpensive: evensmall texturepatchescantake hoursor daysto generate.

Feature Matching: Somealgorithmsmodeltexturesasasetoffeatures,andgeneratenew imagesby matchingthe featuresin anexampletexture [9, 4, 22]. Thesealgorithmsareusuallymoreef-ficient thanMarkov RandomField algorithms.HeegerandBergen[9] modeltexturesby matchingmarginalhistogramsof imagepyra-mids. Their techniquesucceedson highly stochastictexturesbutfails on morestructuredones. De Bonet [4] synthesizesnew im-agesby randomizingan input texturesamplewhile preservingthecross-scaledependencies.This methodworks betterthan [9] onstructuredtextures,but it canproduceboundaryartifactsif the in-put textureis not tileable.SimoncelliandPortilla [22] generatetex-turesby matchingthe joint statisticsof the imagepyramids.Theirmethodcansuccessfullycaptureglobaltextural structuresbut failsto preserve localpatterns.

(a)

(a1) (a2)

(b)

(b1) (b2)

Figure2: Howtexturesdiffer fromimages.(a) is a general imagewhile(b)is a texture. A movablewindowwith two different positionsare drawn asblack squaresin (a) and(b), with thecorrespondingcontentsshownbelow.Different regionsof a texture are alwaysperceivedto be similar (b1,b2),which is not thecasefor a general image (a1,a2).In addition,each pixel in(b) is only relatedto a small setof neighboringpixels. Thesetwo charac-teristicsarecalledstationarityandlocality, respectively.

1.2 Overview

Ourgoalwasto developanalgorithmthatcombinestheadvantagesof previous approaches.We want it to be efficient, general,andable to producehigh quality, tileable textures. It shouldalso beuserfriendly; i.e., the numberof tunableinput parametersshouldbeminimal. Thiscanbeachievedby a carefulselectionof thetex-turemodelingandsynthesisprocedure.For thetexturemodel,weuseMarkov RandomFields (MRF) sincethey have beenprovento cover the widest variety of useful texture types. To avoid theusualcomputationalexpenseof MRFs,we have developeda syn-thesisprocedurewhichavoidsexplicit probabilityconstructionandsampling.

Markov RandomField methodsmodela textureasa realizationof a local andstationaryrandomprocess.That is, eachpixel of atextureimageis characterizedby a smallsetof spatiallyneighbor-ing pixels,andthis characterizationis thesamefor all pixels. Theintuition behindthis modelcanbe demonstratedby the followingexperiment(Figure2). Imaginethataviewer is givenanimage,butonly allowed to observe it througha small movablewindow. Asthewindow is moved theviewer canobserve differentpartsof theimage.Theimageis stationaryif, undera properwindow size,theobservable portion alwaysappearssimilar. The imageis local ifeachpixel is predictablefrom asmallsetof neighboringpixelsandis independentof therestof theimage.

Basedon theselocality andstationarityassumptions,our algo-rithm synthesizesa new texture so that it is locally similar to anexampletexturepatch.Thenew textureis generatedpixel by pixel,andeachpixel is determinedso that local similarity is preservedbetweenthe exampletextureandthe result image. This synthesisprocedure,unlikemostMRF basedalgorithms,is completelydeter-ministic andno explicit probabilitydistribution is constructed.Asaresult,it is efficientandamenableto furtheracceleration.

Theremainderof thepaperis organizedasfollows. In Section2,we presentthe algorithm. In Section3, we demonstratesynthe-sisresultsandcomparethemwith thosegeneratedby previousap-proaches. In Section4, we proposeaccelerationtechniques. InSections5 and 6, we discussapplications,limitations,andexten-sions.

: Neighborhood N

(a)

(d)(c)(b)

p

Figure3: Singleresolutiontexture synthesis.(a) is the input texture and(b)-(d) showdifferent synthesisstages of the output image. Pixels in theoutput image are assignedin a rasterscanordering. Thevalue of eachoutputpixel � is determinedby comparingits spatialneighborhood�����with all neighborhoodsin the input texture. Theinput pixel with the mostsimilar neighborhoodwill be assignedto the correspondingoutputpixel.Neighborhoodscrossingthe output image boundaries(shownin (b) and(d)) arehandledtoroidally, asdiscussedin Section2.4.Althoughtheoutputimage startsasa randomnoise, only the last few rowsandcolumnsof thenoiseare actuallyused.For clarity, wepresenttheunusednoisepixelsasblack. (b) synthesizingthefirst pixel, (c) synthesizingthemiddlepixel, (d)synthesizingthelastpixel.

2 Algorithm

UsingMarkov RandomFieldsasthetexturemodel,thegoalof thesynthesisalgorithm is to generatea new texture so that eachlo-cal region of it is similar to anotherregion from the input texture.We first describehow the algorithmworks in a single resolution,andthenweextendit usingamultiresolutionpyramidto obtainim-provementsin efficiency. For easyreference,we list the symbolsusedin Table1 andsummarizethealgorithmin Table2.

Symbol Meaning ��Input texturesample ��Outputtextureimage� �Gaussianpyramidbuilt from

��� �Gaussianpyramidbuilt from

����� An inputpixel in

�or� �

� An outputpixel in �

or� ���� ��� Neighborhoodaroundthepixel �� ��� � �

th level of pyramid�� ������� �"! � Pixel at level

�andposition

���#�$! � of�%

RxC,k& (2D) neighborhoodcontaining' levels,with sizeRxCat thetop level%

RxCxD,k& 3D neighborhoodcontaining' levels,with sizeRxCxDat thetop level

Table1: Tableof symbols

2.1 Single Resolution Synthesis

The algorithmstartswith an input texture sample ��

anda whiterandomnoise

(�. We forcetherandomnoise

��to look like

�by

transforming ��

pixel bypixel in arasterscanordering,i.e. fromtop

(a) (b) (c)

Figure4: Synthesisresultswith different neighborhoodsizes.Theneigh-borhoodsizesare (a) 5x5,(b) 7x7,(c) 9x9,respectively. All imagesshownareof size128x128.Notethatastheneighborhoodsizeincreasestheresult-ing texturequalitygetsbetter. However, thecomputationcostalsoincreases.

(a)

(b) (c)

Figure5: Causalityof theneighborhood.(a) sampletexture (b) synthesisresultusinga causalneighborhood(c) synthesisresultusinga noncausalneighborhood.Both(b) and(c) aregeneratedfromthesamerandomnoiseusinga 9x9neighborhood.Asshown,a noncausalneighborhoodis unableto generatevalid results.

to bottomandleft to right. Figure3 shows a graphicalillustrationof thesynthesisprocess.

To determinethe pixel value � at ��

, its spatialneighborhood��� ��� (the L-shapedregions in Figure3) is comparedagainstallpossibleneighborhoods

�)� �*�$� from �

. The input pixel �*� withthemostsimilar

�)� �*��� is assignedto � . We usea simple�,+

norm(sumof squareddifference)to measurethesimilarity betweentheneighborhoods.Thegoalof this synthesisprocessis to ensurethatthe newly assignedpixel � will maintainasmuchlocal similaritybetween

�and

��aspossible. The sameprocessis repeatedfor

eachoutputpixel until all thepixelsaredetermined.This is akin toputtingtogetherajigsaw puzzle:thepiecesaretheindividualpixelsandthefitnessbetweenthesepiecesis determinedby thecolorsofthesurroundingneighborhoodpixels.

2.2 Neighborhood

Becausethe setof local neighborhoods��� ���"� is usedasthe pri-

marymodelfor textures,thequality of thesynthesizedresultswilldependon its sizeandshape.Intuitively, thesizeof theneighbor-hoodsshouldbeonthescaleof thelargestregulartexturestructure;otherwisethis structuremaybelost andtheresultimagewill looktoo random.Figure4 demonstratestheeffect of theneighborhoodsizeon thesynthesisresults.

Theshapeof theneighborhoodwill directly determinethequal-ity of

��. It mustbecausal,i.e. theneighborhoodcanonly contain

thosepixels precedingthe currentoutput pixel in the rasterscanordering. The reasonis to ensurethat eachoutputneighborhood��� ��� will includeonly alreadyassignedpixels. For the first fewrowsandcolumnsof

�,�)� ��� maycontainunassigned(noise)pix-

elsbut asthealgorithmprogressesall theother�)� ��� will becom-

pletely “valid” (containingonly alreadyassignedpixels). A non-

O O

O

O

O

O OO

O O

O

Q

Q

Q Q Q

L+1

Y Q

QQ

L

O

X

Figure6: A causalmultiresolutionneighborhoodwith size�5x5,2� . The

current level of the pyramid is shownat left and the next lower resolutionlevel is shownat right. Thecurrentoutputpixel � , markedasX, is locatedat�-/.102.13�� , where - is thecurrent level numberand �02.43�� is its coordinate.At this level - of the pyramid the image is only partially complete. Thus,wemustusetheprecedingpixelsin therasterscanordering(markedasO).Thepositionof the parent of the current pixel, locatedat �-�5)67.(8 + .:9 + � ,is markedasY. Sincetheparent’s level is complete, theneighborhoodcancontainpixelsaroundY, markedbyQ. Whensearchingfor a match for pixelX, theneighborhoodvectoris constructedthat includestheO’s,Q’s,andY,in scanlineorder.

causal��� �*� , whichalwaysincludesunassignedpixels,is unableto

transform �

to look like ��

(Figure5). Thus,the noiseimageisonly usedwhengeneratingthe first few rows andcolumnsof theoutputimage.After this, it is ignored.

2.3 Multiresolution Synthesis

The singleresolutionalgorithmcapturesthe texture structuresbyusingadequatelysizedneighborhoods.However, for texturescon-taining largescalestructureswe have to uselarge neighborhoods,andlargeneighborhoodsdemandmorecomputation.Thisproblemcanbesolvedby usinga multiresolutionimagepyramid[3]; com-putationis saved becausewe can representlarge scalestructuresmorecompactlyby a few pixelsin a certainlower resolutionpyra-mid level.

The multiresolutionsynthesisalgorithm proceedsas follows.Two Gaussianpyramids,

� �and

� �, are first built from

�and �

, respectively. The algorithmthentransforms� �

from lower tohigher resolutions,suchthat eachhigher resolutionlevel is con-structedfrom thealreadysynthesizedlower resolutionlevels. Thisis similar to the sequencein which a pictureis painted: long andthick strokes areplacedfirst, anddetailsare thenadded. Withineachoutputpyramid level

� � ��� � , the pixels aresynthesizedin away similar to the singleresolutioncasewherethe pixels areas-signedin a rasterscanordering. Theonly modificationis that forthemultiresoltioncase,eachneighborhood

��� ��� containspixelsinthecurrentresolutionaswell asthosein thelower resolutions.Thesimilarity betweentwo multiresolutionneighborhoodsis measuredby computingthesumof thesquareddistanceof all pixelswithinthem.Theselower resolutionpixelsconstrainthesynthesisprocesssothattheaddedhigh frequency detailswill beconsistentwith thealreadysynthesizedlow frequency structures.

An exampleof a multiresolutionneighborhoodis shown in Fig-ure6. It consistsof two levels,with sizes5x5and3x3,respectively.Within a neighborhood,we choosethesizesof the lower levelssothat they areabouthalf thesizesof theprevious higherresolutionlevels.For clarity, weusethesymbol

%RxC,k& to indicatemultires-

(a) (b) (c)

Figure7: Synthesisresultswith thesameneighborhood,but differentnum-bers of pyramid levels(a) 1 level, (b) 2 levels,(c) 3 levels. Exceptfor thelowestresolution,which is synthesizedwith a 5x5singleresolutionneigh-borhood,each pyramidlevel is synthesizedusingthemultiresolutionneigh-borhoodshownin Figure 6. Note that as the numberof pyramid levelsincreases,theimage quality improves.

olutionneighborhoodswhichcontain ' levelswith sizeRxC at thetop level.

Figure7 showsresultsof multiresolutionsynthesiswith differentnumbersof pyramid levels. Note that Figure7 (c), althoughsyn-thesizedwith asmall

%5x5,2& multiresolutionneighborhood,looks

comparablewith Figure4 (c), which wasgeneratedwith a larger9x9singleresolutionneighborhood.Thisdemonstratesamajorad-vantageof multiresolutionsynthesis:moderatelysmall neighbor-hoodscanbeusedwithoutsacrificingsynthesisqualities.

2.4 Edge Handling

Proper edge handling for��� ��� near the image boundariesis

very important. For the synthesispyramid the edge is treatedtoroidally. In other words, if

� �;�����"� �$! � denotesthe pixel atlevel

�and position

��� �"! � of pyramid� �

, then� � �����"� �"! �=<� �>�����?�A@CBEDGFH�I!J@CBEDG� � , where

Fand

�arethe num-

berof rows andcolumns,respectively, of� �>��� � . Handlingedges

toroidally is essentialto guaranteethat the resultingsynthetictex-turewill tile seamlessly. 1

For theinputpyramid� �

, toroidalneighborhoodstypically con-taindiscontinuitiesunless

(�is tileable.A reasonableedgehandler

for� �

is to pad it with a reflectedcopy of itself. Anothersolu-tion is to useonly those

��� �*�$� completelyinside� �

, anddiscardthosecrossingthe boundaries.Becausea reflective edgehandlermay introducediscontinuitiesin the derivative, we adoptthe sec-ondsolutionwhichusesonly interiorblocks.

2.5 Initialization

Naturaltexturesoftencontainrecognizablestructuresaswell asacertainamountof randomness.Sinceour goal is to reproducere-alistic textures,it is essentialthatthealgorithmcapturetherandomaspectof the textures. This notion of randomnesscansometimesbe achieved by entropy maximization[28], but the computationalcostis prohibitive. Instead,we initialize the outputimage

�asa

white randomnoise,andgraduallymodify this noiseto look likethe input texture

�. This initialization stepseedsthe algorithm

with sufficiententropy, andletstherestof thesynthesisprocessfo-cuson the transformationof

(�towards

�. To make this random

noiseabetterinitial guess,wealsoequalizethepyramidhistogramof� �

with respectto� �

[9].

1Themultiresolutionalgorithmis alsoessentialfor tileability if acausalneighborhoodis used.Sincea singleresolutioncausalneighborhood�����containsonlypixelsabove � in scanlineorder, theverticaltileability maynotbe enforced. A multiresolutionneighborhood,which containssymmetricregionsat lower resolutionlevels,avoidsthisproblem.

The initial noiseaffects the synthesisprocessin the followingway. For thesingleresolutioncase,neighborhoodsin thefirst fewrows andcolumnsof

�containnoisepixels. Thesenoisepixels

introduceuncertaintyin theneighborhoodmatchingprocess,caus-ing the boundarypixels to be assignedsemi-stochastically(How-ever, thesearchingprocessis still deterministic.Therandomnessiscausedby the initial noise). The restof the noisepixels areover-writtendirectlyduringsynthesis.For themultiresolutioncase,how-ever, moreof thenoisepixelscontributeto thesynthesisprocess,atleastindirectly, sincethey determinetheinitial valueof thelowestresolutionlevel of

� �.

2.6 Summary of Algorithm

Wesummarizethealgorithmin thefollowing pseudocode.

function ��LK

TextureSynthesis( �

,BNM�O � M*OQP*R$SUT )

1 �� K

Initialize(BNM�O � M*OQP*R$SUT );

2� �VK

BuildPyramid( ��

);3

� �LKBuildPyramid(

��);

4 foreach level�

from lower to higherresolutionsof� �

5 loop throughall pixels��� � �"! � � of

� � ��� �6 W K

FindBestMatch(� �

,� �

,�,�"�#�;�"!*�

);7

� � ������� � ��! � � K W ;8

�� KReconPyramid(

� �);

9 return ��

;

function W KFindBestMatch(

� �,� �

,���"� �>�"!��

)1

�X�LKBuildNeighborhood(

� �N�"���"� �>�"!��);

2�GY$Z �"[� K

null; W Knull;

3 loop throughall pixels��� � �"! � � of

� � ��� �4

�C�\KBuildNeighborhood(

� �,���"� �E�"!��

);5 if Match(

�C�,� �

) ] Match(� Y$Z �"[�

,� �

)6

� Y$Z �"[� K^� �; W K � � �����"� � �"! � � ;

7 return W ;

Table2: Pseudocodeof theAlgorithm

The architectureof this algorithm is flexible; it is composedfrom severalorthogonalcomponents.We list thesecomponentsasfollowsanddiscussthecorrespondingdesignchoices.

Pyramid: The pyramids are built from and reconstructedtoimagesusing the standardroutines BuildPyramid and Recon-Pyramid. Various pyramids can be usedfor texture synthesis;examplesare Gaussianpyramids [20], Laplacianpyramids [9],steerablepyramids [9, 22], and feature-basedpyramids [4]. AGaussianpyramid, for example, is built by successive filteringanddownsamplingoperations,andeachpyramid level, exceptforthe highestresolution,is a blurred and decimatedversionof theoriginal image. Reconstructionof Gaussianpyramids is trivial,sincetheimageis availableat thehighestresolutionpyramidlevel.Thesedifferentpyramidsgive different trade-offs betweenspatialand frequency resolutions. In this paper, we chooseto use theGaussianpyramidfor its simplicity andgreaterspatiallocalization(adetaileddiscussionof this issuecanbefoundin [19]). However,otherkindsof pyramidscanbeusedinstead.

Neighborhood: The neighborhoodcan have arbitrary size andshape;theonly requirementis that it containsonly valid pixels. Anoncausal/symmetricneighborhood,for example,canbe usedbyextendingtheoriginalalgorithmwith two passes(Section5.1).

Synthesis Ordering: A rasterscanorderingis usedin line 5 of thefunction TextureSynthesis. This, however, canalsobe extended.For example,a spiralorderingcanbeusedfor constrainedtexture

synthesis(Section5.1). The synthesisorderingshouldcooperatewith the BuildNeighborhood so that the output neighborhoodscontainonly valid pixels.

Searching: An exhaustive searchingprocedureFindBestMatchis employed to determinethe outputpixel values. Becausethis isa standardprocess,variouspoint searchingalgorithmscanbeusedfor acceleration.Thiswill bediscussedin detailin Section4.

3 Synthesis Results

To test the effectivenessof our approach,we have run the algo-rithmonmany differentimagesfrom standardtexturesets.Figure8showsexamplesusingtheMIT VisTex set[16], whichcontainsrealworld texturesphotographedundernaturallighting conditions.Ad-ditional texturesynthesisresultsareavailableon our projectweb-site.

A visual comparisonof our approachwith several otheralgo-rithmsis shown in Figure9. Result(a) is generatedby HeegerandBergen’s algorithm[9] usinga steerablepyramid with 6 orienta-tions.Thealgorithmcapturescertainrandomaspectsof thetexturebut failson thedominatinggrid-likestructures.Result(b) is gener-atedby De Bonet’s approach[4] wherewe choosehis randomnessparameterto make the result look best. Thoughcapableof cap-turing morestructuralpatternsthan(a), certainboundaryartifactsarevisible. This is becausehis approachcharacterizestexturesbylower frequency pyramidlevelsonly; thereforethelateralrelation-shipbetweenpixelsat thesamelevel is lost. Result(c) is generatedby Efros andLeung’s algorithm[6]. This techniqueis basedontheMarkov RandomFieldmodelandis capableof generatinghighquality textures. However, a direct applicationof their approachcanproducenon-tileableresults.2

Result(d) is synthesizedusingour approach.It is tileableandthe imagequality is comparablewith thosesynthesizeddirectlyfrom MRFs. It took about8 minutesto generateusinga 195MHzR10000processor. However, this is not the maximumpossiblespeedachievablewith this algorithm. In the next section,we de-scribemodificationsthatacceleratethealgorithmgreatly.

4 Acceleration

Our deterministicsynthesisprocedureavoids the usualcomputa-tional requirementfor samplingfrom a MRF. However, the algo-rithm asdescribedemploys exhaustive searching,which makes itslow. Fortunately, accelerationis possible.This is achievedby con-sideringneighborhoods

��� �*� aspointsin a multiple dimensionalspace,andcastingtheneighborhoodmatchingprocessasanearest-pointsearchingproblem[17].

The nearest-pointsearchingproblemin multiple dimensionsisstatedasfollows: givenaset

Pof _ pointsandanovel querypoint`

in aD

-dimensionalspace,find apoint in thesetsuchthatits dis-tancefrom

`is lesserthan,or equalto, thedistanceof

`from any

otherpoint in theset.Becausea largenumberof suchqueriesmayneedto be conductedover thesamedataset

P, thecomputational

costcanbe reducedif we preprocessP

to createa datastructurethat allows fastnearest-pointqueries. Many suchdatastructureshavebeenproposed,andwereferthereaderto [17] for amorecom-pletereference.However, mostof thesealgorithmsassumegenericinputsanddo not attemptto take advantageof any specialstruc-turesthey mayhave. Popat[20] observed that theset

Pof spatial

neighborhoodsfrom a texture canoften be characterizedwell by

2Thoughnotstatedin theoriginalpaper[6], wehavefoundthatit is pos-sibleto extendtheirapproachusingmultiresolutionpyramidsandatoroidalneighborhoodto make tileabletextures.

(a) (b)

(c) (d)

(e) (f)

(g) (h)

(i) (j)

Figure8: Texture synthesisresults. The smallerpatches(size128x128)are the input textures,and to their right aresynthesizedresults(size200x200).Eachtextureis generatedusinga 4-level Gaussianpyramid,with neighborhoodsizes

%3x3,1& , % 5x5,2& , % 7x7,2& , % 9x9,2& ,

respectively, from lowerto higherresolutions.VisTex textures:(a)Water0000(b) Misc 0000(c) Metal0004(d) Fabric0015(e)Terrain0000(f) Clouds0000(g) Tile 0007(h) Stone0002(i) Flowers0000(j) Leaves0009.

Sample

(a) (b) (c) (d)

Figure9: A comparisonof texture synthesisresultsusingdifferent algorithms: (a) Heeger and Bergen’s method[9] (b) De Bonet’s method[4] (c) EfrosandLeung’s method[6] (d) Our method.Only EfrosandLeung’s algorithmproducesresultscomparablewith ours. However, our algorithmis two orders ofmagnitudefasterthantheirs (Section4). Thesampletexture patch hassize64x64,andall theresultimagesare of size192x192.A 9x9neighborhoodis usedfor (c), and(d) is synthesizedusingthesameparameters asindicatedin thecaptionof Figure8.

a clusteringprobability model. Taking advantageof this cluster-ing property, we proposeto usetree-structuredvectorquantization(TSVQ,[7]) asthesearchingalgorithm[25].

4.1 TSVQ Acceleration

Tree-structuredvectorquantization(TSVQ)is acommontechniquefor datacompression.It takesasetof trainingvectorsasinput,andgeneratesa binary-tree-structuredcodebook. The first step is tocomputethecentroidof thesetof trainingvectorsanduseit astheroot level codeword. To find thechildrenof this root, thecentroidanda perturbedcentroidarechosenasinitial child codewords. AgeneralizedLloyd algorithm[7], consistingof alternationsbetweencentroidcomputationandnearestcentroidpartition,is thenusedtofind thelocally optimalcodewordsfor thetwo children.Thetrain-ing vectorsaredivided into two groupsbasedon thesecodewordsandthe algorithmrecurseson eachof the subtrees.This processterminateswhenthe numberof codewordsexceedsa pre-selectedsizeor the averagecodingerror is below a certainthreshold.Thefinal codebookis thecollectionof theleaf level codewords.

Thetreegeneratedby TSVQ canbeusedasa datastructureforefficient nearest-pointqueries.To find thenearestpoint of a givenqueryvector, thetreeis traversedfrom theroot in abest-firstorder-ingbycomparingthequeryvectorwith thetwochildrencodewords,andthenfollowstheonethathasaclosercodeword. Thisprocessisrepeatedfor eachvisitednodeuntil a leafnodeis reached.Thebestcodeword is thenreturnedasthecodewordof thatleafnode.Unlikefull searching,theresultcodewordmaynotbetheoptimalonesinceonly partof the treeis traversed.However, the resultcodeword isusuallycloseto theoptimalsolution,andthecomputationis moreefficient thanfull searching.If thetreeis reasonablybalanced(thiscanbe enforcedin the algorithm),a singlesearchwith codebooksize a P a canbe achieved in time b ��c�BEd a P a � , which is muchfasterthanexhaustive searchingwith lineartimecomplexity b � a P a � .

To useTSVQ in our synthesisalgorithm,we simply collect theset of neighborhoodpixels

�)� �*��� for eachinput pixel and treatthemasa vectorof sizeequalto the numberof pixels in

�)� � � � .We use thesevectors

% ��� �*�"� & from each� � ��� � as the train-

ing data,andgeneratethecorrespondingtreestructurecodebooksef��� � . Duringthesynthesisprocess,the(approximate)closestpointfor each

��� �*� at� �;��� � is foundby doinga best-firsttraversalofef��� � . Becausethis treetraversalhastime complexity b ��c�BNd(�Xg �

(where� g

is thenumberof pixelsof� � ��� � ), thesynthesisproce-

durecanbeexecutedveryefficiently. Typical texturestakesecondsto generate;theexacttiming dependsontheinputandoutputimagesizes.

(a)D103

(b) D11

Exhaustive TSVQ

Figure10: AcceleratedsynthesisusingTSVQ.Theoriginal Brodatztex-tures,with size128x128,are shownin the left column. Theresultsgener-atedby exhaustivesearching andTSVQare shownin themiddleandrightcolumns,respectively. All generatedimagesare of size200x200.Theav-erage running time for exhaustivesearching is 360 seconds.Theaveragetraining timefor TSVQis 22 secondsandtheaverage synthesistimeis 7.5seconds.

4.2 Acceleration Results

An example comparingthe resultsof exhaustive searchingandTSVQis shown in Figure10. Theoriginal imagesizesare128x128andthe resultingimagesizesare200x200. The averagerunningtime for exhaustive searchingis 360 seconds.The averagetrain-ing time for TSVQ is 22 secondsandtheaveragesynthesistime is7.5 seconds.Thecodeis implementedin C++ andthetimingsaremeasuredona195MHzR10000processor. As shown in Figure10,resultsgeneratedwith TSVQ accelerationareroughlycomparablein quality to thosegeneratedfrom the unacceleratedapproach.Insomecases,TSVQ will generatemoreblurry images.We fix thisby allowing limited backtrackingin thetreetraversalso thatmorethanoneleaf nodecanbevisited. Theamountof backtrackingcanbeusedasaparameterwhich tradesoff betweenimagequalityandcomputationtime. Whenthenumberof visitedleaf nodesis equalto thecodebooksize,the resultwill be thesameastheexhaustivesearchingcase.

Onedisadvantageof TSVQ accelerationis thememoryrequire-

(a) (b) (c)

Figure11: TSVQacceleration with differentcodebooksizes.Theoriginalimage sizeis 64x64and all thesesynthesizedresultsare of size128x128.Thenumberof codewordsin each caseare (a) 64 (b) 512(c) 4096(all).

Algorithm TrainingTime SynthesisTimeEfrosandLeung none 1941seconds

Exhaustive Searching none 503secondsTSVQacceleration 12 seconds 12 seconds

Table3: A breakdownof runningtimefor thetexturesshownin Figure 9.Thefirst row showsthetimingof EfrosandLeung’s algorithm. Thesecondandthird rowsshowthetimingof our algorithm,usingexhaustivesearchingandTSVQacceleration, respectively. All the timingswere measuredusinga 195MHzR10000processor.

ment.Becauseaninputpixelcanappearin multipleneighborhoods,a full-sizedTSVQtreecanconsumeb ��DIhi� � memorywhere

Dis

theneighborhoodsizeand�

is thenumberof input imagepixels.Fortunately, texturesusuallycontainrepeatingstructures;thereforewe canusecodebookswith fewer codewordsthanthe input train-ing set.Figure11showstexturesgeneratedby TSVQwith differentcodebooksizes.As expectedtheimagequality improveswhenthecodebooksize increases.However, resultsgeneratedwith fewercodewordssuchas(b) look plausiblecomparedwith thefull code-bookresult(c). In our experiencewe canusecodebookslessthan10 percentthesizeof theoriginal trainingdatawithout noticeabledegradationof quality of the synthesisresults. To further reducetheexpenseof training,wecanalsotrainonasubsetratherthantheentirecollectionof inputneighborhoodvectors.

Table3 shows a timing breakdown for generatingthe texturesshown in Figure9. Ourunacceleratedalgorithmtook 503seconds.TheTSVQ acceleratedalgorithmtook12 secondsfor training,andanother12 secondsfor synthesis. In comparison,Efros and Le-ung’s algorithm[6] took half anhour to generatethesametexture(the time complexity of our approachover Efros and Leung’s isb ��c�BEd(� �kj b ��� � where

�is the numberof input imagepixels).

Becausetheir algorithm usesa variablesizedneighborhoodit isdifficult to accelerate.Our algorithm, on the other hand,usesafixed neighborhoodand can be directly acceleratedby any pointsearchingalgorithm.

5 Applications

Oneof the chief advantagesof our texturesynthesismethodis itslow computationalcost. This permitsus to explore a variety ofapplications,in additionto theusualtexturemappingfor graphics,that were previously impractical. Presentedhereare constrainedsynthesisfor imageeditingandtemporaltexturegeneration.

5.1 Constrained Texture Synthesis

Photographs,films and imagesoften contain regions that are insomesenseflawed.A flaw canbeascrambledregionon ascannedphotograph,scratcheson an old film, wires or propsin a movie

(a) (b)

(c) (d)

Figure12: Constrainedtexture synthesis.(a) a texture containinga blackregion that needsto befilled in. (b) multiresolutionblending[3] with an-othertextureregionwill produceboundaryartifacts.(c) Adirectapplicationof thealgorithmin Section2 will producevisiblediscontinuitiesat therightandbottomboundaries.(d) A much betterresultcanbegeneratedbyusinga modificationof thealgorithmwith 2 passes.

film frame,or simply anundesirableobjectin an image.Sincetheprocessescausingtheseflaws areoften irreversible,an algorithmthatcanfix theseflaws is desirable.For example,Hirani andTot-suka[10] developedaninteractive algorithmthatfindstranslation-ally similar regionsfor noiseremoval. Often,theflawedportioniscontainedwithin a region of texture,andcanbe replacedby con-strainedtexturesynthesis[6, 11].

Texture replacementby constrainedsynthesismustsatisfy tworequirements:thesynthesizedregion mustlook like thesurround-ing texture,andtheboundarybetweenthenew andold regionsmustbeinvisible. Multiresolutionblending[3] with anothersimilar tex-ture, shown in Figure12 (b), will producevisible boundariesforstructuredtextures.Betterresultscanbeobtainedby applyingouralgorithmin Section2 over theflawedregions,but discontinuitiesstill appearat the right and bottom boundariesas shown in Fig-ure12 (c). Theseartifactsarecausedby the causalneighborhoodaswell astherasterscansynthesisordering.

To remove theseboundaryartifacts a noncausal(symmetric)neighborhoodmustbeused.However, wehave to modify theorig-inal algorithmso that only valid (alreadysynthesized)pixels arecontainedwithin the symmetricneighborhoods;otherwisethe al-gorithmwill notgeneratevalid results(Figure5). Thiscanbedonewith atwo-passextensionof theoriginalalgorithm.Eachpassis thesameastheoriginalmultiresolutionprocess,exceptthatadifferentneighborhoodis used.During thefirst pass,theneighborhoodcon-tainsonly pixelsfrom thelowerresolutionpyramidlevels.Becausethe synthesisprogressesin a lower to higherresolutionfashion,asymmetricneighborhoodcanbe usedwithout introducinginvalidpixels. This passusesthelower resolutioninformationto “extrap-olate” thehigherresolutionregionsthatneedto bereplaced.In the

Original Result

Figure13: ImageextrapolationofBrodatztextureD36.Theoriginal imageis on the left andthesynthesizedresultis on theright. Theblack region isfilled in sothat it looksconsistentwith therestof theimage.

secondpass,a symmetricneighborhoodthat containspixels fromboth the currentandlower resolutionsis used. Thesetwo passesalternatefor eachlevel of the outputpyramid. In the acceleratedalgorithm, the analysisphaseis alsomodifiedso that two TSVQtreescorrespondingto thesetwo kinds of neighborhoodsarebuiltfor eachlevel of the input pyramid. Finally, we alsomodify thesynthesisorderingin thefollowing way: insteadof theusualraster-scanordering,pixels in the filled regionsareassignedin a spiralfashion. For example,the hole in Figure12 (a) is replacedfromoutsideto inside from the surroundingregion until every pixel isassigned(Figure12 (d)). This spiral synthesisorderingremovesthe directionalbiaswhich causesthe boundarydiscontinuities(asin Figure12(c)).

With aslightchangeof thesynthesisordering,thealgorithmcanbe appliedto otherapplications,suchas the imageextrapolationshown in Figure13. The algorithmcould alsobe extendedasaninteractive tool for imageeditingor denoising[15].

5.2 Temporal Texture Synthesis

The low costof our acceleratedalgorithmenablesus to considersynthesizingtexturesof dimensiongreaterthantwo. An exampleof 3D textureis a temporaltexture. Temporaltexturesaremotionswith indeterminateextentbothin spaceandtime. They candescribea wide varietyof naturalphenomenasuchasfire, smoke, andfluidmotions.Sincerealisticmotionsynthesisis oneof themajorgoalsof computergraphics,atechniquethatcansynthesizetemporaltex-tureswould be useful. Most existing algorithmsmodel temporaltexturesby directsimulation;examplesincludefluid, gas,andfire[23]. Direct simulations,however, are often expensive and onlysuitablefor specifickinds of textures;thereforean algorithmthatcanmodelgeneralmotiontextureswouldbeadvantageous[24].

Temporaltexturesconsistof 3D spatial-temporalvolumeof mo-tion data. If the motion datais local andstationaryboth in spaceandtime, the texturecanbesynthesizedby a 3D extensionof ouralgorithm.Thisextensioncanbesimplydoneby replacingvarious2D entitiesin theoriginalalgorithm,suchasimages,pyramids,andneighborhoods,with their 3D counterparts.For example,the twoGaussianpyramidsareconstructedby filtering anddownsamplingfrom 3D volumetricdata;theneighborhoodscontainlocalpixelsinboththespatialandtemporaldimension.Thesynthesisprogressesfrom lower to higher resolutions,and within eachresolutiontheoutputis synthesizedsliceby slicealongthetimedomain.

Figure14showssynthesisresultsof severaltypicaltemporaltex-tures: fire, smoke, andoceanwaves(animationsavailableon ourwebpage).Theresultingsequencescapturetheflavorof theoriginalmotions,andtile both spatiallyandtemporally. This techniqueisalsoefficient. Acceleratedby TSVQ,eachresultframetook about

20 secondsto synthesize.Currentlyall the texturesaregeneratedautomatically;we plan to extendthe algorithmto allow moreex-plicit usercontrols(suchasthedistributionandintensityof thefireandsmoke).

6 Conclusions and Future Work

Texturesare importantfor a wide variety of applicationsin com-putergraphicsandimageprocessing.On theotherhand,they arehard to synthesize.The goal of this paperis to provide a practi-cal tool for efficiently synthesizinga broadrangeof textures. In-spiredby Markov RandomFieldmethods,ouralgorithmis general:a wide variety of texturescanbe synthesizedwithout any knowl-edgeof their physicalformationprocesses.The algorithmis alsoefficient: by aproperaccelerationusingTSVQ,typical texturescanbegeneratedwithin secondson currentPCsandworkstations.Thealgorithmis alsoeasyto use:only anexampletexturepatchis re-quired.

Thebasicversionof our algorithm(Section2) relatesto anear-lier work by PopatandPicard[20] in that a causalneighborhoodandrasterscanorderingareusedfor texture synthesis.However,insteadof constructingexplicit probability models,our algorithmusesdeterministicsearching.This approachsharesthe simplicityof EfrosandLeung[6], but usesfix-sizedneighborhoodswhichal-low TSVQacceleration.Thefactthatsuchasimpleapproachworkswell on many differenttexturesimplies that theremaybe compu-tational redundanciesin other texture synthesistechniques.Thisalgorithmsharessomeof thesamelimitationsasMarkov RandomField approaches:in particular, only local andstationaryphenom-enacanberepresented.Othervisualcuessuchas3D shape,depth,lighting, or reflectioncannotbecapturedby thissimplemodel.

Aside from constrainedsynthesisand temporal textures, nu-merousapplicationsof our approacharepossible.Otherpotentialapplications/extensionsare:

Multidimensional texture: The notion of texture extendsnatu-rally to multi-dimensionaldata.Oneexamplewaspresentedin thispaper- motionsequences.Thesametechniquecanalsobedirectlyappliedto solid texturesor animatedsolid texture synthesis.Weare also trying to extend our algorithm for generatingstructuredsolid texturesfrom 2D views [9].

Texture compression/decompression: Texturesusuallycontainrepeatingpatternsandhigh frequency information; thereforetheyare not well compressedby transform-basedtechniquessuchas JPEG. However, codebook-basedcompressiontechniqueswork well on textures[1]. This suggeststhat texturesmight becompressableby our synthesistechnique. Compressionwouldconsistof building a codebook,but unlike [1], no code indiceswould be generated;only the codebookwould be transmittedandthe compressionratio is controlledby the numberof codewords.Decompressionwould consistof texture synthesis. This decom-pressionstep,if acceleratedonemoreorderof magnitudeover ourcurrent software implementation,could be usablefor real timetexturemapping.Theadvantageof this approachover [1] is muchgreatercompression,sinceonly thecodebookis transmitted.

Motion synthesis/editing: Some motions can be efficientlymodeledas spatial-temporaltextures. Others,suchas animal orhumanmotion,aretoohighly structuredfor suchadirectapproach.However, it might be possibleto encodetheir motion as jointangles,and then apply texture analysis-synthesisto the resulting1D temporalmotionsignals.

Modeling geometric details: Models scannedfrom real worldobjectsoften contain texture-like geometricdetails, making the

(a) (b) (c)

Figure14: Temporal texture synthesisresults.(a) fire (b) smoke (c) oceanwaves.In each pair of images,thespatial-temporal volumeof theoriginal motionsequenceis shownon theleft, andthecorrespondingsynthesisresultis shownon theright. A 3-level Gaussianpyramid,with neighborhoodsizes

�5x5x5,2� ,�

3x3x3,2� , � 1x1x1,1� , are usedfor synthesis.Theoriginal motionsequencescontain32 frames,andthesynthesisresultscontain64 frames.Theindividualframesizesare (a) 128x128(b) 150x112(c) 150x112.AcceleratedbyTSVQ,thetraining timesare (a) 1875(b) 2155(c) 2131secondsandthesynthesistimesper frameare (a) 19.78(b) 18.78(c) 20.08seconds.To savememory, weuseonly a random10 percentof the input neighborhoodvectors to build the(full)codebooks.

modelsexpensiveto store,transmitor manipulate.Thesegeometricdetailscanbe representedasdisplacementmapsover a smoothersurface representation[13]. The resulting displacementmapsshouldbe compressable/decompressableas2D texturesusingourtechnique. Taking this idea further, missinggeometricdetails,acommonproblemin many scanningsituations[14], couldbefilledin usingour constrainedtexturesynthesistechnique.

Direct synthesis over meshes: Mappingtexturesonto irregular3D meshesby projectionoftencausesdistortions[21]. Thesedis-tortionscansometimesbefixedby establishingsuitableparameter-izationof themesh,but amoredirectapproachwouldbeto synthe-sizethetexturedirectlyoverthemesh.In principle,thiscanbedoneusingour technique.However, thiswill requireextendingordinarysignalprocessingoperationssuchasfiltering anddownsamplingtoirregular3D meshes.

Acknowledgments

We would like to thankKris PopatandAlyoshaEfrosfor answeringques-tionsabouttheir texture synthesisworks,Phil Hubbardfor his helpon thewriting of thepaper, andtheanonymousreviewersfor theircomments.Thetexturethumbnailshown in Figure1 wasacquiredfrom JeremyDeBonet’swebpage.Specialthanksto membersof theStanfordGraphicsGroup.Thisresearchwassupportedby Intel, Interval, andSony undertheStanfordIm-mersive Television Project.

References[1] A. C. Beers,M. Agrawala,andN. Chaddha.Renderingfrom compressedtex-

tures.Proceedingsof SIGGRAPH96, pages373–378,August1996.[2] P. Brodatz. Textures: A PhotographicAlbumfor ArtistsandDesigners. Dover,

New York, 1966.[3] P. J.Burt andE. H. Adelson.A multiresolutionsplinewith applicationto image

mosaics.ACM Transactionson Graphics, 2(4):217–236,Oct.1983.[4] J.S.DeBonet.Multiresolutionsamplingprocedurefor analysisandsynthesisof

texture images.In T. Whitted,editor, SIGGRAPH97 ConferenceProceedings,AnnualConferenceSeries,pages361–368.ACM SIGGRAPH,AddisonWesley,Aug. 1997.

[5] J.Dorsey, A. Edelman,J.Legakis,H. W. Jensen,andH. K. Pedersen.Modelingandrenderingof weatheredstone. Proceedingsof SIGGRAPH99, pages225–234,August1999.

[6] A. EfrosandT. Leung.Texturesynthesisby non-parametricsampling.In Inter-nationalConferenceon ComputerVision, volume2, pages1033–8,Sep1999.

[7] A. GershoandR.M. Gray. VectorQuantizationandSignalCompression. KluwerAcademicPublishers,1992.

[8] R. Haralick.Statisticalimagetextureanalysis.In Handbookof PatternRecogni-tion andImage Processing, volume86,pages247–279.AcademicPress,1986.

[9] D. J. Heeger andJ. R. Bergen. Pyramid-Basedtexture analysis/synthesis.InR. Cook, editor, SIGGRAPH95 ConferenceProceedings, Annual ConferenceSeries,pages229–238.ACM SIGGRAPH,AddisonWesley, Aug. 1995.

[10] A. N. Hirani andT. Totsuka.Combiningfrequency andspatialdomaininforma-tion for fast interactive imagenoiseremoval. ComputerGraphics, 30(AnnualConferenceSeries):269–276,1996.

[11] H. IgehyandL. Pereira.Imagereplacementthroughtexturesynthesis.In Inter-nationalConferenceonImage Processing, volume3, pages186–189,Oct1997.

[12] H. IversenandT. Lonnestad.An evaluationof stochasticmodelsfor analysisandsynthesisof grayscaletexture. PatternRecognitionLetters, 15:575–585,1994.

[13] V. Krishnamurthyand M. Levoy. Fitting smoothsurfacesto densepolygonmeshes.Proceedingsof SIGGRAPH96, pages313–324,August1996. ISBN0-201-94800-1.Held in New Orleans,Louisiana.

[14] M. Levoy, K. Pulli, B. Curless,S.Rusinkiewicz, D. Koller, L. Pereira,M. Ginz-ton, S. Anderson,J. Davis, J. Ginsberg, J. Shade,and D. Fulk. The DigitalMichelangeloProject:3D scanningof largestatues.To appearin ProceedingsofSIGGRAPH2000.

[15] T. MalzbenderandS.Spach.A context sensitive texturenib. In ProceedingsofComputerGraphicsInternational, pages151–163,June1993.

[16] MIT Media Lab. Vision texture. http://www-white.media.mit.edu/vismod/-imagery/VisionTexture/vistex.html.

[17] S. NeneandS. Nayar. A simplealgorithmfor nearestneighborsearchin highdimensions.IEEE Transactionson PatternAnalysisandMachine Intelligence,19:989–1003,1997.

[18] R. PagetandI. Longstaff. Texturesynthesisvia anoncausalnonparametricmul-tiscaleMarkov randomfield. IEEETransactionsonImageProcessing, 7(6):925–931,June1998.

[19] A. C. Popat. Conjoint Probabilistic SubbandModeling. PhD thesis,Mas-sachusettsInstituteof Technology, 1997.

[20] K. PopatandR. Picard. Novel cluster-basedprobabilitymodelfor texturesyn-thesis,classification,andcompression.In Visual Communicationsand ImageProcessing, pages756–68,1993.

[21] M. Segal, C. Korobkin, R. van Widenfelt, J. Foran,and P. E. Haeberli. Fastshadows andlighting effectsusingtexturemapping.ComputerGraphics(Pro-ceedingsof SIGGRAPH92), 26(2):249–252,July1992.

[22] E. Simoncelli and J. Portilla. Texture characterizationvia joint statisticsofwaveletcoefficientmagnitudes.In Fifth InternationalConferenceonImagePro-cessing, volume1, pages62–66,Oct.1998.

[23] J.StamandE. Fiume.Depictingfire andothergaseousphenomenausingdiffu-sionprocesses.Proceedingsof SIGGRAPH95, pages129–136,August1995.

[24] M. SzummerandR. W. Picard. Temporaltexture modeling. In InternationalConferenceon Image Processing, volume3, pages823–6,Sep1996.

[25] L. Wei. Deterministictextureanalysisandsynthesisusingtreestructurevectorquantization. In XII Brazilian Symposiumon ComputerGraphicsand ImageProcessing, pages207–213,October1999.

[26] A. Witkin andM. Kass.Reaction-diffusiontextures.In T. W. Sederberg, editor,ComputerGraphics(SIGGRAPH’91 Proceedings), volume25,pages299–308,July1991.

[27] S. P. Worley. A cellular texture basisfunction. In H. Rushmeier, editor, SIG-GRAPH96ConferenceProceedings, AnnualConferenceSeries,pages291–294.ACM SIGGRAPH,AddisonWesley, Aug. 1996.

[28] S. Zhu, Y. Wu, andD. Mumford. Filters, randomfieldsandmaximunentropy(FRAME) - towardsaunifiedtheoryfor texturemodeling.InternationalJournalof ComputerVision, 27(2):107–126,1998.