Texture SymmetryTexture Symmetry

A lecture by Alexey BurshteinA lecture by Alexey Burshtein

DefinitionsDefinitions

Regular textureRegular texture is a periodic is a periodic pattern containing translation pattern containing translation symmetry and (possibly) rotation, symmetry and (possibly) rotation, reflection and glide-reflection reflection and glide-reflection symmetries.symmetries.

Near-regular textureNear-regular texture is a texture is a texture that is not strictly symmetrical. The that is not strictly symmetrical. The difference may be in color, difference may be in color, deformations, resolution.deformations, resolution.

Definitions – cont.Definitions – cont.

Perfect regularities are uncommon;Perfect regularities are uncommon; Near-regular textures are very common.Near-regular textures are very common.

Definitions – cont.Definitions – cont.

TileTile (also (also lattice itemlattice item) – a smallest ) – a smallest parallelogram (or hexagon), whose orbit parallelogram (or hexagon), whose orbit produces a cover of the original pattern produces a cover of the original pattern with no gaps or overlaps.with no gaps or overlaps.

In a way, In a way, every pictureevery picture as a as a distorted regular distorted regular patternpattern..

Thus, Thus, every pictureevery picture may be generated by a may be generated by a near-regular pattern generator.near-regular pattern generator.

We’ll see how to create this We’ll see how to create this

from this:from this:

Theory of wallpaper-like regular structures Theory of wallpaper-like regular structures exists for more then 100 years; it’s called exists for more then 100 years; it’s called Theory of Wallpaper Groups.Theory of Wallpaper Groups.

There are only 5 possible shapes of tiles:There are only 5 possible shapes of tiles: parallelogram;parallelogram; rectangle;rectangle; rhomb;rhomb; square;square; hexagonal.hexagonal.

You cannot just fill all the image with one You cannot just fill all the image with one tile. The result will be not naturalistic.tile. The result will be not naturalistic.

There are two contradictive objectives:There are two contradictive objectives: Preparing a realistic image Preparing a realistic image (= giving up the (= giving up the

regularityregularity)) Preserving the regularity Preserving the regularity (= filling all image with (= filling all image with

same tile and giving up the naturalismsame tile and giving up the naturalism)) The following algorithm avoid these traps.The following algorithm avoid these traps.

Near-regularNear-regularTexture SynthesisTexture Synthesis

Part 1.

Input: a sample near-regular texture SInput: a sample near-regular texture S Output: a synthesized texture SOutput: a synthesized texture S00

statistically similar to S.statistically similar to S. The color and intensity are captured from The color and intensity are captured from

different tiles to give the output more natural different tiles to give the output more natural appearance.appearance.

This will not affect the regularity of the This will not affect the regularity of the synthesized pattern.synthesized pattern.

Algorithm explainedAlgorithm explained

Algorithm explained – cont.Algorithm explained – cont. First of all, recognize the tile precisely.First of all, recognize the tile precisely.

This is performed with algorithm “regions of dominance” This is performed with algorithm “regions of dominance” which is beyond our scope.which is beyond our scope.

The found lattice is determined by two vectors tThe found lattice is determined by two vectors t11 and t and t22.. Offset of the lattice from each side is NOT determined.Offset of the lattice from each side is NOT determined.

t’2

t’1t2

t1

Algorithm explained – cont.Algorithm explained – cont.

The vectors tThe vectors t11 and t and t22 may be also specified may be also specified

(or verified) by user.(or verified) by user. The lattice has to be placed so that all The lattice has to be placed so that all

original tiles are uniquely defined. This is original tiles are uniquely defined. This is usually controlled by the user.usually controlled by the user.

Minimum tilesMinimum tiles – the tiles carved by the – the tiles carved by the lattice.lattice.

Maximum tileMaximum tile – smallest rectangular shape – smallest rectangular shape which circumscribes the minimum tile.which circumscribes the minimum tile.

Yellow rhombs – Yellow rhombs – minimum tiles minimum tiles recognized by the recognized by the algorithm.algorithm.

White rectangles – White rectangles – maximum tiles maximum tiles constructed over the constructed over the minimum tiles.minimum tiles.

Algorithm explained – cont.Algorithm explained – cont.

After all minimum tiles had been defined, After all minimum tiles had been defined, (and possibly verified by user), construct (and possibly verified by user), construct appropriate set of maximum tiles.appropriate set of maximum tiles.

Start generation:Start generation: Start from top-left corner.Start from top-left corner. Take a random Take a random maximummaximum tile. tile. Stamp it to create a row of tiles in direction of Stamp it to create a row of tiles in direction of

tt11 + t + t22 with step of |(t with step of |(t11+t+t22)/2|.)/2|. After reaching the right bound, place one tile After reaching the right bound, place one tile

at tat t22 – t – t11 with step of |(t with step of |(t11+t+t22)/2| to the left of the )/2| to the left of the

generated row of tiles.generated row of tiles.

Algorithm explained – cont.Algorithm explained – cont.

Algorithm explained – cont.Algorithm explained – cont. The tile to be used as a stamp is a random The tile to be used as a stamp is a random

selection from tiles that fit most closely to selection from tiles that fit most closely to generated color of the place.generated color of the place.

Therefore, at each cell in the generated lattice Therefore, at each cell in the generated lattice the closest tile will be placed.the closest tile will be placed.

The chosen tile is allowed to move a bit The chosen tile is allowed to move a bit around the calculated place to fit the structure around the calculated place to fit the structure even better.even better.

Neighboring tiles are “stitched” together. To Neighboring tiles are “stitched” together. To avoid conflicts, boundaries of tiles are avoid conflicts, boundaries of tiles are blended. blended.

Continue until the image is created.Continue until the image is created.

Algorithm - conclusionsAlgorithm - conclusions Maximum tiles are used instead of Maximum tiles are used instead of

minimum tiles to allow smoother blending minimum tiles to allow smoother blending of tiles’ edges. of tiles’ edges. Each tile has from ½ to ¾ of Each tile has from ½ to ¾ of overlapped areaoverlapped area..

The generated structure is smoother then The generated structure is smoother then the original – the original – because of blendingbecause of blending..

The result may be more regular then the The result may be more regular then the source – source – because of too tight error threshold in because of too tight error threshold in selection of tiles to be used as stamps.selection of tiles to be used as stamps.

It’s either bigger choice of tiles to be used It’s either bigger choice of tiles to be used in every point of lattice, or smoother in every point of lattice, or smoother transition between neighbor tiles.transition between neighbor tiles.

Algorithm - conclusionsAlgorithm - conclusions Damaged tiles may never be used at all.Damaged tiles may never be used at all. Examples:Examples:

Algorithm - conclusionsAlgorithm - conclusions Question: will regularity be preserved if Question: will regularity be preserved if

bigger size of tile will be taken?bigger size of tile will be taken?Answer:Answer: NONO, not really., not really. The regularity will exist, but it will be some other The regularity will exist, but it will be some other

regularity, not the one from the original tile set.regularity, not the one from the original tile set.

Algorithm – advantagesAlgorithm – advantages

1.1. The tile is not necessarily a rectangle.The tile is not necessarily a rectangle.

2.2. The orientation of tiles is not necessarily The orientation of tiles is not necessarily horizontal.horizontal.

3.3. Perfect adaptation to the given sample.Perfect adaptation to the given sample.

4.4. We’ve covered here only translational We’ve covered here only translational symmetry; rotation, reflection and glide symmetry; rotation, reflection and glide reflection can also be used.reflection can also be used.

What if the latticeWhat if the latticeitself is irregular?itself is irregular?

Part 2.

The ProblemThe Problem

The lattice may not be defined with The lattice may not be defined with straight lines.straight lines.

Examples:Examples:

Idea: beneath each successful man Idea: beneath each successful man stands a more successful woman.stands a more successful woman.

The ApproachThe Approach

Idea: beneath each irregular lattice stands Idea: beneath each irregular lattice stands a distorted regular one.a distorted regular one.

Let’s find out the near-regular lattice Let’s find out the near-regular lattice which resembles the given irregular lattice which resembles the given irregular lattice most.most.The lattice is computed using energy The lattice is computed using energy minimization functionminimization function

The difference between the lattices will The difference between the lattices will allow us to compute the allow us to compute the distortion fielddistortion field..

How do we compute the (near) How do we compute the (near) regular lattice from the irregular one?regular lattice from the irregular one?

The function is:The function is:

Where tWhere t11 and t and t22 are the vectors defining the near- are the vectors defining the near-

regular lattice, lregular lattice, l ii, l, ljj, l, lkk, l, lmm are links in directions t are links in directions t11, ,

tt22, t, t11+t+t22, t, t11-t-t22 respectively; N respectively; Nii, N, Njj, N, Nkk, N, Nmm are total are total

number of such links. number of such links. is angle between t is angle between t1 1 and tand t22. .

It may be computed from tIt may be computed from t11, t, t22, t, t11+t+t22..

mkji n

m k

n

k k

n

j j

n

i ittttlttltltlE

1

221

1

221

1

22

1

21

,,)()()()(min

21

Algorithm explainedAlgorithm explained1.1. Compute the near-regular lattice.Compute the near-regular lattice.

2.2. Compute the distortion field.Compute the distortion field.

3.3. Rectify the input texture into near-regular Rectify the input texture into near-regular structure.structure.

4.4. Create 2D-image from near-regular Create 2D-image from near-regular texture as explained in part 1.texture as explained in part 1.

5.5. Create 2D-pattern of size equal to the Create 2D-pattern of size equal to the requested image from the deformation requested image from the deformation field.field.More tight requirements for smoothness on More tight requirements for smoothness on edges!edges!

Algorithm Algorithm explainedexplained

6.6. Apply the resulting Apply the resulting distortion field map distortion field map built at step 5 to the built at step 5 to the image built at step 4.image built at step 4.

Practical Usage –Practical Usage –Generalized Symmetry Generalized Symmetry

TransformTransformPart 3.

Generalized Symmetry TransformGeneralized Symmetry Transform

Input: edge map.Input: edge map. Output: map of points of high symmetry, Output: map of points of high symmetry,

with intensity and orientation.with intensity and orientation. Usage: detection of symmetry; possibility Usage: detection of symmetry; possibility

to detect symmetric patterns.to detect symmetric patterns. The symmetry transform assigns The symmetry transform assigns

continuous symmetry measure to each continuous symmetry measure to each point.point.

Symmetry transform is local, not global!Symmetry transform is local, not global!

How the symmetry is quantified?How the symmetry is quantified?

Each point receives its symmetry value Each point receives its symmetry value calculated as a sum of contributions from calculated as a sum of contributions from all other points.all other points.

We’ve already seen thisWe’ve already seen thisbefore.before.

Symmetry direction isSymmetry direction isdefined as direction ofdefined as direction ofmaximal product.maximal product.

ri

rj

j

i

(i+j)/2

How the symmetry is quantified?How the symmetry is quantified?

It is possible to detect radial symmetry as It is possible to detect radial symmetry as well as other types of symmetry.well as other types of symmetry.

Line is treated as a single object (reminds Line is treated as a single object (reminds Hough transform).Hough transform).

The contributed weight of each symmetry The contributed weight of each symmetry is defined using the Gaussian. This is defined using the Gaussian. This Gaussian may be elliptic – for recognizing Gaussian may be elliptic – for recognizing human eyes, for example.human eyes, for example.

Result of Symmetry TransformResult of Symmetry Transform

From left to right – original image, the From left to right – original image, the edges, symmetric points.edges, symmetric points.

Symmetric transform generalizes most Symmetric transform generalizes most methods for finding interesting regions.methods for finding interesting regions.

Usage of the transformUsage of the transform

Using this transform, it is possible to Using this transform, it is possible to recognize symmetry in patterns. recognize symmetry in patterns. Examples:Examples:

Note: X`es are detected Note: X`es are detected as +`es; circles are as +`es; circles are detected as points.detected as points.

ConclusionsConclusions

This transform can be used to detect This transform can be used to detect symmetric figures on asymmetric symmetric figures on asymmetric background or (even better) vice versa.background or (even better) vice versa.It’s easier to detect ellipses among circles.It’s easier to detect ellipses among circles.

The model may also be used as predictor The model may also be used as predictor of human behavior when recognizing of human behavior when recognizing symmetric and asymmetric patterns.symmetric and asymmetric patterns.

It should be a part of a bigger processing It should be a part of a bigger processing system rather then stand-alone system rather then stand-alone application.application.

ReferencesReferences

1.1. Y. Liu and W. Lin “Deformable Texture: Y. Liu and W. Lin “Deformable Texture: the Irregular-Regular-Irregular Cycle”the Irregular-Regular-Irregular Cycle”http://www.ri.cmu.edu/pubs/pub_4722.htmlhttp://www.ri.cmu.edu/pubs/pub_4722.html

2.2. Y. Liu, Y. Tsin, and W. Lin ,”The Promise Y. Liu, Y. Tsin, and W. Lin ,”The Promise and Perils of Near-Regular Texture”and Perils of Near-Regular Texture”http://www.ri.cmu.edu/pubs/pub_4406.htmlhttp://www.ri.cmu.edu/pubs/pub_4406.html

3.3. J. Beck. “Textural segmentation.” J. Beck. “Textural segmentation.” In J. Beck, editor, Organization and Representation in Perception, In J. Beck, editor, Organization and Representation in Perception, pages 285-318. Hilldale,NJ: Lawrence Erlbaum, 1982.pages 285-318. Hilldale,NJ: Lawrence Erlbaum, 1982.

4.4. www.google.comwww.google.com