detail preserving shape deformation in image editing

28
Detail Preserving Shape Deformation in Image Editing SIGGRAPH 2007 Hui Fang and John C. Hart

Upload: kalei

Post on 24-Feb-2016

51 views

Category:

Documents


0 download

DESCRIPTION

Detail Preserving Shape Deformation in Image Editing. SIGGRAPH 2007 Hui Fang and John C. Hart. Abstract. We propose an image editing system Preserve its detail and orientation by resynthesizing texture from the source - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Detail Preserving Shape Deformation in Image Editing

Detail Preserving Shape Deformation in Image Edit-

ingSIGGRAPH 2007

Hui Fang and John C. Hart

Page 2: Detail Preserving Shape Deformation in Image Editing

We propose an image editing system

◦ Preserve its detail and orientation by resynthesiz-ing texture from the source

◦ Patch-based texture synthesis that aligns texture features with image features

Abstract

Page 3: Detail Preserving Shape Deformation in Image Editing

A novel image editing system that allows a user to select and move one or more image feature curves◦ Replacing any texture stretched by the deforma-

tion with texture resynthesized Anisotropic feature-aligned texture synthesis step to

preserve texture detail Distortion to the texture coordinates for each patch to

align the target image features GraphCut textures [Kwatra et al. 2003]

Introduction

Page 4: Detail Preserving Shape Deformation in Image Editing

A new method that distorts the coordinates of patch◦ Image Analogies [Hertzmann et al. 2001] can synthe-

size a texture to adhere to a given feature line Yields more high-frequency noise unlike modern patch-

based synthesis◦ Image Quilting [Efros and Freeman 2001] could fill dif -

ferent silhouettes with a texture Boundary patches appeared to repeat

◦ Feature matching and deformation for texture synthe-sis [Wu and Yu 2004] distorted neighboring patches to connect their feature lines Not as global as what us did

Introduction

Page 5: Detail Preserving Shape Deformation in Image Editing

Deformation◦ Draw feature curves in the source image, and then

move them to their desired destination positions Curvilinear Coordinates

◦ Define curvilinear coordinates using curve tangent vectors & Euler integration

Textured Patch Generation◦ A pair of curvilinear coordinate is generated◦ Texture synthesis over the destination grid from source

Image Synthesis◦ Finalize the synthesis via GraphCut

Overview

Page 6: Detail Preserving Shape Deformation in Image Editing

Deformation

pi(t)

p'i(t)

D(p'f) = pf – p'f

D(∂I’) = 0

Page 7: Detail Preserving Shape Deformation in Image Editing

Deformation

Original Deformed

Page 8: Detail Preserving Shape Deformation in Image Editing

Curvilinear Coordinates

p'i(t)

T'

Page 9: Detail Preserving Shape Deformation in Image Editing

Since the parametrization of each feature curve is arbitrary, one can encounter global orientation inconsistencies◦ Calculate separate tangent field for each curve

then use only the field which is the closest We integrate these diffused tangents to

construct a local curvilinear coordinate sys-tem extending from any chosen “origin” pixel

Curvilinear Coordinates

Page 10: Detail Preserving Shape Deformation in Image Editing

Curvilinear Coordinates

p'i(t)

jk

Page 11: Detail Preserving Shape Deformation in Image Editing

Time-step ɛ = 1◦ 30 ~ 40 pixels along spines (j direction)◦ 15 ~ 30 pixels wide ribs (k direction)◦ Two pixels short of nearby feature curve to prevent overlap-

ping

Smooth the coordinates with several Laplacian itera-tions

◦ λ = 0.7◦ Removes singularities and self-intersections that can occur ◦ Does not completely solve the problem (Not very noticeable)

Curvilinear Coordinates

Page 12: Detail Preserving Shape Deformation in Image Editing

Curvilinear Coordinates

Page 13: Detail Preserving Shape Deformation in Image Editing

Source origin q0,0 = D(q'0,0)

Bilinear filter to find the color at the source image

Unit-radius cone filter centered at each desti-nation to accumulate the synthesized texture◦ Small reduction in the resolution of the resynthe-

sized texture detail

Textured Patch Generation

Page 14: Detail Preserving Shape Deformation in Image Editing

Use GraphCut [Kwatra et al. 2003]◦ Generate patches individually, using a priority

queue to generate first patches whose origin pixel is closest to the feature curve and adjacent to a previously synthesized patch

◦ Generate a pool of candidate textured patches synthesized from source patches grown from origins randomly chosen from an 11×11 pixel region surrounding the point D(q'0,0)

◦ Choose one with the least overlapping difference with previously synthesized neighboring patches

Image Synthesis

Page 15: Detail Preserving Shape Deformation in Image Editing

Selected patch merges into destination via GraphCut

Use Poission Image Editing when the seam produces by GraphCut is unsatisfactory

Image Synthesis

Page 16: Detail Preserving Shape Deformation in Image Editing

The deformation field D can potentially compress a large source area into a small target area◦ Cause blocky artifacts and seams◦ Occur when the origin pixels of neighboring

patches in the target map to positions in the source with different texture characteristics

Can be overcome by altering the texture synthesis sampling

Scale Adaptive Retexturing

Page 17: Detail Preserving Shape Deformation in Image Editing

Scale Adaptive Retexturing

Page 18: Detail Preserving Shape Deformation in Image Editing

We detect these potential problems with a (real) compression field C'

◦ Clamp the compression field to values in [1,3] to limit its effect

◦ The “spine” length and “rib” breadth of patches are reduced by C'(x,y)

Scale Adaptive Retexturing

Page 19: Detail Preserving Shape Deformation in Image Editing

Scale Adaptive Retexturing

Page 20: Detail Preserving Shape Deformation in Image Editing

Accelerated the construction of source feature curves by using portions of the segmentation boundary produced by Lazy Snapping [Li et al. 2004]◦ Feature curves do not need to match feature con-

tours exactly, as deformed features were often aligned by the texture search

Used the ordinary Laplacian deformation for in-teractive preview◦ Denoted some feature curves as “passive” to aid tex-

ture orientation

Results

Page 21: Detail Preserving Shape Deformation in Image Editing

Filtering used for curvilinear grid resampling removes some of the high frequency detail◦ Could be recovered by sharpening with his-

togram interpolation and matching [Matusik et al. 2005]

Results

Page 22: Detail Preserving Shape Deformation in Image Editing

Results

Page 23: Detail Preserving Shape Deformation in Image Editing

Results

Page 24: Detail Preserving Shape Deformation in Image Editing

Results

Page 25: Detail Preserving Shape Deformation in Image Editing

Failure case

Page 26: Detail Preserving Shape Deformation in Image Editing

Sharp image changes (like shading changes) should identified by feature curves◦ Lack of feature curves will cause unrealistic dis-

continuities in the result

Poisson image editing hides some of these artifacts◦ by softly blending the misaligned features

Results

Page 27: Detail Preserving Shape Deformation in Image Editing

ResultsMeasured on a 3.40GHz

Pentium 4 CPU(31 x 31 search domain for beach)

Page 28: Detail Preserving Shape Deformation in Image Editing

Stretched texture details can be adequately recovered by a local retexturing around user-defined feature curves

Assumes that the orientation of texture detail of an image is related to the orientation of nearby feature curves

Matting can be used to eliminate unwanted artifacts (Fig. 5)

In practice the success of this approach depends pri-marily on the selection of the feature curves◦ The most promising direction of future work in this topic

would be to add the automatic detection and organization of image feature curves

Conclusion