advanced design for a realistic virtual brushi.cs.hku.hk/~fcmlau/papers/eg03.pdf · advanced design...

EUROGRAPHICS 2003 / P. Brunet and D. Fellner(Guest Editors)

Volume 22(2003), Number 3

Advanced Design for a Realistic Virtual Brush

Songhua Xu†,‡, Francis C.M. Lau‡, Feng Tang† and Yunhe Pan†

† CAD & CG State Key Lab of China, Zhejiang University, P.R. China‡ Department of Computer Science and Information Systems, The University of Hong Kong, Hong Kong

AbstractThis paper proposes a novel algorithmic framework for an advanced virtual brush to be used in interactive digitalpainting. The framework comprises the following components: a geometry model of the brush using a hierarchicalrepresentation that leads to substantial savings in every step of the painting process; fast online brush motionsimulation assisted by offline calibration that guarantees an accurate and stable simulation of the brush’s dy-namic behavior; a new pigment model based on a diffusion process of random molecules that considers delicateand complex pigment behavior at dipping time as well as during painting; and a user-adaptation component thatenables the system to cater for the personal painting habits of different users. A prototype system has been im-plemented based on this framework. Compared with other virtual brushes, this new system is designed to presenta realistic brush in the sense that the system accurately and stably simulates the complex painting functionalityof a running brush, and therefore is capable of creating high-quality digital paintings with minute aesthetic de-tails that can rival the real artwork. The advanced features also give rise to a high degree of expressiveness ofthe virtual brush that the user can comfortably manipulate. http://www.csis.hku.hk/∼songhua/e-brush/ providessupplementary materials for this paper.

Categories and Subject Descriptors(according to ACM CCS): I.3.6 [Methodology and Techniques]: Interactiontechniques; I.3.5 [Computational Geometry and Object Modeling]: Physically based modeling; I.3.4 [GraphicsUtilities]: Paint systems;

1. Introduction

Virtual brush is an important tool for interactive painting23, 17, 20, 21, 13, 12, 16, 22, 8, 4, 19, 24, 15. Smith has written a good sur-vey on the early painting systems2. These early systemsoffered a 2D brush for painting. The newer painting sys-tems are more sophisticated, which can for example auto-matically generate painterly rendering results based on someinput reference images3. To provide a virtual brush that canmimic the real 3D brush for painting and calligraphy writ-ing has become a popular area for research in recent years8, 4, 24, 15. The researchers are attracted by the unique abilityof a 3D virtual brush to generate expressive and realistic re-sults through techniques such as physically-based modeling.Common to most existing 3D virtual brushes are three corecomponents that implement the most essential functionali-ties of a 3D virtual brush: a brush geometry model, a brushmotion simulator, and a pigment behavior model. The fol-lowing subsections give an overview of existing work withregard to these three core components.

1.1. Previous Work

1.1.1. Virtual brush geometry models

One of the earliest models was by Strassmann23, wherebrush strokes are created by sweeping a 1D brush bristle overa skeleton. It is effectively a 2D heuristic approach which isnot natural to use for non-computer specialists. Wong andIp’s virtual brush is modeled as an inverse cone and can pro-duce an elliptic drawing mark8. This represents a substantialimprovement over Strassmann’s in terms of usability. How-ever, because of the need to manually specify an intricateset of parameters controlling the shape, density and opac-ity of the current brush drawing mark, the interactivity ofWong and Ip’s system is limited. In the DAB project4, asubdivision surface is wrapped around a spring-mass particlesystem skeleton to represent the brush geometry. One disad-vantage of using a subdivision surface is that the splittingof the brush is very hard to model. To generate the subdivi-sion surface to model the brush head, either interpolation orsome approximating scheme is used. An interpolated brush

c© The Eurographics Association and Blackwell Publishers 2003. Published by BlackwellPublishers, 108 Cowley Road, Oxford OX4 1JF, UK and 350 Main Street, Malden, MA02148, USA.

S. Xu, F.C.M. Lau, F. Tang and Y. Pan / Advanced Design for a Realistic Virtual Brush

head, however, often cannot deform smoothly because of thefrequent occurrence of high curvature in the brush head sur-face, while approximation has the problem of properly plac-ing control points to yield the desired surface. On top of allthis, how to suitably anchor the control points of the surfaceto the mass particles is a non-trivial problem. In the work byXu et al. 24, general sweeping is employed to establish thesolid geometry model of each hair cluster. The problem how-ever is that general sweeping is a time-consuming operation.At different stages of painting, much computation is neededto apply general sweeping operations to update the model.Also, the solid model requires a fair amount of memory forits internal representation, and after the brush is split manytimes, the demand for memory could become a bottleneck.In the work by Chu and Tai15, a single hair bundle is mod-eled by a geometry model that is mathematically equivalentto that by Xu et al.24’s. Unlike the latter which simulates thespreading and splitting of the brush tip by a geometry ap-proach, Chu and Tai use an alpha map to implement clustermodeling for the split brush. This results in an over simpli-fication which limits the expressiveness and the amount offine details that the brush is able to produce.

It appears that none of the above is sufficient to model aphysical brush with a high degree of likeness to the brush’sreal behavior. One frequently occurring situation in reality,for example, is that of the brush splitting into a large num-ber of hair clusters. To model this rather “chaotic” situationis out of reach for all the existing brush geometry models.According to many practising artists, such a high degree ofsplitting is very much desirable. In this paper, we propose ahierarchical geometry model for a “realistic” brush; togetherwith a fast rendering procedure, our model enables efficientsimulation of the most complex effects of a real brush, suchas brush splitting.

1.1.2. Virtual brush motion simulators

Simulating the motion and dynamic deformation of a realpaintbrush is a tough problem because of the complex staticgeometry and dynamics of the modeled brush.

In DAB 4, the motion of the brush geometry is simu-lated by Newtonian dynamics using a pair of first-orderdifferential equations and Aristotelian dynamics through asingle first-order differential equation. Solving differentialequations involves some tradeoff between system stability,computation efficiency and simulation accuracy. In fact, thebrush is a heavily damped system. When the penholder ismotionless, the brush geometry stops deforming. Unfortu-nately, such a large damping force in Newtonian dynamicsis hard to estimate. The authors of DAB therefore apply con-straint satisfaction at the end of each step of motion simula-tion to ensure that the resultant brush geometry is a plausibleconfiguration. This gives rise to the issue of simulation pre-ciseness versus system stability. Unlike other classical mo-tion simulation approaches which care a lot about simulation

accuracy, the approach by Xu et al.24 aims at providing auser with a customized virtual hairy brush. Their brush mo-tion and brush geometry deformation are controlled by a setof “quality parameters” which, through a machine trainingmodule, can be customized for different users. These qualityparameters affect the deformation process of a virtual hairybrush, and are not meant to be directly set or adjusted by theuser. Unless the user can provide enough samples to trainthe system, he/she could only accept the default values forthese quality parameters. The performance of the machinetraining process depends on the number of samples the userwould input into the system. For satisfactory customizationresult, a large number of samples is often needed. The dy-namics of Chu and Tai’s brush15 are modeled as springs,which are deformed through constrained energy minimiza-tion. This method of modeling can simulate small-scale de-formation of the brush geometry but not large-scale bend-ing or stretching due to the restriction of constrained energyminimization.

In this paper, the simulation of the motion and dy-namic deformation of our hierarchical virtual brush geome-try model is performed through a collaboration between on-line and offline computations. We minimize on the amountof online computation, and the result is calibrated by an errorcalibration mechanism using a simulation error database thatwas constructed offline. Compared with other virtual brushsystems, we can simulate the brush motion with high preci-sion using just a small amount of computation on the fly.

1.1.3. Pigment behavior models

Pigment behavior simulation presents yet another challengeto virtual brush system designers. One approach to simu-lating the ink’s behavior is through a cellular automaton18.A classic work on pigment model is that by Curtis et al.6,which studies the complex interaction behind pigment mix-ing. Their approach can synthesize very realistic water-coloreffects. During digital painting, the main loop of their algo-rithm iterates through several sub-processes simulating pig-ment behavior in a sequential manner. One sub-process willpromote or restrain another sub-process. In a real paintingprocess, however, these four sub-processes have subtle in-teraction, and therefore simulating the sub-processes sepa-rately would not necessarily add up to accurately simulatingthe pigment behavior. Another problem with their approachis that solving all the complex differential equations is tootime-consuming for an interactive painting system. Other ex-isting pigment behavior models include Lee’s11 for black-white painting and Kunii et al.’s26 for diffused ink painting.

In Strassmann’s hairy brushes23, the color of brushstrokes can be varied along the stroke’s skeleton direction.In Wong’s virtual brush8, only monochrome calligraphicwriting is supported. In DAB4, a bidirectional, two-layerpaint model is employed. They use essentially an additivecompositive formula to mix the paint. In their paint repre-sentation, they assume that the paint surface contains two

c© The Eurographics Association and Blackwell Publishers 2003.


layers. One layer is completely wet where paint would trans-fer out and another is completely dry. This over simplifica-tion of paint behavior makes the system incapable of achiev-ing exquisite artistic effects demanded by the most seasonedpainters. In Xu’s virtual brush24, a simple alpha blendingstrategy is employed to perform the pigment mixing.

Pigment mixing is a very important factor that contributesto the final results of painterly rendering and therefore de-serves a very careful treatment. An ideal pigment behaviormodel should lead to fast response time in system implemen-tation such as those described in23, 8, 4, 24, and highly realisticresults as demonstrated in6. In this paper, we liken the com-plex pigment’s behavior in a painting process to that of heator moisture, and model the pigment’s behavior as a diffusionprocess. We introduce a strategy to divide the overall com-putation into an offline and an online part, therefore solvingthe complex differential equations modeling the pigment’sbehavior accurately and stably in real time.

There are many other fancy features that previous virtualbrush systems are capable of, such as the ability to do 3Dpainting17, 12, 16. The DAB system4 has haptic feedback. Thesystem by Wei et al.28 uses an intelligent model to beautifythe output. Chan et al.5 create 3D Chinese painting anima-tion using existing commercial software packages.

1.2. Our Virtual Brush

The distinct features of our design for a realistic virtual brushinclude: (1) hierarchical modeling of the complex geometryof real paintbrushes, which leads to substantial savings incomputation and representation in many steps of the simu-lation; (2) division of overall brush motion simulation intosmall online tasks and offline calibration, which makes pos-sible an accurate and stable simulation of the brush’s mo-tion in real-time; (3) modeling of pigment behavior at dip-ping time and during painting as a diffusion process of ran-dom molecules subject to a certain physical condition, whichgives rise to a new expressive pigment model that can simu-late the most delicate and complex pigment behavior; and astrategy of solving the diffusion equations through a look-uptable, which makes the numerical computation fast and sta-ble; (4) a user manipulability adaptation component, whichlets our system cater to the personal painting habits of dif-ferent users.

Section2 presents the hierarchical geometry model. Sec-tion 3 discusses the brush motion simulator and the divisioninto online and offline tasks. Section4 explains the pigmentbehavior model. Section5 briefly introduces the intelligentself-adaptive component for improving user manipulability.

2. Geometry Model

To model a paintbrush with the granularity of a single hairthread which is done in Wong and Ip’s system8 is ineffi-cient because a typical real brush may consist of thousands

Figure 1: Some complex brush geometries of our virtualbrush.

or even tens of thousands of individual hair threads. To over-come this inefficiency, Xu et al.24 model a brush as clustersof brush hair. Their approach however can only handle thecase of a brush splitting into a small number of hair clus-ters but not one with heavy splitting, for reasons explainedin Section1.1.1.

In this paper, we propose a two-level hierarchical brushgeometry model. At the lower level of the hierarchy, hairthreads whose position and geometry in 3D space are closeto each other are gathered together and modeled as onehairmacro. At the upper level, disjoint hair macros whose ge-ometries are similar are classified into the samecluster ofhair macros. The motivation behind having two levels is toeliminate as much as possible the redundancy in represent-ing and simulating the brush hair. In real actions, a brush caneasily split into thousands of disjoint clusters of hair threads.Cluster-based modeling alone as in Xu et al.’s approach24 isnot sufficient to deal with the highly chaotic geometry ofthe brush. It can be easily observed that even in such a sit-uation, there exist only a few sharply distinctive geometriesamong all the geometries of hair clusters. We call these dis-tinct geometriesprimitive geometries. With these primitivegeometries, the geometries of all the hair clusters can be ap-proximately derived via simple affine transformations. Fig-ure 1 shows some complex brush geometries of our virtualbrush. Based on this two-level hierarchical representation,our virtual brush model can efficiently represent the com-plex geometry as well as simulate the dynamic behavior ofreal paintbrushes having thousands of disjoint hair clusters.

2.1. Geometry Model of a Single Hair Macro

2.1.1. Hair macro

A hair macro represents the smallest granularity in the mod-eling. Hence, the overall modeling capability of the systemhinges upon the modeling power of a hair macro.

Strassmann swept a 1D brush bristle over a skeleton tocreate the geometry model for brush strokes23. We go onestep further by sweeping an ellipse along a skeleton in 3Dspace to create the model of a hair macro. During sweeping,we make sure that the ellipse always lies on the normal plane



with respect to its skeleton. The shape of the moving ellipsecan be varied during sweeping.

Denote the skeleton of one hair macro asC(t) (0≤ t ≤ 1),which is a B-spline in 3D space, and the ellipseE(t0) lyingon the normal plane ofC(t)|t=t0 as

E(t0), {(x,y)|x = A(t0)νcosθ, y = B(t0)νsinθ(0≤ θ < 2π,0≤ ν≤ 1)}, (1)

in which A(t0) andB(t0) are half of the lengths of the majorand minor axes of the ellipseE(t0) respectively. HereA(t)andB(t) (0≤ t ≤ 1) are two one-dimensional B-splines. Bycontrolling the three B-splines,A(t),B(t),C(t), the user cantailor the geometry of a hair macro. The definition for thegeometry of hair macro can therefore be written as

H , (A(t),B(t),C(t)) (06 t 6 1). (2)

2.1.2. Cluster of hair macros

We use clustering to achieve efficiency in modeling thehighly complex brush geometry, whereby hair macros whosegeometries are similar are grouped into the same cluster. Theinternal representation of a cluster consists of two parts: theprimitive geometry of the cluster, and some affine transfor-mations each of which to be used to derive the geometry ofa constituent hair macro based on the primitive geometry.This design makes the brush geometry model quite compactin memory, and facilitates fast simulation of the brush’s ac-tions at runtime.

2.2. Fast Rendering of Geometry

Visual feedback is important for any interactive system. Butproviding a good perceptual feedback may cost too muchcomputation, especially for a realistic virtual brush whosestatic and dynamic geometry could be highly complex. Ourhierarchical modeling helps tackle the situation.

All the hair macros of the same cluster not only share thesame data structure but also a unique multiresolution tes-sellation process. As discussed in Section2.1.2, each hairmacro belonging to the same cluster can derive its spe-cific geometry by applying the corresponding affine trans-formation to the cluster’s primitive geometry. We need there-fore only to tessellate the primitive geometry once for allthe hair macros in the cluster. Our tessellation process ismultiresolutional—that is, the density of the collection oftriangles generated is made proportional to the displayedscreen size. The granularity of the tessellation is set to bethe smallest one needed to render the surface of a hair macroin the cluster. We also take advantage of the hardware accel-erated OpenGL command “Display Lists” when computingthe affine transformation and for the rendering. The result ofall this is that our virtual brush can give very realistic visualfeedback on the complex geometry of the virtual brush inaction with a reasonably small amount of computation.

To sum up, we can derive all the specific geometries of

all the hair macros of one cluster based on one primitivegeometry. By this method, with less than say 10 primitivegeometries, we can model efficiently the complex geometryof a realistic split brush even when the number of separatehair clusters goes beyond 10000, as confirmed by our exper-iments. Compared with the solid model representation andsingle-level approach by Xu et al.24, our hierarchic model-ing leads to much reduced memory consumption and avoidsredundant computations.

3. Motion Simulator

(a) (b) (c) (d) (e)

Figure 2: Dynamic deformation of a primitive geometry dur-ing painting.

(a) (b) (c) (d) (e)

Figure 3: Dynamic deformation of the virtual brush. (a) Theinitial geometry, (b) pressed down, (c) rotated, (d) tilted, (e)further pressed down.

Our system performs a highly detailed simulation of themotion of the virtual brush. Most other physically-based sys-tems sacrifice simulation precision for system response time;as a result, some of the important physical factors that arepart of the real physical process are neglected from the verybeginning. Our design tackles the problem using a two-phaseapproach to simulate the dynamic deformation of the brush.The two phases are online computation and calibration us-ing offline data. The approach achieves high realism andinteractivity using relatively little computation. In the on-line simulation, only those computationally inexpensive andinput-sensitive physical processes are simulated. In the sec-ond phase, the online simulated results are calibrated usingoffline data.

In the online simulation, based on the current geometry ofthe virtual brush, the brush’s inner stress is estimated, whichis used to determine the state of the brush for deformationand recovery purposes.

For calibrating the online simulation results, we con-structed in advance a database of system simulation errorsagainst a collection of samples of real brush motion. Theseoffline-acquired data in the database are then used to cal-ibrate the online simulation results. The calibration proce-



dure first looks for one or more situations in the databasethat are closest (i.e., within a certain predefined threshold)in terms of “system condition” to the current brush’s state.Then by a simple linear interpolation, the procedure findsthe correct calibration to be applied to the current simulationresult. Here, “system condition” includes the current geom-etry of the hair macro and the differential of the system’s sixdegrees of freedom (DOFs), to be explained in Section3.5.Figures2 and3 show the dynamic deformation of a primitivegeometry and our virtual brush respectively.

3.1. Estimating Stress in Hair Macro

We adopt a simple strategy to estimate the inner stress of ahair macro. By Equation1, the geometry of a hair macro ismodeled as the volume of sweeping a variable ellipseE(t)along a B-spline. We denote the ellipse that lies on the nor-mal plane at the pointC(t0) in the steady geometry of thehair macro asEs(t0) = {(xs,ys)|xs = As(t0)νscosθs, ys =Bs(t0)νssinθs (0≤ θs < 2π; 0≤ νs ≤ 1)}. Here the sub-scripts indicates that the brush is in its steady state. Supposeafter a certain deformation of the virtual brush, the ellipsebecomesEd(t0) , {(xd,yd)|xd = Ad(t0)νd cos(θd), yd =Bd(t0)νd sin(θd) (0≤ θd < 2π; 0≤ νd ≤ 1)}. The subscriptd indicates that the brush model is deformed. Between theellipsesEs(t0) andEd(t0), we can establish a point to pointcorrespondence according to their coordinates in the(ν,θ)space. Two points(νs,θs) and(νd,θd) are correlated if andonly if νs = νd andθs = θd +τ, whereτ is a parameter to bedetermined later. Correlated points are supposed to representthe same hair thread inside the hair macro during painting.The thread has a displacement from its position(xs,ys) inthe steady geometry to its position(xd,yd) in the deformedgeometry, where the origins of the local coordinate systemsare the centers of the ellipsesEs(t0) andEd(t0) respectively.We can then compute the stressρ(xd,yd) at point(xd,yd) inthe ellipseEd(t0) as

ρ(xd,yd),{

tan(max{k1ϕ,− π2}) whenϕ≤ 0

tan(min{k2ϕ, π2}) whenϕ > 0

. (3)

Here,ϕ, 1−√

x2s+y2

s

x2d+y2

d, andk1,k2 are two positive real num-

bers that control the force response property of the virtualbrush. The assumption we use in this estimation is that thestress in any steady geometry of a hair macro is always thesame, which is a simplified conclusion from material science27.

Based on this point-wise stress estimation, we can derivethe formulae to evaluate the average inner stressρ(Ed(t))of the ellipseEd(t) and the average stressρ(Hd) of the de-formed hair macroHd by integrating the point-wise stressover the ellipse’s inner area and throughout the volume ofthe hair macro respectively. The positiveρ suggests that thestress is inward, which will compress the geometry of hairmacro; whereas the negativeρ suggests that the stress is

outward and will dilate the hair macro; zeroρ means thehair macro is in its steady geometry. In the above computa-tion, the value ofτ will affect the computation result. By ourdesign,τ(t) is supposed to be a number between[0,2π) tomake the absolute value ofρ(Ed(t)) minimum.

3.2. Deforming Geometry of Hair Macro

The brush’s geometry deforms under the outer force due tothe friction between the brush and the paper. According toour geometry model of hair macro (H , (A(t),B(t),C(t)),Equation2), the online simulation of the brush’s motionis carried out in two parts: deforming the skeleton of hairmacroC(t); and deforming the sweeping ellipses of the hairmacro, namelyA(t) andB(t).

3.2.1. Deforming of skeleton

The main constraint in deforming the skeletonC(t) of a hairmacro is thatC(t) cannot penetrate the virtual paper. If atsome moment the skeleton does penetrate the virtual paperat the pointC(t0), we will project the part of skeleton belowthe virtual paper, denoted asCb , C(t) (0≤ t < t0), ontothe virtual paper to getCp. We will also rotateCb aroundthe pointC(t0) by a minimum angle to getCr on which nopoint penetrates the virtual paper. And then we perform asimple linear interpolation betweenCp andCr to get an in-termediate curveCi which is used to replace the part ofCbin C(t). The reason behind this replacement can be foundin the study of material science27. If the perceptual mate-rial performance of the hair macro in our virtual brush ispurely rigid,Cp should be used to replaceCb because thehair macro will actually break atC(t0); if its perceptual ma-terial performance is purely flexible,Cr is the correct choicefor this replacement. Since we model the material of hairmacro as a kind of hybrid rigid-flexible material,Ci is a rea-sonable choice covering the two possible cases. The weightused in the linear interpolation to computeCi is a coeffi-cient representing the degree of rigidity-flexibility in termsof the material property of the hair macro. After the re-placement, we also ensure that the curvature of the skeletonwould not change too abruptly over a small neighbouringareaδ(t0) = (t0− ε, t0 + ε) aroundC(t0) in C(t). This curvesmoothing is done by uniformly samplingC(t) in the rangeof [0, t0−ε] and[t0+ε,1] and then applying a B-spline curvefitting to those sampled points. Finally the resultant B-splinecurve is used as the new skeleton for this hair macro, whichensures a smooth transition of the changing skeleton aroundpointC(t0).

We also simulate the kinking up of the hair macro due tothe friction between the paper and brush head. According toclassic Newton force, high stress would induce larger fric-tion. The larger the friction is, the slower the hair macrowill move. This non-uniform displacement can cause lo-cal prolongation and condensement on the skeleton of thehair macro. Denote the points on the skeleton of the hair



macro touching the paper asp1, p2, · · · , pn. According toEquation3, we can compute the stress at these points asρ(p1),ρ(p2), · · · ,ρ(pn). Let the mean stress of those pointsbe Sm. Suppose the displacement of the hair macro isD;then the displacement of pointpi on the skeleton would beρ(pi)Sm

×D. Thus the distances separatingp1, p2, · · · , pn willbe changed. Extended distances will pull the skeleton whileshortened ones will squeeze the skeleton. If squeezed ex-cessively, the skeleton will kink up. Suppose the distancesbetween pointspi and p j in the original and the squeezedskeletons arelo and ls respectively. We introduce a circulararcR to replace the distance to simulate the effect of kinkingup with R’s arc length betweenpi and p j being lo, andR’schordal length beingls. Here the chord ofR refers to the linesegment that connectspi andp j and which lies on the virtualpaper.

3.2.2. Deforming of ellipse

During dynamic deformation, those ellipses touching thevirtual paper,E(t)’s, will be deformed. Possible deforma-tions include: (1) rotation of the ellipse; (2) prolongation orcondensement of the ellipse’ major or minor axis, and (3)displacement of the centroid of the ellipse,ι. (1) and (2) willyield an inner stressρ(E(t)) for the ellipseE(t), estimatedby the method discussed in Section3.1. Our design picks thestrategy of minimizingρ(E(t))+ k3||ι||, with the hard con-straint that the deformed ellipse cannot penetrate the virtualpaper and the area of the ellipse cannot change, wherek3 isa positive real coefficient.

3.2.3. The degree of distensibility

The area of each sweeping ellipse in a hair macro repre-sents the number of hair threads in the macro being mod-eled. There is another parameter, which we call “degree ofdistensibility”, which affects this area. A high degree of dis-tensibility suggests that the hair threads are not very denselypacked inside the hair macro. For simplicity, we model thedegree of distensibilityβ as a function of the stress in thehair macroH as follows.

β(H), 12− 1

πarctan(k4×ρ(H)) (4)

Sinceρ(H)’s range is the whole real number axis, the rangeof β(H) is (0,1). Herek4 is a positive coefficient controllingthe sensitivity of the distensibility to the inner stress. Foreach hair macroH, if its previous and current degrees ofdistensibility areβ1 andβ2, all of H ’s major and minor axes

will have a length which isβ2β1

times their original values.

3.3. Hair Macro Splitting

If there exists a point in a hair macro, whose stress is abovethe threshold of a maximum tolerable inner stress, the hairmacroH will split. According to the way we compute thestress inH, discussed in Section3.1, the distribution of the

stress withinH is continuous. This means if there existsone such point inH whose stress is greater than the split-ting threshold, it is likely that the neighbouring points arealso above the threshold. Therefore, our splitting procedureworks on groups of points. For one specific sweeping ellipse,we need first of all to separate those points whose stress areabove the threshold from the other points. If the number ofconnected points whose stress are above the threshold is verysmall, there will be no splitting. This is the effect of physi-cal attraction between neighbouring hair threads. After thesmall group elimination, real splitting operation comes intothe picture. The separated point groups will become two el-lipses, the centers of which are the centroids of each group.To determine the length of major and minor axes of theellipse, we impose two constraints: the area of the ellipseshould be equal to the area of the group before splitting, andthe ratio of the major axis to minor axis of the ellipse is equalto the ratio of the length to the width of the bounding boxof the point group. The orientation of the major axis of thenew ellipse is determined by the principle axis of the group.Splitting the hair macro takes place in a stress-descendingorder if multiple point groups meet the splitting criterion.After splittingH at the largest average stress point group, thestress insideH is recomputed. Splitting continues as long asthe splitting condition holds.

3.4. Recovery from Deformation

To simulate recovery from deformation, we model the ma-terial of the hair macro as a kind of hybrid elastic-rigid ma-terial. According to material science27, there exists a steadygeometry for such kind of hybrid elastic-rigid material. If theinner stress of the material does not exceed its threshold ofelasticity, the material remains in an elastic state, in whichall the deformation can be recovered. In the elastic state,once the outer force exerted on the material is released, thebrush geometry will return to its original steady geometry.If the stress keeps increasing so that it exceeds the elasticitythreshold, the steady geometry will be “shifted”. Under thiscircumstance, the brush geometry upon releasing will onlyrecover partially. We implement this shifting by interpolat-ing between the original steady geometry and the current de-formed geometry. Beyond the threshold of elasticity, there isthe threshold of rigidity. The deformed hair macro will nothave any recovery at all if the stress exceeds this threshold.

3.5. Calibrating Simulation Results

As can be seen so far, all the dynamic brush geometry de-formation is simulated by fast online operations. To ensureaccuracy of the simulation with respect to a realistic brush,we introduce a procedure to calibrate the fast online simula-tion results. The procedure relies on data from a “simulationerror database”. These data were sampled previously using areal brush. There are two important advantages to this offline



database approach. First, we do not have to model the ex-tremely complicated physical process of a brush’s dynamicdeformation/recovery with high accuracy. Second, we canavoid the very time-consuming and probably unstable nu-merical computation to solve differential equations on thefly if some truly powerful but complex equations can be es-tablished to simulate all the underlying detailed dynamicsgoverning the brush’s deformation. The brush deformationdatabase operates at the level of hair macro in our two-levelhierarchy. This reduces substantially the number of differentcases that need to be separately sampled and stored in thedatabase.

Since the deforming of a hair macro’s geometry is ac-cording to the six DOFs of the input, the database is in-dexed by: (1) the differential of the virtual brush system’s sixDOFs (∂D) between the current and the previous simulatedtime slice; and (2) the current geometry of the hair macroH , {A(t),B(t),C(t)} (Equation2). The content of eachrecord in the database is the corresponding transformationT on H ’s current geometry, namelyA(t),B(t),C(t). Whencollecting records to establish the calibration database, wechoose the samples in such a way that no two similar brushdynamic deformation processes are sampled and stored.

The above keywords to index the database have many di-mensions. We construct a manageable database by takingadvantage of the inter-relationships among the fields of thekeywords. The product ofA(t0) andB(t0) reflects the area ofthe ellipseE(t0), and hence the number of hair threads in thehair macro under a certain distensibility. If the hair macrowill not split, B(t0) can be computed fromA(t0). Thus, wecan use four control points to represent the geometry defor-mation onA(t) and four control points forC(t). We also as-sume there is no inter-relationship between the dynamic de-formations onA(t) andC(t)—i.e., changing either one of thetwo splines will not have any direct effect on the other spline.This assumption comes from the observation that there isonly a weak relationship between deformations onA(t) andC(t) during a real brush’s deformation. As a result, the wholedatabase can be divided into two sub-databases:DA for cali-brating the simulated dynamic deformation onA(t), andDCfor calibrating the simulated deformation onC(t). For sim-plicity, we ignore the possible but minute deformation onthe brush geometry introduced by thex,y displacement ofthe virtual brush. So only four of the six DOFs will affectthe brush’s geometry deformation process. We also assumethat the effects these four DOFs have on the brush geome-try are separable. ThusDA andDC are further divided intofour sections, each of which is used to calibrate the error inthe simulation of the geometry’s dynamic deformation un-der the change of one DOF. For each of the sections inDAandDC, the index words are of low dimensions: one dimen-sion for the varied DOF and the remaining dimensions forthe control points ofA(t) in DA or C(t) in DC.

During database retrieval, we find the several records in

the database whose distances to the input index are within acertain threshold. The contents of the retrieved records areinterpolated using the distances between the indices of theretrieved records and the input index as the weights. Fordatabase retrieval, the distance between two splines is de-fined to be the sum of the euclidian distances of their corre-sponding four control points.

With the way we organize the database as describedabove, the database in our prototype implementation con-tains a modest number of records (40). This number ofrecords is already sufficient for performing a satisfactory cal-ibration to correct/improve the online simulation results.

4. Virtual Brush with Pigment

Figure 4: Single strokes by the virtual brush with black pig-ment.

4.1. General Pigment Diffusion Equations

Based on Fick’s1 diffusion analysis on a quantitative ba-sis, Fourier’s9 research on heat conduction, and knowledgefrom the textile industry14, we adopt Equation5 to describethe diffusion of pigment in the virtual brush and using thenomenclature of vector analysis:

∂C∂t

= div (D gradC) (5)

whereC is the concentration of pigment at a certain loca-tion and time inside the hair macro, andD is the diffusioncoefficient. In our design,D is supposed to be proportionalto the stress of points where the virtual brush and papertouch and where diffusion takes place, namelyD , k5ρ,wherek5 is a coefficient, and stressρ is evaluated by themethod introduced in Section3.1. After simulating the pig-ment diffusion process, a point on the virtual paper is coloredby taking the color value of the pigment as its color value

andmin{ CCmax

,1} as its alpha value. Here,Cmax works as a

threshold. If the concentration of pigment at a point exceedsthe threshold, the displayed color is the pigment’s originalcolor. Otherwise, some translucency is introduced to reflectthis unsaturation of pigment’s concentration. If multiple pig-ments are deposited at the same pixel, we interpolate thecolor values of all the pigments in theL,U,V color spaceto compute the color value. The interpolation weights arethe percentages of each pigment’s concentration against the

overall pigment concentration at the point.min{ Ca

Cmax,1} is

used as the alpha value for the color in this multiple pigment



situation, whereCa is the overall pigment concentration atthe point. In our virtual brush, diffusion is applied and sim-ulated only when the concentration of pigment is not uni-formly distributed.

4.2. Simulating Pigment Diffusion at Dipping Time

We model the physical process of brush dipping like thetwo-phase dying process in the textile industry. In the firstphase, we only simulate the radial diffusion inside the hairmacro. In the second phase, we compute the axial diffusion.In both phases of the diffusion process, the effects due tofluid dynamics are considered. Section4.2.1 discusses thegeneric equation describing pigment diffusion inside hairmacro; Section4.2.2and Section4.2.3discuss how to simu-late the radial and axial pigment diffusion respectively insidethe hair macro during brush dipping.

4.2.1. Equations for pigment diffusion

We view the geometry of the hair macro defined by Equation2 as a deformed cylinder. Given the pigment’s concentra-tion in a normal non-deformed cylinder, we can use a simpleaffine transformation to map each voxel in the hair macrointo the non-deformed cylinder. The curves ofA(t), B(t)are mapped to the lines ofA(t) ≡ R, B(t) ≡ R, andC(t) ismapped asC(t) = (0,0, t), whereR is the radius of the non-deformed cylinder. By this simple mapping strategy, we onlyneed to simulate pigment diffusion inside an ideal cylinder,to be discussed in the following.

Through transformation of coordinates by replacing(x,y,z) with (r cosθ, r sinθ,z), the equation for diffusionin a cylinder can be derived from Equation5 in terms of thecylindrical coordinates(r,θ,z) as follows.

∂C∂t

=1r{ ∂

∂r(rD

∂C∂r

)+∂

∂θ(Dr

∂C∂θ

)+∂∂z

(rD∂C∂z

)} (6)

4.2.2. Radial diffusion of pigment

For this case, pigment concentration is simplified to be afunction of radiusr and timet only. SupposeR is the radiusof the cylinder,C0 is the concentration of pigment in the inkbottle, andf (r) is the initial concentration of pigment beforethe current diffusion, which is useful when multiple brushdippings are simulated. Crank10 gives the analytic solutionto Equation6 under this condition as follows.

C(r, t) = C0{1− 2R ∑∞n=1

1αn

J0(rαn)J1(Rαn)

exp(−Dα2nt)}+

2R2 ∑∞n=1 exp(−Dα2

nt) J0(rαn)J2

1(Rαn)

Rr f (r)J0(rαn)dr

(7)where theαns are the positive roots ofJ0(Rαn) = 0, in whichJ0(x) is the Bessel function of the first kind of order zero,J1(x) is the Bessel function of the first order, andD is thediffusion coefficient.

4.2.3. Axial diffusion of pigment

Solutions of the diffusion equation in this case can be derivedas:

C(z, t) =M

2√

πDtexp(−z2/4Dt) (8)

wherez is the axial distance for diffusion, andD is the dif-fusion coefficient withM being the overall pigment concen-tration in the diffusion source area.

4.3. Simulating Pigment Diffusion During Painting

We model the physical process of painting using the virtualbrush as a dying process in which the pigment is continu-ously diffusing from the brush head onto the virtual paper atthe points of touching. We assume the virtual paper to be aflat plane. In the simulation, we also take into account theeffect due to fluid dynamics. For the ideal physical process,Crank 10 gives the solution to the the diffusion Equation6for the case of a continuous point source diffusing process—Equation9. To realistically render the current footprint ofthe brush on the paper, we simulate the pigment’s transfor-mation at the points of touching between the brush and thepaper, once for each point. Section4.4 discusses how thispart of the simulation can be accelerated. After the diffu-sion, the source point on the virtual brush has a decrease inconcentration by the amount of the sum of the increase ofpigment concentration on the virtual paper.

C(r, t) = q4πDr er f c( r

2√

Dt)

whereer f c(x) = 1− 2√πR x

0 exp(−η2)dη (9)

In Equation9, q is the ink transfer coefficient which is a con-stant,D is the diffusion coefficient, andC(r, t) is the amountof pigment concentration change introduced by the diffusionprocess for a point at timet and distancer from the sourcepoint.

4.4. Fast Solutions to the Diffusion Equations

Although the analytic solutions to the general diffusionequation (Equation6) can be derived as Equations7, 8, and9, evaluating the numeric values (for Equations7 and9) tosatisfy any real-time requirement is a non-trivial task. Fortu-nately, both solutions,C(r, t), in Equations7 and9 are func-tions of two variablesr andt. Thus we can generate a look-up table which enumerates the discrete combination of thesetwo variables(r i , t j ) under a certain diffusion coefficientDand their corresponding solutionC(r i , t j ). This look-up ta-ble is indeed anR3 → R1 mapping. During online simula-tion, rather than computing the numeric values of Equations7 and9, we search the table, leading to much saving of CPUtime. Suppose the current input to be evaluated isC(r0, t0),we retrieve four values from the table:C(r+

0 , t+0 ), C(r+0 , t−0 ),

C(r−0 , t+0 ), C(r−0 , t−0 ). Herer−0 is the maximum value in thetable which is smaller thanr0 andr+

0 is the minimum value



in the table which is bigger thanr0. Similarly for t−0 andt+0 .We then use an interpolation to deriveC(r0, t0) based on thevalues ofC(r+

0 , t+0 ), C(r+0 , t−0 ), C(r−0 , t+0 ) andC(r−0 , t−0 ).

5. Improving User Manipulability

As with any real brush, it can happen that the user wouldfeel unsatisfied with their creations using the virtual brush.Instead of training the user, our virtual brush system has theunique feature of training the brush. This is done through anintelligent user-manipulability improving component. Thiscomponent applies an additional transformation to the user’sinput before the system commits on the final painting result.

The idea behind the design of this component is as fol-lows. Our virtual brush carries with it a collection of beau-tiful strokes and the input for creating these strokes. Theuser can choose among these “known” brush strokes not forhis/her own strokes, but use them as training samples. Foreach selected sample, the user would then use our virtualbrush to produce a stroke as close to the sample as possible.The system then applies a numerical analysis to compute atransformation from the user’s input to that of the samplestroke. The derived transformation is the “personal habitualbias” of the user. Later, when the user paints using our virtualbrush, his/her input will firstly go through the transformationof his personal habitual bias. Thus, it is the transformed inputrather than his original input that drives the virtual brush topaint. The details are omitted due to the length of this paper.Please refer to the accompanying website for more technicaldetails on this feature of our virtual brush.

6. Experiment Results

Figure4 shows some single strokes by our virtual brush withblack pigment. Figure5 to Figure7 show some real resultsfrom using our virtual brush. Most of them took hours tocomplete, but at any moment during the painting process,our virtual brush running on a PC with256M memory andan AMD Duron 1200MHz Processor can respond interac-tively to user commands. Currently, we use a WACOM penon a tablet to get the position, the pressure (used as verticaldisplacement), and the tilt of the virtual brush; and keyboardinput to get the remaining two DOFs. This input strategy isvery much the same as the one used in Xu et al.’s brush sys-tem 24. A better input device in the future should make thepainting process much smoother and more efficient.

7. Conclusion

We have presented the design of a powerful painting virtualbrush system. We targeted at a realistic modeling by cov-ering as much minute details of the brush’s geometry andmotion as well as the delicate pigment behavior taking placeduring the painting process as possible. Such an approachhas made necessary the many time or space optimizations

Figure 5: “Summer water lily” by the virtual brush.

Figure 6: “Quiet village” by the virtual brush.

Figure 7: “Spring garden” by the virtual brush.

that we have introduced into the design. The result is a highdegree of realism as can be seen in the generated digital art-work.

There are still many interesting problems to be addressedin the future. One of them is that of choosing the appropri-ate samples for the brush motion calibration database thatcan enumerate all the possible motions a painting brushcould experience without repetitive sampling. For the cur-rently employed pigment model, only water-soluble pigment



is simulated. An obvious future task would be to extend themodel to cover oil painting and maybe other kinds of paint-ing by following Small’s 7 and Cockshott’s25 pioneeringapproaches. For the user manipulability improving compo-nent, finding the features that distinguish good input leadingto visually pleasing brush strokes from bad input could helpestablish a mechanism to perform auto-beautification.

References

1. A. Fick. Annln Physics170(59), 1855. 7

2. A.R. Smith. Digital Paint Systems: An Anecdotal andHistorical Overview. IEEE Annals of the History ofComputing, 2001. 1

3. A. Hertzmann. Painterly rendering with curved brushstrokes of multiple sizes.Proc. of SIGGRAPH ’98:453–460, 1998. 1

4. B. Baxter, V. Scheib, M.C. Lin and D. Manocha. DAB:interactive haptic painting with 3D virtual brushes.Proc. of SIGGRAPH ’01:461–468, 2001.1, 2, 3

5. C. Chan, E. Akleman and J. Chen. Two Methods forCreating Chinese Painting.Proc. of Pacific Graphics’02: 403–412, 2002.3

6. C.J. Curtis, S.E. Anderson, J.E. Seims, K.W. Fleis-cher and D.H. Salesin. Computer-generated watercolor.Proc. of SIGGRAPH ’97:421–430, 1997.2, 3

7. D. Small. Simulating Watercolor by Modeling Diffu-sion, Pigment, and Paper Fibers.Proc. of SPIE ’91,February 1991.10

8. H.T.F. Wong and H.H.S. Ip. Virtual brush: a model-based synthesis of Chinese calligraphy.Computers andGraphics24(1):99–113, 2000.1, 2, 3

9. J.B. Fourier. Theorie analytique de la chaleur, 1822.English translation by A. Freeman, Dover Publ., NewYork, 1955. 7

10. J. Crank.The Mathematics of Diffusion, Oxford Press,1975. 8

11. J. Lee. Simulating oriental black-ink painting.IEEEComputer Graphics and Applications19(3):74–81,1999. 2

12. M. Agrawala, A.C. Beers and M. Levoy. 3D paintingon scanned surfaces.Proc. of the 1995 symposium onInteractive 3D graphics:145–150,215, 1995.1, 3

13. M.P. Salisbury, S.E. Anderson, R. Barzel and D.H.Salesin. Interactive pen-and-ink illustration.Proc. ofSIGGRAPH ’94:101–108, 1994.1

14. M.R. Gregor. Diffusion and Sorption in Fibre andFilms, Vol. 1. Introduction with particular reference todyes. Academic Press, London, New York. 1974.7

15. N.S.H. Chu and C.L. Tai. An efficient brush model forphysical-based 3D painting.Proc. of Pacific Graphics’02. 1, 2

16. Pixologic. Z-brush, http://pixologic.com, 2000.1, 3

17. P. Hanrahan and P. Haeberli. Direct WYSIWYG paint-ing and texturing on 3D shapes.Proc. of SIGGRAPH’90:215–223, 1990.1, 3

18. Q. Zhang, Y. Sato, J. Takahashi, K. Muraoka and N.Chiba. Simple cellular automaton-based simulation ofink behavior and its application to Suibokuga renderingof trees.Journal of Visualization and Computer Anima-tion, 1999. 2

19. R.D. Kalnins, L. Markosian, B.J. Meier, M.A. Kowal-ski, J.C. Lee, P.L. Davidson, M. Webb, J.F. Hughes andA. Finkelstein. WYSIWYG NPR: drawing strokes di-rectly on 3D models.Proc. of SIGGRAPH ’02:755–762, 2002. 1

20. S.C. Hsu and I.H.H. Lee and N.E. Wiseman. Skeletalstrokes.Proc. of SIGGRAPH ’93:197–206, 1993.1

21. S.C. Hsu and I.H.H. Lee. Drawing and animation us-ing skeletal strokes.Proc. of SIGGRAPH ’94:109–118,1994. 1

22. S. Saito and M. Nakajima. 3D physics-based modelfor painting.SIGGRAPH ’99 Sketches, Conference Ab-stracts and Applications:226–226, 1999.1

23. S. Strassmann. Hairy brushes,Proc. of SIGGRAPH’86:225–232, 1986.1, 2, 3

24. S. Xu, M. Tang, F.C.M. Lau and Y. Pan. A solid modelbased virtual hairy brush.Computer Graphics Forum(Proc. of Eurographics ’02), 21(3):299–308, 2002.1,2, 3, 4, 9

25. T. Cockshott. Wet and Sticky: A novel model for com-puter based painting. PhD. Thesis, Computing ScienceDepartment (Research Report 91/R20), University ofGlasgow, 1991.10

26. T.L. Kunii, G.V. Nosovskij and T. Hayashi. A diffusionmodel for computer animation of diffuse ink painting.Computer Animation, 1995. 2

27. W.D. Calister and W.D. Callister Jr.Materials Scienceand Engineering: An Introduction, Wiley, John & Sons,Incorporated, 1999.5, 6

28. X. Wei, S. Lu, M. Song and B. Luo. Computer pat-tern design and painting technique based on aesthet-ics knowledge.Computer Aided Drafting, Design andManufacturing, 2(2):32–40, 1992.3


advanced design for a realistic virtual brushi.cs.hku.hk/~fcmlau/papers/eg03.pdf · advanced design...

Documents