Animation of Moving Soil and Construction Machinery

Hui GongDepartment of Computer Science

University of Regina

April 2014

Abstract

The goal of this project is to design a system that creates realistic animations of

moving soil and construction machinery. To simplify the implementation, we use the

Unity game engine.

Terrain is based on a heightmap, which is used for the built-in terrain in Unity.

Pouring dirt on to the ground is treated as increasing the height at some location. Since

increased height can only be applied to the grid coordinates on the heightmap, we

distribute the added height to the nearest four coordinates, weighted according to

distance. Any dirt particles above the critical slope angle are moved and the height of the

surface is changed accordingly. When moving dirt, we examine all grid points in a circle

of radius r and increase the height at the most suitable of these points.

To demonstrate the interaction of a typical machine with the virtual soil, we obtained

and rigged a model of an excavator featuring two tracks, a long arm, and a bucket on the

end of the arm. We defined each track as a series of pieces moving according to physics.

Since the digging arm of the excavator has three movable parts, we rigged the two joints

between them. To avoid costly collision checks between the bucket sub-model and the

terrain, we selected only 56 collision points to check along the back surface of the bucket.

By gradually generating dirt particles while digging, we converted part of the volume

of dirt represented by the 2D heightmap to volume represented by 3D dirt particles. To

allow dirt particles to collide in the bucket, they are represented as cubes, each with an

attached particle system. To achieve realistic effects, we restrict the parameters for cube

particles of dirt before they hit the ground. When the particles hit the ground, piles of dirt

are formed and the volume represented by dirt particles is converted to volume

represented by the 2D heightmap volume again.

We evaluated the performance of our implemented approach by observing its running

time in frames per second while varying the heightmap resolution, the number of

particles, and the number of models. According to the results, the frame rate goes down

as any of the heightmap resolution, the number of particles, and the number of models is

increased.

Table of Contents

1. Introduction......................................................................................................................12. Background......................................................................................................................23. Approach..........................................................................................................................93.1. Deformation and texturing of the terrain where dirt falls...........................................103.1.1. Preprocessing...........................................................................................................143.1.2. Soil placement.........................................................................................................153.2. Rigging of the excavator model and definition of joints for the arm and tracks........213.2.1. Definition and rigging of the arm............................................................................233.2.2. Definition and rigging of the track..........................................................................253.3. Deformation of the terrain when the bucket of the excavator hits the ground...........293.3.1. Digging....................................................................................................................293.2.2. Picking Up...............................................................................................................333.3.3. Holding....................................................................................................................333.3.4. Pouring.....................................................................................................................343.3.5. Hitting......................................................................................................................354. Implementation Details..................................................................................................374.1. Heightmap resolution of the terrain............................................................................374.2. Rigging joints on the excavator..................................................................................394.3. Particle soil.................................................................................................................424.4. Volume conversion.....................................................................................................445. Results............................................................................................................................485.1. Effects achieved..........................................................................................................495.2. Evaluation of the system.............................................................................................535.2.1. Varying the heightmap resolution for the terrain.....................................................535.2.2. Varying the number of dirt particles........................................................................555.2.3. Varying the number of models.................................................................................566. Conclusions and Future Work........................................................................................60References..........................................................................................................................62Appendix A........................................................................................................................66A.1. Switchable camera.....................................................................................................66A.2. Machinery smoke.......................................................................................................67

List of Figures

Figure 1. Unity project structure..........................................................................................7Figure 2. Unity scene hierarchy...........................................................................................7Figure 3. Adding height distribution..................................................................................10Figure 4. Calculation of slope angles based on the heightmap..........................................13Figure 5. Pile of soil with r = 2.3.......................................................................................15Figure 6. Gauss Circle Problem.........................................................................................16Figure 7. Comparison of the excavator models (original in color)....................................22Figure 8. Rigging the arms of the excavator and road header (original in color)..............23Figure 9. Rigging the tracks (original in color).................................................................25Figure 10. Deformation of the terrain (original in color)..................................................29Figure 11. Positions for collision checking on the bucket (original in color)....................30Figure 12. Unity terrain heightmap...................................................................................39Figure 13. Excavator rigging (original in color)................................................................41Figure 14. Dirt particles (original in color).......................................................................43Figure 15. Heightmap structure of terrain.........................................................................44Figure 16. Nearest six tetrahedrons...................................................................................46Figure 17. Total heightmap height change is 0..................................................................47Figure 18. Excavator digging a hole in the ground (original in color)..............................49Figure 20. Excavator forming a soil pile from dirt (original in color)..............................52Figure 21. Movement of excavator model (original in color)...........................................52Figure 22. Road header (original in color)........................................................................53Figure 23. Different heightmap resolutions.......................................................................54Figure 24. With a heightmap resolution of 257, the frame rate was around 6 fps.............54Figure 25. Particle count vs. approximate frame rate........................................................55Figure 26. Frame rate is around 25 fps, when holding a full bucket of dirt......................56Figure 27. Varying the number of road headers.................................................................58Figure 28. Varying the number of excavators....................................................................59Figure 29. View from inside the excavator (original in color)..........................................66Figure 30. Smoke from the excavator (original in color)..................................................67

List of Tables

Table 1. Radius r vs. pile shape.........................................................................................17Table 2. Heightmap resolution vs. approximate frame rate...............................................53Table 3. Particle count vs. approximate frame rate............................................................55Table 4. Number of road header models vs. approximate frame rate................................57Table 5. Number of excavator models vs. approximate frame rate...................................57

1. Introduction

The goal of this project is to create a realistic animation of moving soil and

construction machinery. In particular, we want to implement the ability to form piles of

soil, to animate an excavator, and to animate the interaction of both of these with the

ground and each other. The goal of in this project is to combine the terrain changes with

particle generation, to accomplish the conversion back and forth between 2D terrain

height data and 3D particle volume data, and to implement a flexible operating

environment for simulated construction machinery that interacts with simulated soil. To

simplify the implementation, we use the Unity game engine. This game engine has its

own physics engine and supports the use of JavaScript or C# to implement the animated

components.

The remainder of this document is organized as follows. Chapter 2 describes

previous research on the physics of soil, the idea of the critical slope angle, and particle

systems for soil and similar substances in motion. Chapter 3 describes our approach to

the problem. We describe how to rig a construction machine called an excavator and how

to form dirt piles. Chapter 4 describes some implementation details. Chapter 5 describes

the results. Chapter 6 presents our conclusions and discusses possible future work.

2. Background

The increasing speed and size of the graphics processing units (GPUs) on computers

has allowed for more detailed simulations of reality, which are frequently used in

computer animated films and games. With the increasing use of this technology for

serious purposes [1], aspects of reality that are of little interest in films and games must

also be simulated. One such aspect is the interaction of machinery and soil on the ground.

This is of particular interest when creating animated simulations for the purpose of

training people to use specialized construction equipment.

Physically-based simulations have drawn considerable attention from those in the

computer animation and game fields, since they can produce many realistic special

effects. However, as noted below, many existing techniques are too complex to perform

in real-time.

Several techniques have been proposed for the simulation of ground surfaces and

particle behaviors. Previous systems include virtual sculpting systems from Galyean and

Hughes [2] and virtual clay modeling systems from Oda et al. [3] and Matsumiyael et al.

[4]. These systems collect voxel data from input devices to redraw a structure.

When implementing a virtual world where other models can interact with the ground,

it is useful to convert a portion of, or the entire ground surface, into many particles,

which represent dirt, sand, clay and similar substances. Large particles that we can

distinctly perceive are called granular material [5]. Particles that represent granular

material exist in a broad range of contexts. Granular material is the second most

manipulated material in industrial contexts [6]. Granular material is different from other

materials, such as liquids or solid objects, because of the variety of ways in which it can

behave. Multiple particles frequently interact with each other. When they are scattered in

the air, they can break into pieces. When they are pushed, they can flow like a fluid.

When they are lying on the ground, they act like part of the surface.

To simulate all of these behaviors, it is worthwhile to find an efficient method to

simulate soil treated as either ground or particles, depending on the context.

To determine when particles flow, Li and Moshell [7] propose the idea of a critical

slope angle, which is the maximum slope such that dirt does not slip. This angle depends

on the cohesion of the soil, the size of the particles, and the moisture level [7, 12].

For the simulation of dirt and its movement, Li and Moshell [7] propose a slippage

model to simulate the flow of soil. They divide the soil into chunks, focus on force and

the height at 2D coordinates, and analyze a model based on a bulldozer and a bucket that

allow animation in real time. But there are some limitations to their approach. For

example, the method cannot deal with the 3D effects of soil particles; as well, it does not

apparently work with a rotating bucket, since it is difficult to keep the volumes fixed.

To simulate the ground surface, Chanclou et al. [8] propose a physical model for the

changes that are made to the surface when an object interacts with it from above. They

analyze the surface positions point by point as a plastic sheet. Their research focuses on

soil. They divide soil into several layers, analyze friction and cohesion for each layer, and

propose suitable physical equations for each layer. Their results are based on the data in

the highest layers. Unfortunately, the processing is slow and their model cannot deal with

particles that are being dug up.

When it comes to the movement of the particles, Sumner et al. [9] describe a virtual

model to simulate the changes that occur when an object hits the ground, which is called

dynamic motion. The method is applied to feet hitting the ground to form footprints in

sand, mud and snow. It is not suited to deforming the ground when digging is performed.

Zeng et al. [10] propose a method to dynamically test for collisions between moving

objects and the surface. They simulate a ground surface using columns, which are divided

into grids of a heightmap. Each column has a volume represented by the center top height

of that column. As an object interacts with a surface, ray casting is used to detect the

collision and to distribute the ground column to its surroundings in a manner that is

consistent with Mohr–Coulomb theory [31].

To render particles, Nishita et al. [11] provide a model to simulate snow as metaballs,

which are placed on various objects. The method may be well-suited to simulating dirt

particles as they are being dug up. For particle accumulation, Fearing [12] uses the term

angle of repose, which is similar to the critical angle referred to by Li and Moshell [7].

They also perform stability tests of snow located on top of a variety of objects.

Generally, there are two approaches to combining surfaces and particles when

interaction and real-time rendering are required. The smoothed particle hydrodynamics

(SPH) approach, which simulates a material composed of many particles [13], and the

height field method [23], which maintains a 2D grid of heights.

The SPH approach has the advantage of allowing for more local deformation than

does the height field approach. Moreover, the frame rate varies depending on the number

of particles. Müller et al. propose the idea of surface tracing [14], in which they use a

particle-based method to calculate the shape, and a marching cube method to reconstruct

the surface, of the fluid. They use it to simulate water in a container at 5 fps. Müller et al.

[15] model objects using SPH, which creates a mesh-free model that can change the

shape of the object by controlling the threshold corresponding with the surface. A large

scale particle-based fluid simulation is carried out by Hegeman et al. [16]. A special tree

structure is used to dynamically maintain a particle list in the graphics processing unit

(GPU).

The Lagrangian method, also based on particles, considers an object to be a set of

material particles moving with time. The grid of each particle unit changes shape with the

flowing of the material. Ihmsen et al. (2012) [20] and Ihmsen et al. (2013) [21] showcase

a highly detailed visualization of sand by simulating the forces between particles using

the Lagrangian method [22]. Their result provides a realistic simulation of falling sand

particles. While their method is suitable for the animation of sand simulations, it is not

fast enough to be used in games.

The discrete element method (DEM) simulates the mechanical behavior of a material

that is primarily composed of many particles that can collide with each other and stick to

one another, such as that which occurs during the accumulation of particles.

Rungjiratananon et al. [17] present a model using both SPH and DEM [18] to simulate

wet sand passing through granular materials. Their results include falling sands of

varying levels of moisture. Lenaerts and Dutré [19] consider sand interacting with a fluid.

Their method allows particles to change from dry to wet in animations.

A heightfield or heightmap is an image that stores terrain height information in a

two-dimensional map with indices corresponding to individual locations. The image is

usually encoded as 8-bit grayscale [32]. Pixel shading values are mapped to the terrain

height data. If the size of a 2D heightmap is n x n, then the heightmap resolution is n. A

2D heightmap with resolution n defines a grid of (n – 1) x (n – 1) cells (i.e., spaces

between the grid points); here the n – 1 is the grid size. Height field methods are

generally appropriate when the terrain is of significant size and does not have any

overhangs or caves. These methods represent granular material as 2D terrain [23]. One

height field model, the BCRE model (Bouchaud, Cates, Ravi Prakash, and Edwards

[24]), has been accurately applied to separate layers of different granular materials. The

BCRE model contains two heightmaps, one representing the static layer and the other the

flowing layer. The BCRE method is limited as it cannot model all 3D dynamic behaviors

of a granular material. For example, it cannot simulate 3D effects such as caves or

vertical flows because it is based on a heightmap.

One of the most challenging problems in granular material simulation is that of

integrating 2D and 3D techniques into a single framework while achieving efficiency.

Langston et al. [25] extend the concept of DEM from 2D to 3D by analyzing the

continuous interactions between column layers. In 2005, Bell et al. [26] employ DEM for

the purpose of granular materials animation. Pla-Castells et al. [27] propose a model to

simulate a bulldozer on sandy ground. Their model can simulate a bulldozer pushing sand

upwards. As well, all necessary physical calculations are carried out in real time.

More recently, Zhu and Yang [28] proposed a hybrid method for the simulation of

granular materials, such as sand, in a 3D scene. Their method allows for the dynamic

motion of 3D particles, while using a 2D heightmap to implement an invisible static layer

underneath. Mobile particles are updated using both a DEM and a BCRE model. This

method saves memory and computation time, but cannot run in a real-time terrain

simulation when more than 40,000 surface particles are required.

Background information about the Unity game engine is also relevant to this report.

Figures 1 and 2 show the Unity project structure and Unity scene hierarchy, respectively.

Object models: Can be OBJ or FBX types; include object hierarchy, rig, mesh and animationScripts: Create scriptable gameobject, can change menusPrefabsMaterialsPictures: Textures or surface normalsSoundsPhysical materials: parameter settings for static and dynamic objects, friction typeTerrain dataScenes: see Figure 2

Figure 1. Unity project structure

Separate gameobjects for camera, light, terrain, each dirt particle, etc.The gameobjects for excavator and road header are hierarchicalEach gameobject has multiple components, which can be any of the following:

Mesh: Apply textures to a gameobjectEffects: Particle systemPhysics

Transform: record transformation properties of gameobjectsRigidbody: Apply mass drag and gravity, etcCollider: Apply collision detection for gameobjectsJoint: Constrains on motion, which depend on joint type

Audio: Audio listener; audio sourceRendering: SkyboxScripts: JavaScript; C#

Figure 2. Unity scene hierarchy

In particular, colliders and the update cycle are relevant. A collider is an optional

component that can be defined for objects in Unity. Its purpose is to register any

collisions that are detected with the specified object. In Unity, an existing object can be

associated with the choice of a basic shape collider, a mesh collider, or a generated

convex collider. Several basic shapes, such as box, capsule, and sphere, can be chosen for

colliders. The shape of a mesh collider is represented by the object mesh, which allows

for more accurate detection of collisions. However, the mesh collider is usually of a more

complex shape and is therefore slower at performing collision detection than is a basic

shape collider. A collider with a convex shape can be generated from a mesh. As a result,

a convex collider has a less complex shape and better performance than does the

corresponding mesh collider.

The update cycle in Unity can be specified to be either Update or FixedUpdate. The

former is performed once for every graphics frame. The latter deals with physics or rigid

bodies, such as updating a force or changing a movement. It has a maximum allowed

time step for updating physical interactions. As described in Chapter 4, both update

cycles are used in our implementation. When digging dirt, we use the Update function to

check for collisions with specified points of the bucket. When dropping dirt from the

bucket, if a number of dirt particles hit the ground, we use a FixedUpdate to obtain an

immediate update. If this is not done, the dirt particles will collide with each other,

thereby generating extra movements.

3. Approach

In our implementation, terrain is based on a heightmap, which is the method used for

the built-in terrain in Unity. If the heightmap had higher resolution, the terrain would be

more detailed and more CPU time would be consumed in calculating information about

the surface.

Dirt is simulated using many cubes, each with a collider, which act as dirt particles.

Piles of dirt are formed by moving the particles around. We ensure that the surface is

sloped at no more than the critical slope angle after the dirt comes to rest. Resting

particles above the critical slope angle are removed and the height of the surface is

changed accordingly.

The main components of our solution are given as follows:

1. Deformation and texturing of the terrain where dirt falls;

2. Rigging of the excavator model and definition of joints for the arm and

tracks;

3. Deformation of the terrain when the bucket of the excavator hits the

ground.

Each of these components is discussed in what follows.

3.1. Deformation and texturing of the terrain where dirt falls

Figure 3. Adding height distribution

Since our approach is based on a heightmap, the height of the terrain is represented

as if it consisted of the heights of poles at individual grid coordinates, as shown in Figure

3. Suppose dirt with height H0 is added at point P0. in 3D space, which is vertically

aligned with coordinates (nx, ny) in the grid space. The variable wx represents the distance

from the left boundary of the grid cell to the coordinates (nx, ny) in the x direction of the

terrain, and wy is the distance from the bottom boundary of this grid cell to the

coordinates (nx, ny) in the y direction of the terrain. Since the increased height can only be

applied to the grid coordinates on the heightmap, we distribute the added height from

position P0 to the nearest four grid coordinates, here called P00, P01, P10, and P11. To add

the small height b from each dirt particle to the heightmap, such that the height added to

each coordinate is related to its distance from the point of impact, we use the formulas

below.

// lower-left coordinate P00 = ( floor(nx), floor(ny) )H00 = b * ( 1 - wx ) * ( 1 - wy );

P00.height += H00;

// upper-right coordinate P01 = ( floor(nx) + 1, floor(ny) )

H01 = b * wx * ( 1 – wy );P01.height += H01;

// lower-right coordinate ( floor(nx), floor(ny) + 1 )H10 = b * ( 1 - wx ) * wy;P10.height += H10;

// upper-right coordinate ( floor(nx) + 1, floor(ny) + 1 )

H11 = b * wx * wy;P11.height += H11;

To ensure that the dirt is stable, we check that it does not exceed the critical slope

angle. So after distributing the increase in height at the four coordinates as described

above, we check the slope angle at each of these coordinates against the critical angle. We

check the slope angle using the four heightmap coordinates instead of P0 because they are

available in the representation. Recall that the critical slope angle is the maximum slope

angle for which the dirt does not slip. Consider position Pij to which we have added

height Hij. For example, suppose that H1, ..., Hn are the heights of n nearby poles at grid

coordinates P1, ..., Pn. The slope angle ak between any pair of grid coordinates (e.g. Pij

and Pk) is calculated from the difference ΔH = Hk - Hij in the height of the poles as

follows:

ak = arctan ( H k - H ij

distance ( Pij , Pk ) ),

where k = 1, 2, …, n.

Assuming that the critical slope angle is θ, then we repeatedly choose the nearby

point Pk for which the angle ak is greatest and move dirt from Pij to Pk. The pseudocode

for determining slippage is as follows:

// move dirt until all points are below critical angledo {ak = max( /* a1, a2, …, an */ );

// slip occurs?if(ak > θ) { /* slip, one unit of dirt goes to Pk */ }

} while(ak > θ);The method that we use to deform the terrain when dirt falls is one of adjusting the

height of the surface at each grid point in a circle around the location where the dirt falls.

We use the terrain data of the terrain object to change the pile heights on the heightmap

near (nx, ny). The goal is to form a round pile with a slope of θ.

(a) Calculation of slope angles at Pij

(b) Offset heights of neighbors

Figure 4. Calculation of slope angles based on the heightmap

We now describe the approach to adding dirt to terrain in more detail. In Section

3.1.1, we explain the preprocessing steps and in Section 3.1.2, we give the detailed

algorithm.

3.1.1. Preprocessing

1) Get the heightmap 2D coordinates (nx, ny) from the 3D location P0 = (x, y, z) where

the dirt is falling;

float nx = InverseLerp( 0, terrain size, dirt localPos(x) )

* ( heightmap.width - 1 );float ny = InverseLerp( 0, terrain size, dirt localPos(z)

) * (heightmap.height - 1 );

2) Set the fractional parts of the coordinates (nx, ny) as weight parameters wx and wy;

float wx = nx - floor(nx);float wy = ny - floor(ny);

3) Let b be a float variable representing the total height of the particles deposited

from all dirt particles in a given frame. Set b to be a fixed value related to a particle’s

volume (for details see Section 4.4) when initializing the terrain to ensure that the terrain

changes exactly by the same amount when each dirt particle is falling. Note that if b is

too small, there will be no change to the height of the heightmap.

float b = terrain size / (heightmap.width - 1);

4) Add height H00, H01, H10 and H11 to the four grid points in the heightmap.

3.1.2. Soil placement

Figure 5. Pile of soil with r = 2.3

To ensure that the method applies to any shape of terrain, we first distribute the

height to the nearest grid points. Then we distribute it again from those points and so on.

By using these local distributions, we avoid the need to consider the overall shape of the

terrain. To form a round pile, we examine all grid points in a circle of radius r from H0,

for example r = 4. The most appropriate value for r depends on the heightmap resolution

and the detail we want to achieve. Values of less than √5

tend to result in square piles.

Let n represent the number of neighbors located in a circle of radius r. The best value for

r can then be determined by solving the Gauss circle problem [29].

Figure 6. Gauss Circle Problem

As Figure 6 shows, Gauss proved that the number of integer coordinates N(r) located

in a circle of radius r have the following relationship:

N(r) = π

r2 + E(r),

where |E(r)| ¿ 2√2 π r. We define n = N(r) – 1, since we do not count the center as a

neighbor of itself.

For all neighbors, we determine the angle between the tops of the neighbor’s pole

and the pole at the current point Pij. If any angle is greater than the critical slope angle,

then move one unit height of dirt toward one of the directions where the angle is greatest,

repeating the process until no angle is greater than the critical slope angle. In this case,

we cannot equally distribute the amount among several poles, as moving less than one

unit of dirt will not change the height because of imprecision in the representation of

numbers in the physics engine of Unity.

If a sufficient number of neighbors are used, then the result will be a pile shaped

roughly like a cone. Table 1 shows the number of neighbors and the resulting pile shape

for a range of possible r. For example, if r is 4, then n is 46 and the shape has 20 edges.

Table 1. Radius r vs. pile shape

r n Base Shape

1≤r<√2 4 Square

√2≤r <2 8 Square

2≤r<√5 12 Square

√5≤r<2√2 20 Octagon

2√2≤r <3 24 Square

3≤r<√10 28 16 edges

√10≤r<√13 36 Octagon

√13≤r <4 42 Octagon

4≤r<√17 46 20 edges

The algorithm for creating cone-shaped dirt piles, given as function CreatePile, is as

follows.

1) Store neighbor’s position and height in an array named neighborh for temporary

use. In the code below, unit is the conversion factor used to convert between terrain

coordinates and heightmap coordinates and MAX_SLOPE_ANGLE is the critical slope

angle θ (in radians).

#define MAX_SLOPE_ANGLE ( 30 / 180f * Mathf.PI )float unit = td.size.x / ( td.heightmapWidth – 1 );float[] neighborh = {td.heightmapWidth,

td.heightmapHeight,0};

2) Generate a roughly circular shape of radius r to cover all nearby integer

coordinates. Select those coordinates at which there is a maximum offset height (see

Figure 4(b)) to corresponding height, based on the maximum slope among the neighbor’s

coordinates in the circle range, and the height in the neighbor array mentioned in step 1.

In the loop below, nc is the neighbor’s maximum offset height based on the coordinates’

distance and the maximum slope angle.

// current point (i,j) in circle around point (x,y)int i, j;

// number of points distributed horizontally and// vertically across circleint nx, ny;

// neighbor at which maximum offset height occursfloat neighborh[3];

ny = Mathf.FloorToInt(r);

// process all points vertically across circle radiusfor(j = y - ny; j <= y + ny; j++)

{nx = Mathf.FloorToInt(

Mathf.Sqrt(r * r - (j - y) * (j - y));// process all points horizontally across circle

radiusfor(i = x – nx; i <= x + nx; i++){

// check to ensure that point (i,j) is not at the// center of the circle and not beyond the edge of// the terrain and is in the circleif((j == y && i == x) ||

(j < 0 || j > td.heightmapHeight – 1 || i < 0 || i > td.heightmapWidth - 1)||(j - y) * (j - y) + (i - x) * (i - x) > r * r) continue;

// determine the height based on the distance between // it and the center of the dropping coordinatesfloat nc = Mathf.Tan(MAX_SLOPE_ANGLE) *

Mathf.Sqrt((i - x) * (i - x) +(j - y) * (j - y)) * unit;

// if this height is lower than the determined height,

// then update the selection of the neighbor float tdheight = heightmapData [j, i] *

td.heightmapScale.yif ((tdheight < height – nc)) &&

(height – nc – tdheight> neighborh[2])){

neighborh = new float[3]{i,j, height - nc – tdheight};

}}

}

3) Determine whether to let the dirt at this coordinate flow to the neighbor found

above or not. If so, do each of the steps below using this neighbor. In the following code,

c is the maximum offset height of the neighbor found above.

float c = Mathf.Tan(MAX_SLOPE_ANGLE) * Mathf.Sqrt((neighborh[0] – x) * (neighborh[0] - x) + (neighborh[1] - y) * (neighborh[1] - y)) * unit;

// positive offset?if( neighborh[2] > 0 ){

// determine additional height to addfloat extrah = height –

Mathf.Max( td.GetHeight((int)neighborh[0], (int)neighborh[1]) + c, td.GetHeight(x, y));

// increase height of pileCreatePile((int)neighborh[0],

(int)neighborh[1], td.GetHeight((int)neighborh[0],

(int)neighborh[1]) + extrah);

// record change in height in heightmapif (y >= 0 && y <= td.heightmapHeight – 1 &&

x >= 0 && x <= td.heightmapWidth - 1)heightmapData[y,x] = ( height - extrah ) /

td.heightmapScale.y;}else if(y >= 0 && y <= td.heightmapHeight – 1 &&

x >= 0 && x <= td.heightmapWidth - 1)heightmapData[y,x] = height / td.heightmapScale.y;

4) Change texture when dirt is falling on the ground. When the height at a

neighboring location is above that allowed according the maximum slope, we assume that

the location is already solid. In this case, we do not change the textures on it.

if(td.GetHeight(x, y) <

td.GetHeight((int)neighborh[0], (int)neighborh[1]) + c)

// process points (i,j) around current point (x,y)for(j = -1; j <= 0; j++)

for(i = -1; i <= 0; i++)for(k = 0; k < td.alphaMapLayers; k++){

// is dirt falling outside of the map?if(y + j < 0 ||

x + i < 0 ||y + j >= td.heightMapHeight – 1 ||x + i >= td.heightMapWidth – 1) continue;

// change texture layer number to the maximum, // if it represents the texture of dirt;// otherwise, change that layer to the

minimumif(k == 0)

splatMapData[y + j, x + i, k] = 1;else

splatMapData[y + j, x + i, k] = 0;}

3.2. Rigging of the excavator model and definition of joints for the arm and tracks

To demonstrate the interaction of a typical machine with the virtual soil, we used an

excavator. A model of an excavator was obtained from Turbosquid, Inc. [30]: The original

model is shown in Figure 7(a). This model has a relatively few polygons (17,767) and

uses low-detail textures, making it suitable for real-time animation. The surface normals

in the original model were not all correctly set. We used Maya to reverse any that were

pointing inward. The result was exported from Maya in the FBX format and then

imported into Unity. The rigging information in the original model was in a format

compatible with Maya but not with Unity, so we used Maya to delete it. We therefore

rigged the model again using Unity. The tracks in the model were replaced since the

original tracks were not suitable for control by Unity. Specifically, the original track

model performs scaling while rolling, i.e., it increases the size of the pieces that are in the

middle of the track and reduces their size when they are at the two end of wheels, which

is not suitable for physics control. The revised model is shown in Figure 7(b). The

visible change is that the tracks are totally different.

(a) original model

(b) revised model

Figure 7. Comparison of the excavator models (original in color)

3.2.1. Definition and rigging of the arm

(a) Rigging the arms of excavatavor

(b) Rigging the arms of read header

Figure 8. Rigging the arms of the excavator and road header (original in color)

The digging arm of the excavator has three movable parts, as shown in Figure 8(a),

namely the upper arm, lower arm, and bucket. There are also a number of movable

pistons. The front arm of the road header has similar arms and a head instead of a bucket,

as shown in Figure 8(b). Since the machines are similar, we will describe the excavator.

(1) Since Unity requires the bone structure to be in a hierarchical form, the parts of

the excavator are organized as follows:

The body is divided into three parts: upper body, base, and tracks;

The upper body is divided into two parts: arm and main body;

The arm consists of a lower arm and the remainder;

The lower arm consists of a bucket and the remainder.

(2) The joints of the arm are represented using a Unity configurable joint, which

defines the axis and limits of rotation. The purpose of the joint is to maintain the

connection between the parent part and the child part when the child part moves. It is

essential that both parts have rigid body components. Thus, a small amount of weight is

added to the arm. The weights of the two connected game objects should be roughly

equal so that the joint is steady and the colliders on the two arms do not cause them to be

jumpy. We rig the arm to allow it to move in only one direction. To let the movement of a

parent object drive a child object, we use a velocity type drive, which prevents the

backward movement of the parent. To ensure that movement stops correctly, we change

to the position type drive after calculating the target position for the child part.

(3) The pistons on the arm can be regarded as subordinate parts. The body and jack

of the piston are rigidly attached together, such that the body of a piston behaves as if it is

moving in and out. Forces are not calculated for the pistons. To ensure that the two parts

of the piston remain facing each other, the following code segment is used.

void LateUpdate () {From.transform.LookAt(

To.transform.position, From.transform.up);

}

3.2.2. Definition and rigging of the track

(a) The track model for the excavator

(b) Pieces in the track model

Figure 9. Rigging the tracks (original in color)

The track model for the excavator is shown in Figure 9(a). The motion of the track

can be implemented either by simply changing the picture (using texture UV shifting on

the texture) or by defining the track as a series of physically moving pieces. The latter

method was selected for this work, as shown in Figure 9(b). The procedure for creating

the track model consists of the following steps.

(1) Define a collider for each piece of the track.

(2) Define a hinge joint to connect each pair of adjacent pieces.

(3) Define appropriate driving wheels.

(4) Set the amount friction for the wheels.

The first two steps are straightforward, but some further explanation on the last two

steps is needed. With respect to the third step, we use the back wheels as the driving

wheels. This choice results in steadier motion. We set the wheels to be attached to the

track, and set the wheel’s physical material to be of maximum friction. As the wheels

move, they stick to the track and the friction forces some of the pieces of the track to

move. Finally, the whole track moves as if the pieces are connected together. It is

important to carefully choose the physical materials used. We can define different values

for the friction of different game objects. It is important to choose the multiply option for

Unity’s friction combine method for combining the friction due to the group of colliders

related to a track; otherwise the track will slide or jump since the force is not equally

distributed. The driving part of the wheel should include the angularXMotion parameter

of the configurable joint component, to free the rotation along the wheels. It should also

include the AddRelativeTorque parameter, to add force to each wheel.

When the friction settings of the game objects are set to multiply, one needs to stop

the motion whenever no key on the keyboard is pressed. Otherwise, even though there is

no force on the wheels, the machine will go further. This will continue until its velocity

becomes zero, because the machinery has weight which will slow it down. In our

approach, to simulate the realistic behavior of heavy construction machinery, we force the

movement to stop immediately by setting the velocity to be zero when no forward or

backward key is pressed and by setting the angular velocity to zero when no turning key

is pressed. The detailed code is given below.

// This function allows each of the four wheels to rotate freely // around the X axis in the local model space of the wheels.// By default, the wheels are locked and this functions allows// them to rotate freely. It sets the angular motion in the X// dimension for the configurable joint associated with each // wheel to its free state.function freeWheel(){

frontLeft.GetComponent(ConfigurableJoint).angularXMotion =

ConfigurableJointMotion.Free;rearLeft.GetComponent(ConfigurableJoint).angularXMotion =

ConfigurableJointMotion.Free;frontRight.GetComponent(ConfigurableJoint).angularXMotion

= ConfigurableJointMotion.Free;

rearRight.GetComponent(ConfigurableJoint).angularXMotion =

ConfigurableJointMotion.Free;}

// update the torque on the two driving wheels according to // the current input movement when physics time step is reached. function FixedUpdate(){

const float MAX_TORQUE = 90000;// for forward motion, apply torque in a positive

direction// around the X axis to the driving wheels on both sidesif(Input.GetKey(KeyCode.UpArrow)) {

freeWheel();LW = 1;RW = 1;wheelTorque = MAX_TORQUE;

}// for backwards motion, apply torque in a negative

direction// around the X axis to the driving wheels on both sideselse if(Input.GetKey(KeyCode.DownArrow)) {

freeWheel();LW = -1;RW = -1;wheelTorque = MAX_TORQUE;

}else { //stop motion forward or backwards

rigidbody.velocity = Vector3.zero;frontLeft.rigidbody.angularVelocity = Vector3.zero;rearLeft.rigidbody.angularVelocity = Vector3.zero;frontRight.rigidbody.angularVelocity = Vector3.zero;rearRight.rigidbody.angularVelocity = Vector3.zero;//sthis.rigidbody.angularVelocity.y = 0;LW = 0;RW = 0;wheelTorque = 0;

}

// turn left by applying torque in a positive direction// around the X axis on the driving wheel on the right

side// and in a negative direction on the left sideif(Input.GetKey(KeyCode.LeftArrow)) {

freeWheel();LW = -1;RW = 1;wheelTorque = 90000;

}// turn right by applying torque in a positive direction // on the left side and a negative direction on the rightelse if(Input.GetKey(KeyCode.RightArrow)) {

freeWheel();LW = 1;RW = -1;wheelTorque = 90000;

}else { //stop turning motion

rigidbody.angularVelocity = Vector3.zero;}

// add calculated torque to each of the four wheelsfrontLeft.AddRelativeTorque

(Vector3.forward * LW * wheelTorque);rearLeft.AddRelativeTorque

(Vector3.forward * LW * wheelTorque);frontRight.AddRelativeTorque

(Vector3.forward * RW * wheelTorque);rearRight.AddRelativeTorque

(Vector3.forward * RW * wheelTorque);}

3.3. Deformation of the terrain when the bucket of the excavator hits the ground

Figure 10. Deformation of the terrain (original in color)

Several steps are involved in the deformation of the terrain when the bucket hits the

ground, specifically digging, picking up, holding, pouring, and hitting.

3.3.1. Digging

To avoid the costly collision check between the bucket and terrain, as well as the

resulting bouncing after a collision, our method ignores collisions between the two. Since

deforming the ground will cause the terrain collider to be run again, we also ignore such

collisions in the FixedUpdate function, which executes more often than the Update

function. Alternatively, we could have set collision layers. A collision layer is a

conceptual group of colliders that are labeled with the same tag. Parameters can be set to

control whether collision detection is applied to the colliders in one layer or between

layers.

When the excavator digs into terrain, the terrain is changed according to the shape of

the bucket. For simplicity, the shape of bucket is calculated by selecting several key

positions on its outside surface. The key positions are distributed on the left and right end

of the bucket along different back curves. The seven selected positions along one edge of

the bucket are shown in Figure 11(a); the matching positions on the other side were also

used. The fourteen key points were stored in the KeyPoints array. To find the points for

collision detection, we define a series of collision points between the left and right sides

of the bucket, as shown in Figure 11(b). We linearly interpolate h of these points between

the 3D locations of each pair of points in the KeyPoints array (in positions 2k and 2k+1, 0

≤ k ≤ 6). By using these h points across the outside of the bucket, we ensure that the

collision points are relatively evenly distributed in areas where the bucket collides with

dirt. For example, if h is 8, as shown in Figure 11(b), the total number of collision points

is 7h = 7(8) = 56.

(a) Side view (b) back view

Figure 11. Positions for collision checking on the bucket (original in color)

Dirt particles are generated wherever the bucket penetrates the surface of the terrain.

To see whether this has happened, we check all collision points along the bucket to

determine if the terrain height at the position is higher in the new heightmap than in the

old one at the same coordinates. If so, we store the differences. We use these differences

to calculate the number of particles to be generated by summing up the values at the four

adjacent coordinates of the heightmap. The specific code is given below.

#define MAPGRIDSIZE 256#define MAX_TERRAIN_HEIGHT 64

// dif is a 2D array to contain the maximum decrease in heightfloat[,] dif = new float[td.heightmapHeight, td.heightmapWidth];// low is a 2D array to store the minimal height for updatingfloat[,] low = new float[td.heightmapHeight, td.heightmapWidth];

void dig(){

// initialize 2D arraysfor (int j=0; j < td.heightmapHeight; j++)

for (int i=0; i < td.heightmapWidth; i++){

low[j, i] = MAX_TERRAIN_HEIGHT;dif[j, i] = 0f;

}

float h = 9;

// k is the number of current pair of key points on the bucket

for(int k = 0; k < 7; k++){

for(int q = 1; q <= h - 1; q++){

// Among the key points, number 2k gives the left side

// of the bucket and number 2k + 1 gives the right side

// for the kth pairVector3 left = KeyPoints[2 * k].transform.position;Vector3 right = KeyPoints[2 * k +

1].transform.position;

// interpolate a collision point p between the left// and right key pointsVector3 p = left + q / h * (right – left);

// convert a 3D world position to 2D terrain heightmap

// coordinatesint nx = wtonx(p), ny = wtony(p);

// ignore any point p outside of the terrainif(nx > MAPGRIDSIZE - 1 || ny > MAPGRIDSIZE - 1)

continue;

// calculate the position of the four gridpoints that

// surround point p in the heightmapfor(int j = 0; j <= 1; j++)

for(int i = 0; i <= 1; i++){

// calculate change in height at the current// grid point and if it is lower than those// already examined, it becomes the new lowest // pointfloat change = heightmapData[ny + j, nx + i] *

td.heightmapScale.y - p.y;if(change > dif[ny + j, nx + i])

dif[ny + j, nx + i] = change;if(p.y < low[ny + j, nx + i])

low[ny + j, nx + i] = p.y;}

}}

// for each location in the terrain, generate the appropriate

// number of dirt particles and reduce the terrain heightfor (int j = 0; j < td.heightmapHeight - 1; j++)

{ for (int i = 0; i < td.heightmapWidth - 1; i++){

// the number of dirt particles is calculated // according to the change in height divided by// the height of a cube (b)// ntow(i, j) convert current height in array heightmap// at heightmap coordinates (i, j) to the world positionfor(int k = 1; k <= dif[j, i] / b; k++){

heightmapData [j, i] -= b / td.heightmapScale.y;mylist.Add((GameObject) Instantiate (dust,

ntow(i,j) + Vector3.up * b, Quaternion.identity);

}}

}}

3.2.2. Picking Up

To allow the dirt particles to occupy a finite amount of space in the bucket, they are

represented as cubes, each with an attached particle system. The default cube created in

Unity has a mesh and collider. We delete the mesh, attach a particle system, and set this

cube as a Unity prefab. Since we need the particle system to emit all particles at once, an

idea known as a “one-shot” in the legacy particle system, we select the new shuriken

particle system currently provided by Unity. We set parameters to request no-loop, zero

particles emitted, and bursts at start time to produce the effect. The render method is a

two-color mixed billboard.

The physical materials of the dirt particles are set to zero friction so that there will be

less bouncing. To prevent too much sliding from happening, additional constraints are set

on the velocity of the dirt particles. When the bucket moves up, the velocity of the

particles is set to be downward in each frame. As a result, the dirt stays in the bucket,

instead of being thrown up when the movement of the bucket is slowing down. This

reduces the number of dirt particles tossed from the bucket as they are being picked up.

3.3.3. Holding

To be sure that the soil remains in the bucket when the arm of the excavator is

moving, we employ velocity constraints, as described in the previous section. We set the

parameters of these dirt particles so as to let them behave as solids. While the bucket

holds the particles, we continue to replace the upward velocity of the particles with a

downward one. Hence, the dirt will not easily fall out of the edges of the bucket when

moving in a level fashion.

3.3.4. Pouring

To achieve realistic effects, some parameters of the dirt particles are adjusted before

the particles hit the ground. When particles fall from the edge of the bucket, the speed in

the XOZ plane is set to a smaller value, producing a visual result that seems more like

dirt than ice balls. At the moment that a falling cube hits the ground, the height of its

highest point is at most(√3/2 ) b

, where b is the side length of the cube. To avoid

recalculating the square root, (√3/2 )

is represented as 0.86 in the code. The movement of

the dirt particle is implemented in the FixedUpdate function since a smaller time step is

needed to increase accuracy of the movements. The code follows below.

#define LEVEL_VEL_CONS 1.1f#define DOWNWARD_VEL_CONS -10.0f

void drop(){

// process each dirt particleforeach(GameObject d in mylist.ToList()){

d.rigidbody.velocity = new Vector3(d.rigidbody.velocity.x / LEVEL_VEL_CONS, d.rigidbody.velocity.y, d.rigidbody.velocity.z / LEVEL_VEL_CONS);

// if particle is moving upwards, change it to moving // downwardsif(d.rigidbody.velocity.y > 0)

d.rigidbody.velocity = new Vector3(0, DOWNWARD_VEL_CONS, 0);

// convert the 3D world position of object d to// 2D terrain heightmap coordinatesVector3 p = d.transform.position;int nx = wtonx (p), ny = wtony (p);

// point outside of terrain?

if(nx > MAPGRIDSIZE || ny > MAPGRIDSIZE)continue;

// find the height of the terrain at the neighboring// grid point that has the greatest height and the // height of the lowest part of the bucketfloat max = 0; // maximum heightfloat min = MAX_TERRAIN_HEIGHT; // minimum heightfor(int j = 0; j <= 1; j++)

for(int i = 0; i <= 1; i++){

float tmp = heightmapData[ny + j, nx + i] * td.heightmapScale.y;

if (tmp > max) // highest neighboring pointmax = tmp;

tmp = low[ny + j, nx + i]; // lowest part of bucket

if (tmp < min)min = tmp;

}

// if the dirt particle is sufficiently far under the bucket

// (i.e., it is no longer touching the bucket) and it is// touching the terrain, then it becomes part of the // terrain. The maximum height of a cube with side length// b stood on its corner is sqrt(3)/2 * b.const float SQRT_3_OVER_2 = 0.86;if ((p.y <= min – SQRT_3_OVER_2 * b) &&

(p.y <= max + SQRT_3_OVER_2 * b)){

hit(p);Destroy(d);mylist.Remove(d);

}}

}

3.3.5. Hitting

To simulate the action of fallen soil becoming a part of the ground, we remove the

particles representing this soil from the simulation, as described at the end of the previous

section, and adjust the terrain heightmap at the exact positions of the particles. Here we

describe the details of the hit function that performs the adjustment.

We apply a hit function to every position where a dirt particle lands on the ground.

The hit function deforms the ground slightly for each particle. It implements the approach

described in Section 3.1. The only parameter used is the (x, y, z) position of the hit.

void hit(Vector3 pos){

// compute standard local positions and offsetsVector3 terrainLocalPos = pos - terr.transform.position;float nx = Mathf.InverseLerp( 0, td.size.x,

terrainLocalPos.x ) * (td.heightmapWidth - 1);

float ny = Mathf.InverseLerp( 0, td.size.z, terrainLocalPos.z )

* (td.heightmapHeight - 1);float fx = Mathf.Floor(nx);float fy = Mathf.Floor(ny);float wx = nx - fx;float wy = ny - fy;

// deform each neighboring grid point in the heightmapfor(int j = 0; j <= 1; j++)

for(int i = 0; i <= 1; i++){

// compute increased height resulting from impactfloat addHeight = (float)((1 - i) + (2 * i - 1) *

wx) * ((1 - j) + (2 * j - 1) * wy) * b;

float oldHeight = heightmapData [(int)fy+j, (int)fx+i] * td.heightmapScale.y;

// add increased height to current pointfloat[,] newHeightData =

new float[1, 1] { { oldHeight + addHeight } };

CreatePile((int)fx+i , (int)fy+j, newHeightData[0, 0]);}

}

4. Implementation Details

Three aspects of the implementation deserve additional commentary, in particular

heightmap resolution of the terrain, rigging joints, particle soil and volume conversion.

Each of these aspects is now discussed.

4.1. Heightmap resolution of the terrain

The original ground surface, and any piles added to it, is implemented using Unity’s

built-in terrain. The terrain data structure uses a heightmap to hold all of the information

about the shape of this surface as it is deformed. The height at a particular location can be

changed using the SetHeight function, which causes the physics engine not only to update

the heightmap, but also to recalculate the entire collider for the terrain. Terrain data

controlled by the heightmap script and terrain collider are all components of a terrain.

Thus, this system is resource-intensive. This recalculation can consume so much CPU

time that the simulation runs slower than intended. Two solutions were evaluated. One is

to use this function less often, and the other is to use a low resolution heightmap, which

restricts the size of a realistic-looking terrain.

In the first solution, called the Secondary Array approach, a secondary 2D array is

used to store a complete version of the Unity heightmap. Updates to the heights stored in

this secondary array are made at any time between when frames are rendered. Such

updates can be made at a lower cost than updating the primary heightmap. Immediately

before rendering any frame, we copy the data from the secondary array to the primary

heightmap. At the same time, the collider is calculated once for the entire terrain. Display

is done by constructing a geometric model for the surface from the primary heightmap.

With this approach, the terrain collider and all collision checking between objects and the

ground is done by setting different parts of the machinery in different collision layers.

This approach separates the update cycle for the heightmap from that of the display.

In the second solution, called the Low Resolution approach, the number of

calculations is bounded by a fixed limit, regardless of the size of the scene. This is done

by establishing a fixed, relatively small resolution for the heightmap. For example,

suppose that the resolution of the heightmap is set to 257. Then, regardless of the size of

the simulated scene (in meters), it is represented by a 257 x 257 heightmap. The

advantage of doing this is that the processing time is fast in all cases. This technique

suffers, however, when the terrain is very large. Specifically, the ground appears very

irregular, since the simple points are distributed too widely across the surface.

Figure 12(a) shows the complete terrain as represented by a tree-structured

heightmap. Figure 12(b) demonstrates the change in detail that occurs in the heightmap

during a deformation.

(a) Scene with wireframe and texture

(b) Deformation when digging (wireframe view)

Figure 12. Unity terrain heightmap

4.2. Rigging joints on the excavator

As mentioned in Section 3.2, rigging refers to the organization of the joint hierarchy

of a model. We wish to rig the excavator, in Unity, such that it is as steady as possible,

even though it has a long arm involving several joints. It is important that the arm not

bounce a great deal. Thus, we added a mesh collider to each part connected to a joint to

help stabilize the joint. Moreover, the weight of the two parts connected at a joint should

be roughly equal. If one part is much lighter than the other, then it can be easily moved,

which causes the configuration of the arms to change easily. The weights are,

accordingly, set to 10 units for the main body, 2 units for the upper and lower arms, and 1

unit for the bucket. Despite these settings, when we used the default settings for the time

step (0.02s) and solver iteration count (6) in the Unity physics engine, the arm bounced in

an unacceptable fashion whenever the main part of the excavator moved Two means of

increasing the frequency of calculation are to reduce the time step or to increase the

number of iterations that the solver runs during each cycle. Neither technique is really

better than the other. Both of these changes will cause the physics calculator to consume

more system resources. We choose to set the physics timestep to 0.005 (default is 0.02),

while leaving the iteration count fixed at 6 (default).

(a) Right-side view

(b) Underground view

Figure 13. Excavator rigging (original in color)

Figure 13 illustrates the colliders built for the excavator. In Figure 13(a), one can see

all colliders set for the excavator in green. Each part of the excavator has a separate

collider. The four pans located on the two sides of the two tracks keep the other wheels

moving forward and backward inside of the tracks as the excavator is turned by its

driving wheels. In Figure 13(b), one can see the colliders in a view from underground. In

this image, the axes of the joints are indicated with yellow arrows.

4.3. Particle soil

Two possible solutions to represent the soil dug up by the excavator were considered.

The first method is to simulate all pieces of the dirt in the bucket by separate particles,

whereas the second is to simulate only the surface of the soil in the bucket as particles,

and to treat the remainder as a heightmap. We selected the first method.

In the first method, the number of particles in the bucket of the excavator is large. To

allow for collision checks among these particles, we used a number of cubes, each with a

box collider, instead of a real particle system. Using a Unity particle system is not

effective because Unity does not check for collisions between such particles. This is not

suitable when implementing a large volume of dirt, such as that in the bucket of the

excavator. We did not use sphere-shaped colliders because they are not easy to dig up and

they slide away easily. As well, the physical behaviors of soil particles resemble that of a

box collider more than that of a sphere collider. To model the cohesiveness of the soil

particles, we analyzed the forces between the cubes. The calculations required for

collision detection between these cubes gave us the forces inside of the soil.

Unfortunately, this method could not be computed fast enough to be used in each frame.

Instead, we used velocity constraints to limit the movement of the cubes. When a cube

has an upward velocity, we replaced it with a downward velocity in every physics update,

thereby gradually reducing the upward movement. When a cube has an additional

downward acceleration, it will force the movement of soil faster when going within the

bucket, which will simulate the stickiness of the soil and bucket well. This approach

prevents cubes from falling out of the bucket when the excavator arm is turning or lifting.

This method gives more realistic results, but, at the same time, is slow, since the number

of cubes required is more than the Unity physics engine can handle.

In the second method, a heightmap is used to simulate the surface of the soil in the

bucket. Since the bucket can turn at any angle, it is difficult to recalculate the heightmap.

As the bucket rotates in 3D, the heightmap is recalculated in each frame. Since no

method of conveniently updating the heightmap from frame to frame was found, the

method was deemed to be infeasible and was not implemented.

(a) Excavator with dirt in bucket

(b) Particles in bucket (c) Collision models for particles

Figure 14. Dirt particles (original in color)

Figure 14 shows the effects when some soil, in particle form, is held by the bucket of

the excavator. Figure 14(a) shows an overall view of the excavator with the soil, which

illustrates that it looks natural from a distance. Figure 14(b) shows a close up of the

particles in the bucket, which illustrates that the particles look a little cloudy on close

inspection. Finally, Figure 14(c) shows the collision models for the particles. Although

the particles appear spherical, their collision models are in fact cubes.

4.4. Volume conversion

When the dirt is dug up, a unit of dirt is converted from terrain to dirt particles. Since

the change of terrain is calculated by changing the heightmap, we can get the total

changed volume by accumulating all the changes from every 2D-coordinate of the

heightmap. For each change in a coordinate, we obtain the volume change by considering

the nearest six coordinates as the terrain heightmap using a tree structure to represent the

change.

Figure 15. Heightmap structure of terrain

Figure 15 shows the wireframe view of a tree structured heightmap of terrain after

changes by the excavator. One typical quadrilateral on the terrain surface is indicated in

the figure. The grid size for the 2D heightmap is l.

The volume change at each coordinate (x, y) actually consists of six tetrahedrons

shared with the same height h(x,y), one for each two neighboring coordinates. The formula

for the change is as follows:

V(x,y) = 6×[ 1

3×Δh(x,y )×( 1

2×l2) ]

= Δh( x , y )×l2

,

where

l =

Terrain sizeHeightmsp resolution

From Figure 16(a), we can see the six tetrahedrons generated by increasing the

height of the center coordinate on a flat surface. The tetrahedrons have three side faces.

They also have triangle bases, as shown in Figure 16(b), the area of which is l2/2, as

shown in Figure 16(c). Six similar tetrahedrons are generated when we decrease the

height of the center coordinate. If the surface is not flat, we consider six pairs of

tetrahedrons, representing the old and new surfaces. Overall, the result of changing a

height always involves calculations with the same shape and formula.

As the size of the particles being generated is based on the same length l, the

converted volume is only affected by . So at runtime, we can calculate the number of

cubes to be generated by considering only the changes in the height values.

(a) 3D view of six tetrahedrons

(b) base view of six tetrahedrons (c) area of base of one

Figure 16. Nearest six tetrahedrons

When a dirt cube drops on the ground, we distribute the height of dirt to the nearest

four heightmap coordinates by parameters which add up to equal the dirt height. Thus,

the total height and the total volume are not changed by a completed movement of some

dirt. While some dirt is in the excavator, the total height of the terrain is reduced, but

when the dirt is dropped on the ground, it is restored to its initial value. By examining the

total height of terrain heightmap over time, one can examine the accuracy of the

implementation. Figure 17 shows one result after a terrain change. The total change to

height, displayed in the upper left corner, is zero.

Figure 17. Total heightmap height change is 0

5. Results

The results of running the implementation of the soil and construction machinery

animation system are described in this chapter. In Section 5.1, we describe the effects

achieved with respect to the soil, the model, and their interaction. In Section 5.2, we

describe a quantitative evaluation of the approach with respect to the frame rate, while

varying the heightmap resolution of the terrain, the number of particles, and the number

of models. Each dirt particle in all tests is based on a cube collider with an internal

particle system.

In all experiments, we used the Low Resolution approach to model the terrain. We

set the size of the terrain to be 102.4 x 102.4 and unless otherwise stated we used a

heightmap resolution of 257. We set the side length of each dirt cube to be 0.4 (equal to

102.4/256). As the number of particles is increased in the experiments, we decreased the

size of the particles (cubes) to ensure that the visual appearance of the cloud of particles

is appropriate. This decrease in size has no significant effect on speed.

Experiments were carried out on a PC with the following hardware and software:

Processor: Intel(R) Core(TM) i7-3770

Memory: 12288MB

Graphics card: NVIDIA GeForce GTX 660

Software: Unity 4.3.4

Operating System: Windows 8 Pro 64-bit (6.2, Build 9200)

5.1. Effects achieved

The images in Figures 18 show the model of the excavator interacting with the

ground under different types of deformation, such as digging, picking up, holding, and

pouring the soil. Figure 19 illustrates a well-formed pile of soil, produced by digging a

hole in the terrain. Figures 20 and 21 show different views of the movements of the

excavator and the resulting dirt pile.

(a) Digging

(b) Picking up dirt

Figure 18. Excavator digging a hole in the ground (original in color)

(c) Holding

(d) Pouring and hitting the ground

Figure 18 (continued). Excavator digging a hole in the ground (original in color)

(e) Soil poured back into a hole

Figure 18 (continued). Excavator digging a hole in the ground (original in color)

Figure 19. A well-formed pile of soil (original in color)

Figure 20. Excavator forming a soil pile from dirt (original in color)

Figure 21. Movement of excavator model (original in color)

Figure 22 shows a model of a road header, a machine used in coal mining. It is

rigged and can turn its arm and spin its heads. This model has a relatively low polygon

count (11,377) and contains no textures.

Figure 22. Road header (original in color)

5.2. Evaluation of the system

In this section, we evaluate the performance of our implemented approach while

varying the heightmap resolution, the number of particles, and the number of models.

5.2.1. Varying the heightmap resolution for the terrain

Although the terrain size determines the dimensions of the simulated world, it is the

heightmap resolution that controls the number of values related to height that are stored

and manipulated. Thus, this resolution has a strong influence on the overall frame rate of

the animation.

Table 2. Heightmap resolution vs. approximate frame rate

Heightmap resolution 129 257 513

Approximate frame rate (fps) 75 60 15

Figure 23. Different heightmap resolutions

Table 2 shows the approximate frame rates obtained under different heightmap

resolutions. The system was tested with one excavator and without any particles. We

varied the heightmap resolution from 129 to 513, as shown in Figure 23, and observed

the changes in the frame rate. For example, a resolution of 129 means that the heightmap

is composed of 129 x 129 grid points. From Table 2, we can see that as the heightmap

resolution increases, the frame rate sharply decreases. Figure 24 shows a case where the

heightmap resolution is 257 and the frame rate is approximately 60 fps.

Figure 24. With a heightmap resolution of 257, the frame rate was around 6 fps

5.2.2. Varying the number of dirt particles

The frame rate depends on the number of dirt particles. Since a cube representation

of a dirt particle is implemented as an individual Unity object, each cube requires a

collision check every physical update. Making numerous collision checks reduces the

frame rate, even though the checks are rather simple.

Figure 25. Particle count vs. approximate frame rate

Table 3. Particle count vs. approximate frame rate

Particle Count 25 50 75 100 125 150 175 200

Approximate Frame Rate (fps)

35 25 20 15 11 9 8 7

The system was tested with one excavator model and a variable numbers of particles.

The numbers of the particles increases when the excavator digs into the ground. We

varied the number of particles from 25 to 150 and recorded the frame rate. From the

results given in Table 3 and Figure 25, one can see that as the number of the particles

increases, the frame rate decreases. Figure 26 illustrates a case where the number of dirt

particles is 50 and the resulting frame rate is approximately 25 fps.

Figure 26. Frame rate is around 25 fps, when holding a full bucket of dirt

5.2.3. Varying the number of models

The frame rate also depends on the number and complexity of the models. We

evaluated the system with varying numbers of two models. The system was tested using

different numbers of models without any particles. We varied the numbers of models

from 1 to 4 and observed the frame rate.

When using a road header, the initial frame rate is approximately 90 fps. As shown in

Table 4, increasing the number of road headers has little effect on performance. If four of

them are placed in a scene, the frame rate is reduced to approximately 85 fps.

As seen in Table 5, increasing the number of excavators reduces the frame rate more

substantially. The excavator not only has many joints, but it also has two physical tracks.

Table 4. Number of road header models vs. approximate frame rate

Numbers of models 1 2 3 4

Approximate frame rate (fps) 90 88 87 85

Table 5. Number of excavator models vs. approximate frame rate

Numbers of models 1 2 3 4

Approximate frame rate (fps) 60 40 30 20

The visual appearances of varying numbers of road headers and excavators are

illustrated in Figures 27 and 28 respectively. From the results, we can see that frame rate

goes down as the number of models is increased. The reduction in the frame rate is larger

for the excavator model because it has more parts and rigging information than the road

header.

(a) One road header

(b) Four road headers

Figure 27. Varying the number of road headers

(a) Two excavators

(b) Four excavators

Figure 28. Varying the number of excavators

6. Conclusions and Future Work

In this project, we generated well-formed soil piles and added them to the terrain,

animated construction machines by rigging their joints, and simulated the interaction

between an excavator and the terrain. We dynamically created and destroyed dirt particles

while ensuring that mass and volume were conserved during conversions between

particles and terrain.

The major challenges overcome in this project were (1) implementing soil in a

commercial game engine, (2) gradually generating a pile using slippage, (3) devising

fully functional, physics-based tracks for the excavator, and (4) converting volumes

between a 2d heightmap and 3d particles.

With our approach, soil dropped in the same place accumulates as a cone. In the

future, we could apply noise to the shape, which would add a bit of irregularity to the

shape. Alternatively, we could also let the angle vary a bit or put noise in the surface

normal.

The soil in the bucket was implemented by treating small cubes with box colliders as

particles. In the future, one could design a more efficient model that might allow the

process to be done more smoothly. For example, one could treat the lower part of the soil

in the bucket as a single object and replace the dirt particles there with a mesh edited to

fit into the bucket. Another possibility is to replace the built-in terrain in the Unity game

engine by voxel terrain so that the construction machinery is able to dig a cave.

The soil density and wetness could also be considered in the future. For example, the

density of the soil could be reduced after digging, with a corresponding increase in soil

volume. To allow for different densities in the physics calculates, we could vary the mass

values for the soil particles. We can simulate wetness of soil by changing its critical

angle, its texture, and its stickiness values.

The rendering of the soil in the air can be improved. For example, we could render

the particles that are falling using a variety of sizes and textures.

References

[1] M. Zyda. From visual simulation to virtual reality to games. Computer, 2005, 38(9):

25-32.

[2] T. A. Galyean and J. F. Hughes. An interactive volumetric modeling technique.

Proceedings of the ACM SIGGRAPH Computer Graphics, 1991: 267-274.

[3] Y. Oda, K. Muraoka, and N. Chiba. Particle-based visual simulation of virtual clay.

Information Processing Society of Japan, 2001, 42(5): 1142-1150.

[4] M. Matsumiya, H. Takemura, and N. Yokoya. An immersive modeling system for 3D

free-form design using implicit surfaces. Proceedings of the ACM symposium on

virtual reality software and technology, New York, 2000: 67-74.

[5] H. M. Jaeger, S. R. Nagel, and R. P. Behringer. Granular solids, liquids, and gases.

Reviews of Modern Physics, 1996, 68(4): 1259-1273.

[6] P. Richard, M. Nicodemi, R. Delannay, P. Ribière, and D. Bideau. Slow relaxation and

compaction of granular systems. Nature Materials, 2005, 4 (2): 121-128.

[7] X. Li and J. M. Moshell. Modeling soil: Realtime dynamic models for soil slippage

and manipulation. ACM Computer Graphics (SIGGRAPH ’93 Proceedings), 1993,

361-368.

[8] B. Chanclou, A. Luciani, and A. Habibi. Physical models of loose soils dynamically

marked by a moving object. Computer Animation ’96, 1996: 27-35.

[9] R. W. Sumner, J. F. O’Brien, and J. K. Hodgins. Animating sand, mud, and snow.

Computer Graphics Forum, 1999, 18(1): 3-15.

[10] Y. L. Zeng, C. I. Tan, W. K. Tai, M. T. Yang, C. C. Chiang, and C. C.Chang. A

Momentum Based Deformation System for Granular Material. Computer Animation

and Virtual Worlds - CASA 2007, 2007, 18(4-5): 289-300.

[11] T. Nishita, H. Iwasaki, Y. Dobashi, and E. Nakamae. A modeling and rendering

method for snow by using metaballs. Computer Graphics Forum, 1997, 16(3): 357-

364.

[12] P. Fearing. Computer modelling of fallen snow. ACM Computer Graphics

(SIGGRAPH ’00 Proceedings), 2000: 37-46.

[13] R. A. Gingold and J. J. Monaghan. Smoothed particle hydrodynamics: Theory and

application to non-spherical stars. Monthly Notices of the Royal Astronomical

Society, 1977, 181: 375-389.

[14] M. Müller, D. Charypar, M. Gross. Particle-based fluid simulation for interactive

applications. Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium

on Computer Animation, San Diego: Eurographics Association, 2003: 154-159.

[15] M. Müller, R. Keiser, and A. Nealen. Point based animation of elastic, plastic and

melting objects. Proceedings of the 2004 ACM SIGGRAPH/Eurographics

Symposium on Computer Animation, Aire-la-Ville: Eurographics Association, 2004:

141-151.

[16] K. Hegeman, N.A. Carr, and G.S.P. Miller. Particle-based fluid simulation on the

GPU. Computational Science–ICCS 2006, 2006, 3994: 228-235.

[17] W. Rungjiratananon, Z. Szego, Y. Kanamori, and T. Nishita. Real-time animation of

sand-water interaction. Computer Graphics Forum, 2008, 27(7): 1887-1893.

[18] P.A. Cundall, and O.D.L. Strack. A discrete numerical model for granular

assemblies. Geotechnique, 1979, 29: 47-65.

[19] T. Lenaerts, and P. Dutré. Mixing fluids and granular materials. Computer Graphics

Forum, 2009, 28: 213-218.

[20] M. Ihmsen, A. Wahl, and M. Teschner. High-resolution simulation of granular

material with SPH. Proceedings of the VRIPHYS, 2012, 53-60.

[21] M. Ihmsen, A. Wahl, and M. Teschner. A Lagrangian framework for simulating

granular material with high detail. Computers & Graphics, 2013, 37(7): 800-808.

[22] I. Alduán, and M. A. Otaduy. SPH granular flow with friction and cohesion.

Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer

Animation, 2011, 25-32.

[23] K. Onoue, and T. Nishita. Virtual sandbox. 11th Pacific Conference on Computer

Graphics and Applications, Washington: IEEE Computer Society, 2003, 252-262.

[24] J. P. Bouchaud, M. Cates, J. Ravi Prakash, and S. Edwards. A model for the

dynamics of sandpile surfaces. Journal de Physique I, 1994, 4(10): 1383-1410.

[25] P. A. Langston, U. Tüzün, and D. M. Heyes. Discrete element simulation of granular

flow in 2D and 3D hoppers: Dependence of discharge rate and wall stress on particle

interactions. Chemical Engineering Science, 1995, 50(6): 967-987.

[26] N. Bell, Y. Yu, and P. J. Mucha. Particle-based simulation of granular materials, SCA

'05 Proceedings of the 2005 ACM SIGGRAPH/Eurographics Symposium on

Computer animation. New York, 2005, 77-86.

[27] M. Pla-Castells, I. García-Fernández, and R. J. Martinez-Dura. Physically-based

interactive sand simulation. Eurographics 2008, Crete: Eurographics Association,

2008, 21-24.

[28] B. Zhu, and X. B. Yang. Animating sand as a surface flow. Eurographics 2010.

Norrkoping: Eurographics Association, 2010, 1-4.

[29] Gauss circle problem, http://en.wikipedia.org/wiki/Gauss_circle_problem#cite_ref-2

[30] TurboSquid, Inc., http://www.turbosquid.com

[31] Mohr–Coulomb theory, http://en.wikipedia.org/wiki/Mohr–Coulomb_theory

[32] Heightmap, http://en.wikipedia.org/wiki/Heightmap

Appendix A

A.1. Switchable camera

An option is available to the user to switch the camera between an outside viewpoint

and an inside one. We can set two cameras using different scripts. When the user switches

the camera, we can change the active state of them, or edit the depth of the cameras. The

outside viewpoint is controlled by a built-in script called “Mouse Orbit”, which requires

us to set the target as the construction machinery and the distance from the camera to the

target. The inside look is controlled by a built-in script called “Mouse Look”, which lets

the viewer see in all directions from inside of the construction machinery. We placed the

camera inside the driver’s compartment. Since the current models of construction

machinery do not have the driver’s compartment modelled, the user sees only a basic

room frame when driving.

Figure 29. View from inside the excavator (original in color)

A.2. Machinery smoke

When the construction machinery is on, there is smoke emitted from the exhaust

pipe. To accomplish this, we use a particle system attached to the end of the exhaust pipe,

with a cone-shaped area and upward forces. The size of each particle becomes bigger

over time, and has a random-length lifetime so that it will naturally scatter and disappear

into the air.

Figure 30. Smoke from the excavator (original in color)