introduction to haptic rendering ming c. lin [email protected]

Introduction to Haptic Rendering

Ming C. Lin

[email protected]

http://gamma.cs.unc.edu/interactive

UNC Chapel Hill M. C. Lin

What Is Haptic Rendering?

Simulation

HapticDevice

Robot

Virtual Reality

Master-Slave Systems

Human

Force Feedback

Tactile Feedback

Human-in-the-Loop

Pictures: http://haptic.mech.nwu.edu/intro/gallery/


Why Haptics?

Natural 3D interaction with a simulated environment. Not constrained by a 2D projection.

Additional sensorial input. Visual is dominant. Exploit extra dimensions, ‘cheat the user through the eyes’… In case of inconsistencies, visual wins.


Inter-disciplinary Research

ComputerScience

ElectricalEngineering

MechanicalEngineering

Haptic Rendering Control and actuators

Mechanical design

Computation of the forces output by the device


To be covered

Haptic rendering for pen-type devices:

• 3DOF Haptics(3D force output)

• 6DOF Haptics (3D force + torque output)

Point-Object interaction

Object-Object interaction

DOF = Degree of Freedom


Some History

Argonne ’54, first master-slave systems. Master=slave. Salisbury ’80, independent master and slave. Cartesian control of robot arm. Later, interaction with computer simulated slave (VR). GROPE project UNC ’67-’90, molecular docking. Minsky ’90, the Sandpaper. Massie & Salisbury ’94, the Phantom. Early 90’s to ’97, 3DOF haptics. Late 90’s to today, 6DOF haptics.


Control of Haptic Devices

• Impedance devices • Admittance devices

User

Actuators

Simulation User

Actuators

SimulationF

x*x

x, v

F*F

• Collision detection + contact response model

• More popular

• Collision detection + position constraints

• Bulkier and more expensive• Very good for virtual walls


Control of Haptic Devices

• Impedance Devices • Admittance devices

6-DOF Phantom COBOTs


1KHz Performance Requirement

The user becomes part of the simulation loop. 1KHz is necessary so that the whole system

doesn’t suffer from disturbing oscillations.

Think of the analogy with numerical integration of a system with spring, mass and damper, where the frequency of the haptic loop sets the integration step.

The Phantom haptic devices run their control loop at 1KHz.

Consequence: we are very limited on the amount of computation that we can do.


3DOF Haptics: Intro

Output: 3D force 3DOF hapticsLimited to applications where point-

object interaction is enough.– Haptic visualization of data– Painting and sculpting– Some medical applications

Object-object

Point-object


3DOF Haptics: Basic approach

Check if point penetrates an object.Find closest point on the surface.Penalty-based force.

xFxkF


3DOF Haptics: The problems

Force discontinuities when crossing boundaries of internal Voronoi cells.

F1

F2

Unexpected force discontinuities (both in

magnitude and direction) are very disturbing!


3DOF Haptics: The problems

Pop-through thin objects.

After the mid line is crossed, the force helps

popping through.

motion


3DOF Haptics: Point-to-plane

Mark et al., 1996. For distributed applications. The simulator sends the

equation of a plane to the haptic loop. The update of the plane is asynchronous.

Forces are computed between the haptic point and the plane. Possible stiffness is limited, because too stiff would bring a

jerky behavior at low update rates.


3DOF Haptics: God-object

Zilles and Salisbury, 1995.Use the position of the haptic interface point

(HIP) and a set of local constraint surfaces to compute the position of god-object (GO).

Constraint surfaces defined using heuristics.

Compute GO as the point that minimizes the distance from HIP to the constraint surfaces. Lagrange multipliers.



Constraint surfaces:– Surfaces impeding motion– GO is outside (orientation test) and in

the extension of the surface.– The HIP is inside the surface.

motion

edge/vertexConcavity: 2/3

constraint planes in 3D

motion



Constraint plane equations:

Energy function that will account for the distance.

Define cost function using Lagrange multipliers.

Minimize, solving for x, y, z, 1, 2 and 3.

0 iiii DzCyBxA

222

2

1HIPHIPHIP zzyyxxkE

3 planes at most

3

1

222

2

1

iiiiii

HIPHIPHIP

DzCyBxA

zzyyxxkC



Partial derivatives on x, y, z, 1, 2 and 3 yield 6 linear equations:

In case of less than 3 planes, the problem has a lower dimension.

3

2

1

3

2

1

333

222

111

321

321

321

000

000

000

100

010

001

D

D

D

z

y

x

z

y

x

CBA

CBA

CBA

CCC

BBB

AAA

HIP

HIP

HIP


3DOF Haptics: Virtual proxy

Ruspini et al., 1997. Based on god-object. Virtual proxy is a small sphere, instead of a

point. Use configuration-space obstacles (C-obstacles), from robotics.

More formal definition of constraint planes. Implementation of additional features,

based on relocation of the virtual proxy.



C-obstacles: for a spherical object, is reduced to computing offset surfaces at a distance equal to the radius of the sphere.

Check the HIP against the offset surface. This is done to avoid problems with small

gaps in the mesh.



Finding the virtual proxy is based on an iterative search.

Basically, find subgoals based on the same distance minimization as for the god-object.

At each subgoal, all the planes that go through that point are potential constraints. The minimum set of active constraints is selected.

If the subgoal is in free space, set as new subgoal the HIP. The path might intersect the C-obstacles. Add the first plane intersected as a constraint and the intersection point as the current subgoal.

The process ends when the virtual proxy becomes stable.



HIP(t)

Perform collision detection between the path of the HIP

and the C-obstacles

HIP(t+t)



HIP(t)

Set the subgoal and the constraint plane(s)

HIP(t+t)



HIP(t)

HIP(t+t)

Find a new subgoal using the active planes and the minimization based on Lagrange

multipliers



HIP(t) Since the subgoal is in free space, drop the

constraints, set the HIP as the new subgoal and

perform collision detection between the

path and the C-obstaclesHIP(t+t)



HIP(t)

Recompute subgoal with new constraints

HIP(t+t)



HIP(t)The path to the new subgoal intersects another plane, so

this is added to the set of constraints

HIP(t+t)



HIP(t) Compute active constraints (in 2D

there are only 2) and find subgoal

HIP(t+t)

For this example, this is the final position of the virtual proxy



Quadratic programming approach:– The constraint planes define an open convex

region (bounded by the plane at infinity).– The function to minimize is the distance from

the haptic device (HIP) to the new subgoal (VPi+1):

– The translation from the current location to the new subgoal cannot intersect the constraint planes. Define linear constraints based on the normals of the planes.

Quadratic function)(1 ttHIPVPC i

01 iii VPVPN Linear constraints



Special case: simplified approach.– Let’s look at the convex region defined

by the constraint planes

Plane at

Plane at Plane at

Plane at

Vertex EdgeFace (all surrounded

by the plane at )



– Normals (n) become points (-n) in a dual space (Gauss map).

– The plane at infinity is the origin (o).– Points (-n) are joint together if the associated

planes intersect.– The position of the virtual proxy also has a dual:

– The vertices of the closest feature to P are the duals of the active constraint planes that are used for the minimization with Lagrange multipliers.

i

i

VPHIP

VPHIPP



Force output: PD (proportional- derivative) control. Produces a force that will try to keep the VP and the HIP at the same position.

User

SimulationVP

PD control ActuatorsF* F

HIP

dt

HIPVPdKHIPVPKF dp

)(*

It’s like a spring+damper, but the authors look at it from a

control engineering approach


3DOF Haptics: Additional features

Force shading by Basdogan et al., 1997.– Interpolate per-vertex normals using

baricentric coordinates (same as Gouraud shading)

– Effect: edges and vertices seem ‘rounded’

A1A2

A3 v2

v1

v3

'

''

3

1

3

1

N

NN

A

NA

N

ii

iii



Force shading by Ruspini et al., 1997.– Modify the position of the VP.– A subgoal (HIP’) is computed changing

the normal of the plane.– This subgoal replaces the HIP, and the

final VP is computed.

HIP

HIP’

VPOriginal

plane

Force-shaded plane



Other effects by Ruspini et al., 1997.– Friction.– Textures.– Deformation.

All of them are achieved performing operations that modify the position of the VP.



Friction by Hayward and Armstrong, 2000.– Model friction as an elasticity between the

actual contact point (x) and a fictitious point, called adhesion point (w).

– The adhesion point remains static or follows the actual contact depending on the ‘state’ (sliding, static…) xw


3DOF Haptics: H-Collide

What about the collision detection?– Spatial uniform decomposition of space. Locate

the path traversed by the probe in that grid, using a hash table.

– Test the path against an OBBTree. Optimized test.– Use frame-to-frame coherence, caching the

previous contact, and perform incremental computation.

– When the leaf intersects, compute the surface contact point (SCP). Pass this to GHOST (haptic rendering library used by the Phantoms).


3DOF Haptics: H-Collide

Spatial partitioning with hash table

Ray Vs. OBBTree test

SCP computation

SCP


6DOF Haptics: Intro

Output: 3D force + 3D torque For applications related to manipulation.

– Assembly and maintenance oriented design. Removal of parts from complex structures.

Typical problem: peg-in-the-hole.

There is a net torque


6DOF Haptics: Intro

Formulate the problem as a rigid body simulation problem and output to the user the forces exerted on the object. Why not?– Too expensive

– Constantly facing a situation of resting contacts.

– Too much time performing collision detection to find non-penetrating situations.

– Solve the linear complementarity problem.

– Theoretically, it seems like the natural extension of the virtual proxy, though.


6DOF Haptics: Intro

The position of the center of mass (com) and the orientation cannot be

governed directly by the haptic device.Interpenetrations would be unavoidable.

com

Simulated object (xs, qs)

com

Haptic device (xh, qh)

com


6DOF Haptics: Contacts

We cannot afford to compute ‘exact’ contacts. Allow tolerances. Consider as valid contact

whatever is inside a distance tolerance. Outward offset surfaces are better. Distance is

cheaper to compute than penetration.

Contact volume Contact area Contact points (e.g. local minima of distance function)


6DOF Haptics: Major approaches

All the approaches suffer from the fact that it’s an output-dependant problem.– Point cloud Vs. voxelized model. It can be

linear on the number of points. The number of points grows quadratically with the resolution.

– Polygonal models. At least linear on the number of convex pairs inside the tolerance.

– NURBS. Very complex exact distance computation between NURBS surfaces.


6DOF Haptics: Voxel sampling

McNeely et al., 1999. Voxelize all the models in the scene. For the haptic object, represent each voxel

by a point Point Shell. Test points against voxels of the scene. Compute force at each point that penetrates

an object. Add up all the force and torque, and apply

them to the user through virtual coupling.



All the objects are voxelized. For the haptic object, store points at the center of

the voxels, with inward normals determined by the polygonal model.

For the objects in the scene, create the voxmap, a 512-tree with 3 levels (the 512-tree is an extension of the octree, with 512 children per node, cube of 8*8*8). The 512-tree optimizes memory.

The initial implementation is only for static objects.



Original object Voxels Pointshell



All the voxels in the voxmap store info about their position in the object: interior, surface, proximity or free space.

interior

surface

proximity

free space



To start the collision query, all the pointshell has to be transformed to the local coordinates of the voxelized objects.

Points that are in the surface or proximity voxels are considered as contacts.



Forces at each contact are computed according to the tangent-plane force model.

Contacts are valid if d<0 (penetration). If penetration to the interior happens, the contact

is missed. One might think of computing a distance field in the interior, but the distance depends on the normal N.

d

Take the plane with normal N that goes through the center of the

voxel. Compute d as the distance from the point to that plane.

N



The force is continuous in direction, but discontinuous in magnitude.

The virtual coupling (see later) is supposed to filter these discontinuities.

d2

d1d2The contact might even show

up or disappear suddenly

Nd1

motion



Force per contact: Add all the forces together.Several contacts act as a single contact with

higher stiffness.Limit the total stiffness:

There can still be problems of irregular distribution of stiffness.

iii NdkF

n

iiFF

1

n

iiF

nF

1

10If n<10 If n>10



Irregular distribution of stiffness.

Geometry-driven instabilities

F2=F1 d2=5*d1

Risk of penetration!F1=5*k*d1 F2=k*d2

1 contact

5 contacts

Object bounces from one side to the other.

More common due to torque on long objects.



Penetration has to be avoided.Pre-contact braking forces. Strong

viscous force applied in the proximity layer.

The viscosity is adjusted so that all the kinetic energy is dissipated in a distance equal to the width of the layer.



Virtual coupling, see Adams and Hannaford, 1998.

Spring-damper between the position and orientation of the haptic device and the position and orientation of the object.

Filters discontinuities, to enhance stability. We can run into numerical integration problems, though.


6DOF Haptics: Polygonal models

Gregory et al., 2000; Kim et al., 2002. Convex decomposition of polygonal

models. Collision detection (modified SWIFT++),

returning all pairs of convex pieces within a tolerance. Handle shallow penetrations.

Cluster contacts. Combination of penalty-based restoring

forces and impulsive response.



SWIFT++, by Ehmann and Lin, 2001. Hierarchy of convex pieces. Voronoi marching between pairs of

pieces. Travel down if the distance is less than the tolerance.



Modifications to SWIFT++:– Return all pairs of pieces inside the

tolerance (group that contains the set of local minima of the distance function).

– Handle shallow penetrations, computing penetration depth between convex pieces. Check for pairs of features with overlapping normals.



Convex decomposition:– Hard problem. The decomposition might not

be efficient. Explosion of pairs O(n4).

– Surface decomposition Convex patches, completed with virtual faces. Linear size.

Solid decomposition Surface decomposition



Surface decomposition good for finding global minimum in disjoint situation. Unique pair of features, both original.

Problems for local minima and penetration. The features might be virtual.

If a single face is a convex piece itself, penetrating contacts might be missed.



Cluster contacts. Find unique contact (distance, normal and point),

averaging contacts in a neighborhood. Weighted average, using distance as the weight. This solves the problem of the irregular

distribution of stiffness. Pretty smooth varying forces in practice, without

need of virtual coupling. Additional features (texture, friction…) can be

rendered in a per-contact basis.


6DOF Haptics: Unsolved problems

Generalization of Virtual Proxy to 6-DOF haptics

Haptic Texture SynthesisApplications

– Endoscopic sinus surgery– 3D Painting with deformable brushes– 3D deformable objects


References

R. J. Adams and B. Hannaford. A Two-port Framework for the Design of Unconditionally Stable Haptic Interfaces. In Proc. of IEEE/RSJ Int. Conference on Intelligent Robots and Systems, 1998.

C. Basdogan, C. H. Ho and M. Srinivasan. A Ray-based Haptic Rendering Technique for Displaying Shape and Texture of 3D Objects in Virtual Environments. In the Proc. of ASME Dynamic Systems and Control Division, 1997.

G. Burdea. Force and Touch Feedback for Virtual Reality. John Wiley and Sons, 1996.

S. A. Ehmann and M. C. Lin. Accurate and Fast Proximity Queries between Polyhedra Using Surface Decomposition. In Proc. of Eurographics, 2001.

A. Gregory, M. C. Lin, S. Gottschalk and R. M. Taylor II. H-Collide: A Framework for Fast and Accurate Collision Detection for Haptic Interaction. In Proc. of IEEE Virtual Reality Conference, 1999.

A. Gregory, A. Mascarenhas, S. Ehmann, M. C. Lin and D. Manocha. Six Degree-of-Freedom Haptic Display of Polygonal Models. In Proc. of IEEE Visualization, 2000.


References

V. Hayward and B. Armstrong. A New Computational Method of Friction Applied to Haptic Rendering. In Lecture Notes in Control and Information Sciences, Vol. 250, Springer-Verlag, 2000.

Y. J. Kim, M. A. Otaduy, M. C. Lin and D. Manocha. Six-Degree-of-Freedom Haptic Display Using Localized Contact Computations. To appear in Proc. of Haptic Symposium, 2002.

W. R. Mark, S. C. Randolph, M. Finch, J. M. Van Verth, and R. M. Taylor II. Adding Force Feedback to Graphics Systems: Issues and Solutions. In Proc. of ACM SIGGRAPH, 1996.

W. A. McNeely, K. D. Puterbaugh and J. J. Troy. Six Degree-of-Freedom Haptic Rendering Using Voxel Sampling. In Proc. of ACM SIGGRAPH, 1999.

D. C. Ruspini, K. Kolarov, and O. Khatib. The Haptic Display of Complex Graphical Environments. In Proc. of ACM SIGGRAPH, 1997.

C. Zilles and K. Salisbury. A Constraint-based God Object Method for Haptic Display. In Proc. of IEEE/RSJ Int. Conference on Intelligent Robots and Systems, 1995.

introduction to haptic rendering ming c. lin [email protected]

Documents