high-performance wide-area optical trackingwelch/media/pdf/pres101_welch_etal.pdf ·...

Greg [email protected]

Gary [email protected]

Leandra [email protected]

Stephen [email protected]

Kurtis [email protected] of Computer ScienceUniversity of North Carolina atChapel Hill

D’nardo [email protected] Realities Corporation

Presence, Vol. 10, No. 1, February 2001, 1–21

© 2001 by the Massachusetts Institute of Technology

High-Performance Wide-AreaOptical TrackingThe HiBall Tracking System

Abstract

Since the early 1980s, the Tracker Project at the University of North Carolina atChapel Hill has been working on wide-area head tracking for virtual and augmentedenvironments. Our long-term goal has been to achieve the high performance re-quired for accurate visual simulation throughout our entire laboratory, beyond intothe hallways, and eventually even outdoors.

In this article, we present results and a complete description of our most recentelectro-optical system, the HiBall Tracking System. In particular, we discuss motiva-tion for the geometric configuration and describe the novel optical, mechanical,electronic, and algorithmic aspects that enable unprecedented speed, resolution,accuracy, robustness, and flexibility.

1 Introduction

Systems for head tracking for interactive computer graphics have beenexplored for more than thirty years (Sutherland, 1968). As illustrated infigure 1, the authors have been working on the problem for more than twentyyears (Azuma, 1993, 1995; Azuma & Bishop, 1994a, 1994b; Azuma & Ward,1991; Bishop, 1984; Gottschalk & Hughes, 1993; UNC Tracker Project,2000; Wang, 1990; Wang et al., 1990; Ward, Azuma, Bennett, Gottschalk, &Fuchs, 1992; Welch, 1995, 1996; Welch & Bishop, 1997; Welch et al., 1999).From the beginning, our efforts have been targeted at wide-area applicationsin particular. This focus was originally motivated by applications for which webelieved that actually walking around the environment would be superior tovirtually “flying.” For example, we wanted to interact with room-filling virtualmolecular models, and to naturally explore life-sized virtual architectural mod-els. Today, we believe that a wide-area system with high performance every-where in our laboratory provides increased flexibility for all of our graphics,vision, and interaction research.

1.1 Previous Work

In the early 1960s, Ivan Sutherland implemented both mechanical andultrasonic (carrier phase) head-tracking systems as part of his pioneering workin virtual environments. He describes these systems in his seminal paper “AHead-Mounted Three Dimensional Display” (Sutherland, 1968). In the

Welch et al. 1

ensuing years, commercial and research teams have ex-plored mechanical, magnetic, acoustic, inertial, and op-tical technologies. Complete surveys include Bhatnagar(1993); Burdea & Coiffet (1994); Meyer, Applewhite,& Biocca (1992); and Mulder (1994a, 1994b, 1998).Commercial magnetic tracking systems for example(Ascension, 2000; Polhemus, 2000) have enjoyed popu-larity as a result of a small user-worn component andrelative ease of use. Recently, inertial hybrid systems(Foxlin, Harrington, & Pfeifer, 1998; Intersense, 2000)have been gaining popularity for similar reasons, withthe added benefit of reduced high-frequency noise anddirect measurements of derivatives.

An early example of an optical system for tracking ormotion capture is the Twinkle Box by Burton (Burton,1973; Burton & Sutherland, 1974). This system mea-sured the positions of user-worn flashing lights withoptical sensors mounted in the environment behind ro-tating slotted disks. The Selspot system (Woltring, 1974)used fixed, camera-like, photodiode sensors and target-mounted infrared light-emitting diodes that could betracked in a one-cubic-meter volume. Beyond theHiBall Tracking System, examples of current opticaltracking and motion-capture systems include the Flash-

Point and Pixsys systems by Image Guided Technologies(IGT, 2000), the laserBIRD system by Ascension Tech-nology (Ascension, 2000), and the CODA Motion Cap-ture System by B & L Engineering (BL, 2000). Thesesystems employ analog optical-sensor systems to achieverelatively high sample rates for a moderate number oftargets. Digital cameras (two-dimensional, image-formingoptical devices) are used in motion-capture systems suchas the HiRes 3D Motion Capture System by the MotionAnalysis Corporation (Kadaba & Stine, 2000; MAC,2000) to track a relatively large number of targets, al-beit at a relatively low rate because of the need for 2-Dimage processing.

1.2 Previous Work at UNC-Chapel Hill

As part of his 1984 dissertation on Self-Tracker,Bishop put forward the idea of outward-looking track-ing systems based on user-mounted sensors that esti-mate user pose1 by observing landmarks in the environ-ment (Bishop, 1984). He described two kinds of

1. We use the word pose to indicate both position and orientation(six degrees of freedom).

Figure 1.

2 PRESENCE: VOLUME 10, NUMBER 1

landmarks: high signal-to-noise-ratio beacons such aslight-emitting diodes (LEDs) and low signal-to-noise-ratio landmarks such as naturally occurring features.Bishop designed and demonstrated custom VLSI chips(figure 2) that combined image sensing and processingon a single chip (Bishop & Fuchs, 1984). The idea wasto combine multiple instances of these chips into anoutward-looking cluster that estimated cluster motionby observing natural features in the unmodified environ-ment. Integrating the resulting motion to estimate poseis prone to accumulating error, so further developmentrequired a complementary system based on easily de-tectable landmarks (LEDs) at known locations. ThisLED-based system was the subject of a 1990 disserta-tion by Jih-Fang Wang (Wang, 1990).

In 1991, we demonstrated a working, scalable, elec-tro-optical head-tracking system in the Tomorrow’s Re-alities gallery at that year’s ACM SIGGRAPH confer-ence (Wang et al., 1990; Wang, Chi, & Fuchs, 1990;Ward et al., 1992). The system (figure 3) used four,head-worn, lateral-effect photodiodes that looked up-ward at a regular array of infrared LEDs installed in pre-cisely machined ceiling panels. A user-worn backpackcontained electronics that digitized and communicatedthe photo-coordinates of the sighted LEDs. Photo-grammetric techniques were used to compute a user’shead pose using the known LED positions and the cor-responding measured photo-coordinates from eachLEPD sensor (Azuma & Ward, 1991). The system wasground-breaking in that it was unaffected by ferromag-

netic and conductive materials in the environment, andthe working volume of the system was determinedsolely by the number of ceiling panels. (See figure 3,top.)

Figure 2.

Figure 3.

Welch et al. 3

1.3 The HiBall Tracking System

In this article, we describe a new and vastly im-proved version of the 1991 system. We call the new sys-tem the HiBall Tracking System. Thanks to significantimprovements in hardware and software, this HiBallsystem offers unprecedented speed, resolution, accuracy,robustness, and flexibility. The bulky and heavy sensorsand backpack of the previous system have been replacedby a small HiBall unit (figure 4, bottom). In addition,the precisely machined LED ceiling panels of the previ-ous system have been replaced by looser-tolerance pan-els that are relatively inexpensive to make and simple toinstall (figure 4, top; figure 10). Finally, we are using anunusual Kalman-filter-based algorithm that generatesvery accurate pose estimates at a high rate with low la-tency, and that simultaneously self-calibrates the system.

As a result of these improvements, the HiBall Track-ing System can generate more than 2,000 pose esti-mates per second, with less than 1 ms of latency, better

than 0.5 mm and 0.03 deg. of absolute error and noise,everywhere in a 4.5 m 3 8.5 m room (with more thantwo meters of height variation). The area can be ex-panded by adding more panels, or by using checker-board configurations that spread panels over a largerarea. The weight of the user-worn HiBall is approxi-mately 300 grams, making it lighter than one opticalsensor in the 1991 system. Multiple HiBall units can bedaisy-chained together for head or hand tracking, pose-aware input devices, or precise 3-D point digitizationthroughout the entire working volume.

2 Design Considerations

In all of the optical systems we have developed(see section 1.2), we have chosen what we call an inside-looking-out configuration, in which the optical sensorsare on the (moving) user and the landmarks (for in-stance, the LEDs) are fixed in the laboratory. The corre-sponding outside-looking-in alternative would be toplace the landmarks on the user and to fix the opticalsensors in the laboratory. (One can think about similaroutside-in and inside-out distinctions for acoustic andmagnetic technologies.) The two configurations are de-picted in figure 5.

There are some disadvantages to the inside-looking-out approach. For small or medium-sized working vol-umes, mounting the sensors on the user is more chal-lenging than mounting them in the environment. It isdifficult to make user-worn sensor packaging small, andcommunication from the moving sensors to the rest ofthe system is more complex. In contrast, there are fewermechanical considerations when mounting sensors inthe environment for an outside-looking-in configura-tion. Because landmarks can be relatively simple, small,and cheap, they can often be located in numerous placeson the user, and communication from the user to therest of the system can be relatively simple or even un-necessary. This is particularly attractive for full-bodymotion capture (BL, 2000; MAC, 2000).

However, there are some significant advantages to theinside-looking-out approach for head tracking. Byoperating with sensors on the user rather than in the

Figure 4.


environment, the system can be scaled indefinitely. Thesystem can evolve from using dense active landmarks tofewer, lower signal-to-noise ratio, passive, and some daynatural features for a Self-Tracker that operates entirelywithout explicit landmark infrastructure (Bishop, 1984;Bishop & Fuchs, 1984; Welch, 1995).

The inside-looking-out configuration is also moti-vated by a desire to maximize sensitivity to changes inuser pose. In particular, a significant problem with anoutside-looking-in configuration is that only positionestimates can be made directly, and so orientation mustbe inferred from position estimates of multiple fixedlandmarks. The result is that orientation sensitivity is afunction of both the distance to the landmarks from thesensor and the baseline between the landmarks on theuser. In particular, as the distance to the user increasesor the baseline between the landmarks decreases, thesensitivity goes down. For sufficient orientation sensitiv-ity, one would likely need a baseline that is considerablylarger than the user’s head. This would be undesirablefrom an ergonomic standpoint and could actually re-strict the user’s motion.

With respect to translation, the change in measuredphoto-coordinates is the same for an environment-mounted (fixed) sensor and user-mounted (moving)landmark as it is for a user-mounted sensor and an envi-ronment-mounted landmark. In other words, the trans-lation and corresponding sensitivity are the same foreither case.

3 System Overview

The HiBall Tracking System consists of threemain components (figure 6). An outward-lookingsensing unit we call the HiBall is fixed to each user tobe tracked. The HiBall unit observes a subsystem offixed-location infrared LEDs we call the Ceiling.2

Communication and synchronization between thehost computer and these subsystems is coordinated

2. At the present time, the LEDs are in fact entirely located in theceiling of our laboratory (hence the subsystem name Ceiling), butLEDs could as well be located on walls or other fixed locations.

Figure 5.

Welch et al. 5

by the Ceiling-HiBall Interface Board (CIB). In sec-tion 4, we describe these components in more detail.

Each HiBall observes LEDs through multiple sen-sor-lens views that are distributed over a large solidangle. LEDs are sequentially flashed (one at a time)such that they are seen via a diverse set of views foreach HiBall. Initial acquisition is performed using abrute-force search through LED space, but, once ini-tial lock is made, the selection of LEDs to flash is tai-lored to the views of the active HiBall units. Pose es-timates are maintained using a Kalman-filter-basedprediction-correction approach known as single-constraint-at-a-time (SCAAT) tracking. This tech-nique has been extended to provide self-calibration ofthe ceiling, concurrent with HiBall tracking. In sec-tion 5, we describe the methods we employ, includ-ing the initial acquisition process and the SCAAT ap-proach to pose estimation, with the autocalibrationextension.

4 System Components

4.1 The HiBall

The original electro-optical tracker (figure 3, bot-tom) used independently housed lateral-effect photo-diode units (LEPDs) attached to a lightweight tubularframework. As it turns out, the mechanical frameworkwould flex (distort) during use, contributing to estima-tion errors. In part to address this problem, the HiBallsensor unit was designed as a single, rigid, hollow ballhaving dodecahedral symmetry, with lenses in the uppersix faces and LEPDs on the insides of the opposing sixlower faces (figure 7). This immediately gives six pri-mary “camera” views uniformly spaced by 57 deg. Theviews efficiently share the same internal air space and arerigid with respect to each other. In addition, light enter-ing any lens sufficiently off-axis can be seen by a neigh-boring LEPD, giving rise to five secondary views throughthe top or central lens, and three secondary views

Figure 6.


through the five other lenses. Overall, this provides 26fields of view that are used to sense widely separatedgroups of LEDs in the environment. Although the extraviews complicate the initialization of the Kalman filter asdescribed in section 5.5, they turn out to be of greatbenefit during steady-state tracking by effectively in-creasing the overall HiBall field of view without sacrific-ing optical-sensor resolution.

The lenses are simple plano-convex fixed-focus lenses.Infrared (IR) filtering is provided by fabricating thelenses themselves from RG-780 Schott glass filter mate-rial which is opaque to better than 0.001% for all visiblewavelengths and transmissive to better than 99% for IRwavelengths longer than 830 nm. The longwave filter-ing limit is provided by the DLS-4 LEPD silicon photo-detector (UDT Sensors, Inc.) with peak responsivity at950 nm but essentially blind above 1150 nm.

The LEPDs themselves are not imaging devices;rather, they detect the centroid of the luminous fluxincident on the detector. The x-position of the centroiddetermines the ratio of two output currents, and the

y-position determines the ratio of two other output cur-rents. The total output current of each pair are com-mensurate and are proportional to the total incidentflux. Consequently, focus is not an issue, so the simplefixed-focus lenses work well over a range of LED dis-tances from about half a meter to infinity. The LEPDsand associated electronic components are mounted on acustom rigid-flex printed circuitboard (figure 8). Thisarrangement makes efficient use of the internal HiBallvolume while maintaining isolation between analog anddigital circuitry, and increasing reliability by alleviatingthe need for intercomponent mechanical connectors.

Figure 9 shows the physical arrangement of thefolded electronics in the HiBall. Each LEPD has fourtransimpedance amplifiers (shown together as one“Amp” in figure 9), the analog outputs of which aremultiplexed with those of the other LEPDs, then sam-pled, held, and converted by four 16-bit Delta-Sigmaanalog-to-digital (A/D) converters. Multiple samplesare integrated via an accumulator. The digitized LEPDdata are organized into packets for communication backto the CIB. The packets also contain information toassist in error detection. The communication protocol issimple, and, while presently implemented by wire, themodulation scheme is amenable to a wireless implemen-tation. The present wired implementation allows multi-ple HiBall units to be daisy-chained, so a single cablecan support a user with multiple HiBall units.

Figure 7.

Figure 8.

Welch et al. 7

4.2 The Ceiling

As presently implemented, the infrared LEDs arepackaged in 61 cm square panels to fit a standard false-ceiling grid (figure 10, top). Each panel uses five printedcircuit boards: a main controller board and four identi-cal transverse-mounted strips (bottom). Each strip ispopulated with eight LEDs for a total of 32 LEDs perpanel. We mount the assembly on top of a metal panelsuch that the LEDs protrude through 32 correspondingholes. The design results in a ceiling with a rectangularLED pattern with periods of 7.6 cm and 15.2 cm. Thisspacing is used for the initial estimates of the LED posi-tions in the lab; then, during normal operation, theSCAAT algorithm continually refines the LED position

estimates (section 5.4). The SCAAT autocalibration notonly relaxes design and installation constraints, but pro-vides greater precision in the face of initial and ongoinguncertainty in the ceiling structure.

We currently have enough panels to cover an areaapproximately 5.5 m by 8.5 m with a total of approxi-mately 3,000 LEDs.3 The panels are daisy-chained toeach other, and panel-selection encoding is position(rather than device) dependent. Operational commandsare presented to the first panel of the daisy chain. Ateach panel, if the panel-select code is zero, the

3. The area is actually L-shaped; a small storage room occupies onecorner.

Figure 9.


controller decodes and executes the operation; other-wise, it decrements the panel-select code and passes italong to the next panel (controller). Upon decoding, aparticular LED is selected and the LED is energized.The LED brightness (power) is selectable for automaticgain control as described in section 5.2.

We currently use Siemens SFH-487P GaAs LEDs,which provide both a wide-angle radiation pattern andhigh peak power, emitting at a center wavelength of880 nm in the near IR. These devices can be pulsed upto 2.0 Amps for a maximum duration of 200 ms with a1:50 (on:off) duty cycle. Although the current ceilingarchitecture allows flashing of only one LED at a time,LEDs may be flashed in any sequence. As such, no sin-gle LED can be flashed too long or too frequently. Weinclude both hardware and software protection to pre-vent this.

4.3 The Ceiling-HiBall Interface Board

The Ceiling-HiBall Interface Board (CIB)(figure 11) provides communication and synchroniza-tion between a host personal computer, the HiBall(section 4.1), and the ceiling (section 4.2). The CIB hasfour ceiling ports allowing interleaving of ceiling panelsfor up to four simultaneous LED flashes and/or higher

ceiling bandwidth. (The ceiling bandwidth is inherentlylimited by LED power restrictions as described in sec-tion 4.2, but this can be increased by spatially multiplex-ing the ceiling panels.) The CIB has two tether inter-faces that can communicate with up to four daisy-chained HiBall units. The full-duplex communicationwith the HiBall units uses a modulation scheme (BPSK)allowing future wireless operation. The interface fromthe CIB to the host PC is the stable IEEE1284C ex-tended parallel port (EPP) standard.

The CIB comprises analog drive and receive compo-nents as well as digital logic components. The digitalcomponents implement store and forward in both direc-tions and synchronize the timing of the LED “on” in-terval within the HiBall dark-light-dark intervals(section 5.2). The protocol supports full-duplex flowcontrol. The data are arranged into packets that incor-porate error detection.

5 Methods

5.1 Bench-Top (Offline) HiBallCalibration

After each HiBall is assembled, we perform an off-line calibration procedure to determine the correspon-dence between image-plane coordinates and rays inspace. This involves more than just determining theview transform for each of the 26 views. Nonlinearitiesin the silicon sensor and distortions in the lens (such asspherical aberration) cause significant deviations from asimple pinhole camera model. We dealt with all of theseissues through the use of a two-part camera model. Thefirst part is a standard pinhole camera represented by a

Figure 10.

Figure 11.

Welch et al. 9

3 3 4 matrix. The second part is a table mapping realimage-plane coordinates to ideal image-plane coordi-nates.

Both parts of the camera model are determined usinga calibration procedure that relies on a goniometer (anangular positioning system) of our own design. Thisdevice consists of two servo motors mounted togethersuch that one motor provides rotation about the verticalaxis while the second motor provides rotation about anaxis orthogonal to vertical. An important characteristicof the goniometer is that the rotational axes of the twomotors intersect at a point at the center of the HiBalloptical sphere; this point is defined as the origin of theHiBall. (It is this origin that provides the reference forthe HiBall state during runtime as described in section5.3.) The rotational positioning motors were rated toprovide twenty arc-second precision; we further cali-brated them to six arc seconds using a laboratory gradetheodolite—an angle measuring system.

To determine the mapping between sensor image-plane coordinates and three-space rays, we use a sin-gle LED mounted at a fixed location in the laboratorysuch that it is centered in the view directly out of thetop lens of the HiBall. This ray defines the z or upaxis for the HiBall coordinate system. We sampleother rays by rotating the goniometer motors undercomputer control. We sample each view with raysspaced about every six minutes of arc throughout thefield of view. We repeat each measurement 100 timesto reduce the effects of noise on the individual mea-surements and to estimate the standard deviation ofthe measurements.

Given the tables of approximately 2,500 measure-ments for each of the 26 views, we first determine a3 3 4 view matrix using standard linear least-squarestechniques. Then, we determine the deviation of eachmeasured point from that predicted by the ideal linearmodel. These deviations are resampled into a 25 3 25grid indexed by sensor-plane coordinates using a simplescan-conversion procedure and averaging. Given a mea-surement from a sensor at runtime (section 5.2), weconvert it to an “ideal” measurement by subtracting adeviation bilinearly interpolated from the nearest fourentries in the table.

5.2 Online HiBall Measurements

Upon receiving a command from the CIB (section4.3), which is synchronized with a CIB command to theceiling, the HiBall selects the specified LEPD and per-forms three measurements, one before the LED flashes,one during the LED flash, and one after the LED flash.Known as “dark-light-dark,” this technique is used tosubtract out DC bias, low-frequency noise, and back-ground light from the LED signal. We then convert themeasured sensor coordinates to “ideal” coordinates us-ing the calibration tables described in section 5.1.

In addition, during runtime we attempt to maximizethe signal-to-noise ratio of the measurement with anautomatic gain-control scheme. For each LED, we storea target signal strength factor. We compute the LEDcurrent and number of integrations (of successive accu-mulated A/D samples) by dividing this strength factorby the square of the distance to the LED, estimatedfrom the current position estimate. After a reading, welook at the strength of the actual measurement. If it islarger than expected, we reduce the gain; if it is less thanexpected, we increase the gain. The increase and de-crease are implemented as online averages with scalingsuch that the gain factor decreases rapidly (to avoidoverflow) and increases slowly. Finally, we use the mea-sured signal strength to estimate the noise on the signalusing (Chi, 1995), and then use this as the measure-ment noise estimate for the Kalman filter (section 5.3).

5.3 Recursive Pose Estimation(SCAAT)

The online measurements (section 5.2) are used toestimate the pose of the HiBall during operation. The1991 system collected a group of diverse measurementsfor a variety of LEDs and sensors, and then used amethod of simultaneous nonlinear equations called col-linearity (Azuma & Ward, 1991) to estimate the poseof the sensor fixture shown in figure 3 (bottom). Therewas one equation for each measurement, expressing theconstraint that a ray from the front principal point ofthe sensor lens to the LED must be collinear with a rayfrom the rear principal point to the intersection with the


sensor. Each estimate made use of a group of measure-ments (typically twenty or more) that together overcon-strained the solution.

This multiple constraint method had several draw-backs. First, it had a significantly lower estimate rate dueto the need to collect multiple measurements per esti-mate. Second, the system of nonlinear equations did notaccount for the fact that the sensor fixture continued tomove throughout the collection of the sequence ofmeasurements. Instead, the method effectively assumesthat the measurements were taken simultaneously. Theviolation of this simultaneity assumption could intro-duce significant error during even moderate motion.Finally, the method provided no means to identify orhandle unusually noisy individual measurements. Thus,a single erroneous measurement could cause an estimateto jump away from an otherwise smooth track.

In contrast, the approach we use with the new HiBallsystem produces tracker reports as each new measure-ment is made, rather than waiting to form a completecollection of observations. Because single measurementsunderconstrain the mathematical solution, we refer tothe approach as single-constraint-at-a-time (SCAAT)tracking (Welch, 1996; Welch & Bishop, 1997). Thekey is that the single measurements provide some infor-mation about the HiBall’s state, and thus can be used toincrementally improve a previous estimate. We inten-tionally fuse each individual “insufficient” measurementimmediately as it is obtained. With this approach, we areable to generate estimates more frequently, with lesslatency, and with improved accuracy, and we are able toestimate the LED positions online concurrently whiletracking the HiBall (section 5.4).

We use a Kalman filter (Kalman, 1960) to fuse themeasurements into an estimate of the HiBall state x# (thepose of the HiBall). We use the Kalman filter—a mini-mum-variance stochastic estimator—both because thesensor measurement noise and the typical user-motiondynamics can be modeled as normally distributed ran-dom processes, and because we want an efficient onlinemethod of estimation. A basic introduction to the Kal-man filter can be found in chapter 1 of Maybeck(1979), and a more complete introductory discussioncan be found in Sorenson (1970), which also contains

some interesting historical narrative. More-extensivereferences can be found in Brown and Hwang (1992),Gelb (1974), Jacobs (1993), Lewis (1986), Maybeck(1979), and Welch and Bishop (1995). Finally, wemaintain a Kalman filter Web page (Welch & Bishop,2000) with introductory, reference, and research mate-rial.

The Kalman filter has been used previously to addresssimilar or related problems. See, for example, Az-arbayejani and Pentland (1995), Azuma (1995), Emuraand Tachi (1994), Fuchs (Foxlin) (1993), Mazuryk andGervautz (1995), and Van Pabst and Krekel (1993). Arelevant example of a Kalman filter used for sensor fu-sion in a wide-area tracking system is given in Foxlin etal. (1998), which describes a hybrid inertial-acousticsystem that is commercially available today (Intersense,2000).

The SCAAT approach is described in detail by Welch(1996), and Welch and Bishop (1997). Included thereis discussion of the benefits of using the approach, asopposed to a multiple-constraint approach such as thatby Azuma and Ward (1991). However, one key benefitwarrants discussion here. There is a direct relationshipbetween the complexity of the estimation algorithm, thecorresponding speed (execution time per estimation cy-cle), and the change in HiBall pose between estimationcycles (figure 12). As the algorithmic complexity in-creases, the execution time increases, which allows forsignificant nonlinear HiBall motion between estimationcycles, which in turn implies the need for a more com-plex estimation algorithm.

The SCAAT approach, on the other hand, is an at-tempt to reverse this cycle. Because we intentionally usea single constraint per estimate, the algorithmic com-plexity is drastically reduced, which reduces the execu-tion time, and hence the amount of motion betweenestimation cycles. Because the amount of motion is lim-ited, we are able to use a simple dynamic (process)model in the Kalman filter, which further simplifies thecomputations. In short, the simplicity of the approachmeans that it can run very fast, which means it can pro-duce estimates very rapidly, with low noise.

The Kalman filter requires both a model of the pro-cess dynamics and a model of the relationship between

Welch et al. 11

the process state and the available measurements. Inpart due to the simplicity of the SCAAT approach, weare able to use a simple position-velocity (PV) processmodel (Brown & Hwang, 1992). Consider the simpleexample state vector x#(t) 5 [xp(t), xv(t)]

T, where thefirst element xp(t) is the pose (position or orientation)and the second element xv(t) is the corresponding ve-locity; that is, xv(t) 5 (d/dt) xp(t). We model the con-tinuous change in the HiBall state with the simple dif-ferential equation

ddt x#(t) 5 F0 1

0 0GFxp(t)xn(t)G 1 F0

mGu(t), (1)

where u(t) is a normally distributed white (in the fre-quency spectrum) scalar noise process, and the scalar m

represents the magnitude or spectral density of thenoise. We use a similar model with a distinct noise pro-cess for each of the six pose elements. We determine theindividual noise magnitudes using an offline simulationof the system and a nonlinear optimization strategy thatseeks to minimize the variance between the estimatedpose and a known motion path. (See section 6.2.2.).The differential equation (1) represents a continuousintegrated random walk, or an integrated Wiener orBrownian-motion process. Specifically, we model each

component of the linear and angular HiBall velocities asa random walk, and then use these (assuming constantintermeasurement velocity) to estimate the HiBall poseat time t 1 dt as follows:

x#(t 1 dt) 5 F1 dt0 1Gx#(t) (2)

for each of the six pose elements. In addition to a rela-tively simple process model, the HiBall measurementmodel is relatively simple. For any ceiling LED (section4.2) and HiBall view (section 4.1), the 2-D sensor mea-surement can be modeled as

FunG 5 Fcx/cz

cy/czG (3)

where

Fcx

cy

czG 5 VRT(#lxyz 2 x#xyz), (4)

V is the camera viewing matrix from section 5.1, #lxyz isthe position of the LED in the world, x#xyz is the posi-tion of the HiBall in the world, and R is a rotation ma-trix corresponding to the orientation of the HiBall inthe world. In practice, we maintain the orientation ofthe HiBall as a combination of a global (external to thestate) quaternion and a set of incremental angles as de-scribed by Welch (1996) and Welch and Bishop (1997).

Because the measurement model (3) and (4) is non-linear, we use an extended Kalman filter, making use ofthe Jacobian of the nonlinear HiBall measurementmodel to transform the covariance of the Kalman filter.Although this approach does not preserve the presumedGaussian nature of the process, it has been used success-fully in countless applications since the introduction ofthe (linear) Kalman filter. Based on observations of thestatistics of the HiBall filter residuals, the approach alsoappears to work well for the HiBall. In fact, it is reason-able to expect that it would, as the speed of the SCAATapproach minimizes the distance (in state space) overwhich we use the Jacobian-based linear approximation.This is another example of the importance of the rela-tionship shown in figure 12.

Figure 12.


At each estimation cycle, the next of the 26 possibleviews is chosen randomly. Four points corresponding tothe corners of the LEPD sensor associated with thatview are projected into the world using the 3 3 4 view-ing matrix for that view, along with the current esti-mates of the HiBall pose. This projection, which is theinverse of the measurement relationship describedabove, results in four rays extending from the sensorinto the world. The intersection of these rays and theapproximate plane of the ceiling determines a 2-Dbounding box on the ceiling, within which are the can-didate LEDs for the current view. One of the candidateLEDs is then chosen in a least-recently-used fashion toensure a diversity of constraints.

Once a particular view and LED have been chosen inthis fashion, the CIB (section 4.3) is instructed to flashthe LED and take a measurement as described in sec-tion 5.2. This single measurement is compared with aprediction obtained using equation (3), and the differ-ence (or residual) is used to update the filter state andcovariance matrices using the Kalman gain matrix. TheKalman gain is computed as a combination of the cur-rent filter covariance, the measurement noise variance(section 6.2.1), and the Jacobian of the measurementmodel. This recursive prediction-correction cycle con-tinues in an ongoing fashion, a single constraint at atime.

A more detailed discussion of the HiBall Kalman filterand the SCAAT approach is beyond the scope of thispaper. For additional information see Welch (1996) andWelch and Bishop (1997).

5.4 Online LED Autocalibration

Along with the benefit of simplicity and speed, theSCAAT approach offers the additional capability of be-

ing able to estimate the 3-D positions of the LEDs inthe world concurrently with the pose of the HiBall, on-line, in real time. This capability is a tremendous benefitin terms of the accuracy and noise characteristics of theestimates. Accurate LED position estimates are so im-portant that, prior to the introduction of the SCAATapproach, a specialized offline approach was developedto address the problem (Gottschalk & Hughes, 1993).

The method we now use for autocalibration involvesdefining a distinct SCAAT Kalman filter for each LED.Specifically, for each LED, we maintain a state #l (esti-mate of the 3-D position) and a 3 3 3 Kalman filtercovariance. At the beginning of each estimation cycle,we form an augmented state vector xÜ using theappropriate LED state and the current HiBall state:xÜ 5 [x#T, l#T]T. Similarly, we augment the Kalman filtererror covariance matrix with that of the LED filter. Wethen follow the normal steps outlined in section 5.3,with the result being that the LED portion of the filterstate and covariance is updated in accordance with themeasurement residual. At the end of the cycle, we ex-tract the LED portions of the state and covariance fromthe augmented filter, and save them externally. The ef-fect is that, as the system is being used, it continuallyrefines its estimates of the LED positions, thereby con-tinually improving its estimates of the HiBall pose.Again, for additional information, see Welch (1996)and Welch and Bishop (1997).

5.5 Initialization and Reacquisition

The recursive nature of the Kalman filter (section5.3) requires that the filter be initialized with a knownstate and corresponding covariance before steady-stateoperation can begin. Such an initialization (or acquisi-tion) must take place prior to any tracking session, butalso upon the (rare) occasion when the filter divergesand “loses lock” as a result of blocked sensor views, forexample.

The acquisition process is complicated by the fact thateach LEPD sees a number of different widely separatedviews (section 4.1). Therefore, detecting an LED pro-vides at best an ambiguous set of potential LED direc-tions in HiBall coordinates. Moreover, before acquisition,

Figure 12a.

Welch et al. 13

no assumptions can be made to limit the search space ofvisible LEDs. As such, a relatively slow brute-force algo-rithm is used to acquire lock.

We begin with an exhaustive LED scan of sufficientlyfine granularity to ensure that the central primary fieldof view is not missed. For the present ceiling, we flashevery thirteenth LED in sequence, and look for it withthe central LEPD until we get a hit. Then, a sufficientlylarge patch of LEDs, centered on the hit, is sampled toensure that several of the views of the central LEPD willbe hit. The fields of view are disambiguated by using theinitial hits to estimate the yaw of the HiBall (rotationabout vertical); finally, more-selective measurements areused to refine the acquisition estimate sufficiently toswitch into tracking mode.

6 Results

Three days after the individual pieces of hardwarewere shown to be functioning properly, we demon-strated a complete working system. After months ofsubsequent tuning and optimization, the system contin-ues to perform both qualitatively and quantitatively aswell—or, in some respects, better—than we had antici-pated (section 6.1). The articulation of this success isnot meant to be self-congratulatory, but to give creditto the extensive and careful modeling and simulationperformed prior to assembly (section 6.2). In fact, theKalman filter parameters found by the optimization pro-cedure described in section 6.2.2 were, and continue tobe, used directly in the working system. Likewise, muchof the software written for the original simulations con-tinues to be used in the working system.

6.1 Online Operation

The HiBall system is in daily use as a tool for edu-cation and research. For example, it was used by MartinUsoh et al. to perform virtual reality experiments com-paring virtual “flying,” walking in place, and real walk-ing (Usoh et al., 1999). (See figure 13.) The researchersused the HiBall system to demonstrate that, as a modeof locomotion, real walking is simpler, more straightfor-

ward, and more natural, than both virtual flying andwalking in place. The unprecedented combination oflarge working volume and the high performance of theHiBall system led the researchers to claim that there wasnowhere else that they could have meaningfully per-formed the experiments.

6.1.1 Robustness. As a result of a mechanicaldesign tradeoff, each sensor field of view is less than sixdegrees. The focal length is set by the size of the sensorhousing, which is set by the diameter of the sensorsthemselves. Energetics is also a factor, limiting howsmall the lenses can be while maintaining sufficientlight-collecting area. As a result of these designtradeoffs, even a momentary small error in the HiBallpose estimate can cause the recursive estimates to di-verge and the system to lose lock after only a few LEDsightings. And yet the system is quite robust. In prac-tice, users can jump around, crawl on the floor, leanover, even wave their hands in front of the sensors, andthe system does not lose lock. During one session, wewere using the HiBall as a 3-D digitization probe, a Hi-Ball on the end of a pencil-shaped fiberglass wand (fig-ure 14, left). We laid the probe down on a table at onepoint, and were amazed to later notice that it was stilltracking, even though it was observing only three orfour LEDs near the edge of the ceiling. We picked upthe probe and continued using it, without it ever losinglock.

6.1.2 Estimate Noise. The simplest quantitativemeasurement of estimate noise is the standard deviationof the estimates when a HiBall is held stationary. With atracker as sensitive as the HiBall, it is important to becertain that it really is stationary. The raised floor in our

Figure 13.


laboratory allows motion, for example when a personwalks by, that is larger than the expected error in theHiBall. We made careful measurements by resting thesupport for the HiBall on the concrete subfloor in ourlaboratory. The standard deviation of the HiBall esti-mates while stationary was approximately 0.2 mm and0.03 deg. The distribution of the noise fit a normal dis-tribution quite well.

To make measurements of the noise when the HiBallis in motion, we rely on the assumption that almost allof the signal resulting from normal human motion is atfrequencies below 2 Hz. We use a high-pass filter(Welch, 1967) on the pose estimates, and assume theoutput is noise. The resulting statistics are comparableto those made with the HiBall stationary, except atposes for which there are very few LEDs visible in onlyone or two views. In these poses, near the edge of theceiling, the geometry of the constraints results in ampli-fication of errors. For nearly all of the working volumeof the tracker, the standard deviation of the noise onmeasurements while the HiBall is still or moving isabout 0.2 mm and 0.03 deg.

6.1.3 Absolute Accuracy. We have performedseveral experiments to measure the accuracy of the Hi-Ball system; however, the most objective experimenttook place in July of 1999. Boeing Phantom Works sci-entists David Himmel and David Princehouse (AssociateTechnical Fellows) visited our laboratory for two days toassess the accuracy of the HiBall system and its potentialuse in providing assembly workers with real-time feed-

back on the pose of handheld pneumatic drills duringthe aircraft manufacturing process. (The right image infigure 14 shows the HiBall attached to a pneumaticdrill.)

The scientists designed some controlled experimentsto assess the accuracy of the HiBall system. Theybrought with them an aluminum “coupon” (see figure14 and figure 15) with 27 shallow holes drilled on 1.5-in. centers using a numerically controlled milling ma-chine with a stated accuracy of 1/1000 in. The holes(except one) were not actually drilled through the cou-pon, but instead formed conical dimples with a finepoint at the center. The center-most hole (hole 14) wasactually drilled completely through to provide a mount-ing point. Using that hole, we attached the coupon to amilitary-grade tripod situated on the (false) floor of ourlaboratory, under the HiBall ceiling. As shown in theleft image of figure 14, we mounted the HiBall on ourstandard probe, a rigid plastic, pencil-like object with apointed steel tip. We used one of the coupon holes toperform our normal HiBall probe calibration procedure,which involves placing the tip of the probe in the hole,pivoting the probe about the point while collecting sev-eral seconds of pose data, and then estimating the trans-formation from the HiBall to the probe tip. (We have astandard application that assists us with this procedure.)Together with Himmel and Princehouse, we performedseveral experiments in which we placed the tip of theHiBall probe in each hole in succession, sampling theHiBall pose estimates only when we pressed the probebutton. We performed several such sessions over the

Figure 14.

Welch et al. 15

course of one afternoon and the next morning. (Werecalibrated the probe in the morning.)

For the data from each session, we used a least-squares optimization method to find an estimate of thefull 6-D transformation (translation and rotation) thatminimized the Euclidian distance from the probe datato a 2-D plane with 27 holes on 1.5-in. spacing. Theresulting fit consistently corresponded to an averagepositioning error of 20/1000 in. (0.5 mm) at the metaltip of the HiBall probe, which is within the target Boe-ing specifications. The system might actually be moreaccurate than our experiments indicated. For one, thediameter of the (rounded) tip of the HiBall probe is 0.5mm. In addition, at the time of the experiments, weunfortunately did not heed our own advice to positionthe experimental platform on the rigid concrete sub-floor. In any case, we are encouraged by the results, andare excited about the possibility that the HiBall systemhas uses beyond tracking for virtual reality.

6.2 Offline Simulation and Modeling

During the design of the HiBall system, we madesubstantial use of simulation, in some domains to a verydetailed level. For example, Zemax (Focus Software,1995) was used extensively in the design and optimiza-tion of the optical design, including the design of thefilter glass lenses, and geometry of the optical-compo-nent layout. AutoCAD was used to design, specify, andfit check the HiBall body mechanicals, to visualize thephysical design, and to transmit the design to our col-laborators at the University of Utah for fabrication bythe Alpha 1 System (Thomas, 1984; University of UtahComputer Science, 1999). A custom ray-tracing system

was built by Stefan Gottschalk (UNC) for the purposeof evaluating the optical behavior and energetics of theprimary, secondary, and tertiary fields of view; the re-sults were used by the noise model developed by Chi(1995) as described in section 6.2.1.

In addition, a complete simulator of the system waswritten in C11. This simulator, discussed further insection 6.2.2, was used to evaluate the speed, accuracy,and robustness of the system. In addition, it was used to“tune” the Kalman filter for realistic motion dynamics.This simulator continues to be used to evaluate me-chanical, optical, and algorithmic alternatives.

6.2.1 HiBall Measurement Noise Model.Signal-to-noise performance is a prime determiner ofboth accuracy and speed of the system, so an in-depthstudy (Chi, 1995) was performed to develop a detailednoise model accounting for properties of the LED, theLEPD (sensor), the optical system, the physical distanceand pose, the electronics, and the dark-light-dark inte-grations described in section 5.2. The predominantnoise source is shot noise, with Johnson noise in thesheet resistivity of the LEPD surfaces being the nextmost significant. Careful measurements made in the lab-oratory with the actual devices yielded results that werealmost identical to those predicted by the sophisticatedmodel in Chi (1995). A simplified version of this modelis used in the real system with the automatic gain con-trol (section 5.2) to predict the measurement noise forthe Kalman filter (section 5.3).

6.2.2 Complete System Simulations. To pro-duce realistic data for developing and tuning our algo-rithms, we collected several motion paths (sequences ofpose estimates) from our first-generation electro-opticaltracker (figure 3) at its 70 Hz maximum report rate.These paths were recorded from both naive users visit-ing our monthly “demo days” and from experiencedusers in our labs. In the same fashion as we had donefor Azuma and Bishop (1994a), we filtered the raw pathdata with a noncausal zero-phase-shift, low-pass filter toeliminate energy above 2 Hz. The output of the low-pass filtering was then resampled at whatever rate wewanted to run the simulated tracker, usually 1,000 Hz.

Figure 15.


For the purposes of our simulations, we consideredthese resampled paths to be the “truth”—a perfect rep-resentation of a user’s motion. Tracking error was deter-mined by comparing the true path to the estimated pathproduced by the tracker.

The simulator reads camera models describing the 26views, the sensor noise parameters, the LED positionsand their expected error, and the motion path describedabove. Before beginning the simulation, the LED posi-tions are perturbed from their ideal positions by addingnormally distributed error to each axis. Then, for eachsimulated cycle of operation, the “true” poses are up-dated using the input motion path. Next, a view is cho-sen and a visible LED within that view is selected, andthe image-plane coordinates of the LED on the chosensensor are computed using the camera model for theview and the LED as described in section 5.3. Thesesensor coordinates are then perturbed based on the sen-sor noise model (section 6.2.1) using the distance andangle to the LED. These noise-corrupted sensor read-ings are then fed to the SCAAT filter to produce an up-dated position estimate. The position estimate is com-pared to the true position to produce a scalar errormetric that is described next.

The error metric we used combines the error in posein a way that relates to the effects of tracker error on ahead-worn display user. We define a set of points ar-rayed around the user in a fixed configuration. We com-pute two sets of coordinates for these points: the trueposition using the true pose and their estimated positionusing the estimated pose. The error metric is then thesum of the distances between the true and estimatedpositions of these points. By adjusting the distance ofthe points from the user, we can control the relativeimportance of the orientation and the position error inthe combined error metric. If the distance is small, thenthe position error is weighted most heavily; if the dis-tance is large, then the orientation error is weightedmost heavily. Our two error metrics for the entire runare the square root of the sum of the squares of all thedistances, and the peak distance.

6.2.3 Tuning. Determining the magnitudes ofthe SCAAT Kalman filter noise parameters (section 5.3)

is called system identification or tuning. We use Powell’smethod (Press, Teukolsky, Vetterling, & Flannery,1990) to minimize the error metric described above.Starting with a set of parameters, we run the simulatorover a full motion run to determine the total error forthe run. The optimizer makes a small adjustment to theparameters and the process is repeated. These runs re-quired hours of computer time and some skill (andluck) in choosing the initial parameters and step sizes.Of course, it is important to choose motion paths thatare representative of expected target motion. For exam-ple, a run in which the target is very still would result invery different tuning from a run in which the targetmoves very vigorously.

7 Future Work

7.1 Improving the HiBall

The current SCAAT filter form (section 5.3) andtuning values (section 6.2.3) are a compromise betweenthe responsiveness desired for high dynamics, and theheavy filtering desired for smooth estimates during veryslow or no motion. As such, we are investigating the useof a multimodal or multiple-model Kalman filter frame-work (Bar-Shalom & Li, 1993; Brown & Hwang,1992). A multiple-model implementation of the HiBallshould be able to automatically, continuously, andsmoothly choose between one Kalman filter tuned forhigh dynamics and another tuned for little or no mo-tion. We have this working in simulation, but not yetimplemented in the real system.

As mentioned in section 4.3, the system was designedto support wireless communication between the HiBalland the CIB, without significant modification or addedinformation overhead. Despite the fact that commercialhead-worn displays are themselves tethered at this time,we are beginning work on a completely wireless HiBalland head-worn display system. We also intend to usethe wireless HiBall with projector-based displays wherethe user is otherwise wearing only polarized glasses.Furthermore, the HiBall was designed with extrabuilt-in digital input-output capabilities. We are consid-ering possibilities for providing access to these signals

Welch et al. 17

for (wireless) user-centered input devices and evenbody-centric limb tracking.

Finally, we note that a private startup company called3rdTech (3rdTech, 2000) has negotiated a technologylicense with UNC for the existing HiBall Tracking Sys-tem. 3rdTech is now marketing an updated system withsimpler LED “strips” instead of ceiling panels.

7.2 Wide-Field-of-View HiBall

Beyond improving the existing system, we con-tinue to head down a path of research and developmentthat will lead to systems with reduced dependency onthe laboratory infrastructure. For example, our currentceiling-panel design with 32 LEDs per panel providesfar more dense coverage than we believe is necessary.The density of ceiling LEDs is a result of design basedon the original sensor fixture shown in figure 3. Given amore sparse field of LEDs, we believe that we couldachieve similar performance with a version of the HiBallthat has a small number of wide-field-of-view opticalsensor units. This would further reduce the packagingsize of the user-worn sensor component.

7.3 To the Hallway and Beyond

By leveraging the knowledge gained from success-ful work in the laboratory, our long-term goal is toachieve similar performance with little or no explicitinfrastructure: for example, throughout a building oreven (some day) outdoors. Although high-performance6-D tracking outdoors is a tremendous challenge that isunlikely to be solved any time soon, we believe that theeventual solution will involve a clever and careful com-bination of multiple complementary technologies. Inparticular, we are pursuing the hybrid approach initiallypresented by Welch (1995). We look forward to a daywhen high-performance 6-D tracking outdoors enablespose-aware devices for work such as Feiner’s outdooraugmented reality (Feiner, MacIntyre, Hollerer, &Webster, 1997; Hollerer, Feiner, Terauchi, Rashid, &Hallaway, 1999), the “WorldBoard” initiative (Spohrer,1999a, 1999b), and other wonderful applications.

Acknowledgments

We acknowledge former Tracker Project members and con-tributors (alphabetically): Ronald Azuma, Henry Fuchs, StefanGottschalk, Pawan Kumar, John Thomas, Jih-Fang Wang,Mark Ward, Scott Williams, Mary Whitton, and Philip Wins-ton. We thank Al Barr (California Institute of Technology)and John “Spike” Hughes (Brown University) for their contri-butions to the original offline LED calibration work that ledto the simpler ceiling panels (figure 1 and figure 10). Finally,we want to acknowledge our many collaborators in the NSFScience and Technology Center for Computer Graphics andScientific Visualization (below), and in particular our collabo-rators in mechanical design and fabrication at the University ofUtah: Rich Riesenfeld, Sam Drake, and Russ Fish.

This work was supported in part by DARPA/ETO contractDABT 63-93-C-0048 “Enabling Technologies and Applica-tion Demonstrations for Synthetic Environments” (PrincipalInvestigators Frederick P. Brooks, Jr. and Henry Fuchs(UNC)), and by the National Science Foundation Coopera-tive Agreement ASC-8920219 “Science and Technology Cen-ter for Computer Graphics and Scientific Visualization,” Cen-ter Director Rich Riesenfeld (University of Utah) (PrincipalInvestigators Al Barr (Caltech), Don Greenberg (Cornell Uni-versity), Henry Fuchs (UNC), Rich Riesenfeld, and Andy vanDam (Brown University)).

References

3rdTech. (2000, July 15). 3rdTech™ [Webpage]. RetrievedJuly 19, 2000, from the World Wide Web: http://www.3rdtech.com/.

Ascension. (2000). Ascension Technology Corporation [Web-page]. Retrieved September 15, 2000 from the World WideWeb: http://www.ascension-tech.com/ [2000, September15].

Azarbayejani, A., & Pentland, A. (1995). Recursive estimationof motion, structure, and focal length. IEEE Trans. PatternAnalysis and Machine Intelligence, 17(6), 562–575.

Azuma, R. T. (1993). Tracking requirements for augmentedreality. Communications of the ACM, 36(July), 50–51.

———. (1995). Predictive tracking for augmented reality. Un-published doctoral dissertation, University of North Caro-lina at Chapel Hill, Chapel Hill.

Azuma, R. T., & Bishop, G. (1994a). A frequency-domain


analysis of head-motion prediction. Computer Graphics(SIGGRAPH 94 Conference Proceedings ed.) (pp.401–408). Los Angeles: ACM Press, Addison-Wesley.

———. (1994b). Improving static and dynamic registration inan optical see-through hMD. Computer Graphics (SIG-GRAPH 94 Conference Proceedings ed.) (pp. 197–204). Or-lando: ACM Press, Addison-Wesley.

Azuma, R. T., & Ward, M. (1991). Space-resection by col-linearity: Mathematics behind the optical ceiling head-tracker(Tech. Rep. 91-048). Chapel Hill: University of NorthCarolina at Chapel Hill.

Bar-Shalom, Y., & Li, X.-R. (1993). Estimation and Tracking:Principles, Techniques, and Software. Boston: Artec House,Inc.

Bhatnagar, D. K. (1993). Position trackers for head mounteddisplay systems: A survey (Tech. Rep. TR93-010). ChapelHill: University of North Carolina at Chapel Hill.

Bishop, G. (1984). The self-tracker: A smart optical sensor onsilicon. Unpublished doctoral Dissertation, University ofNorth Carolina at Chapel Hill, Chapel Hill.

Bishop, G., & Fuchs, H. (1984, January 23–25). The self-tracker: A smart optical sensor on silicon. Paper presented atthe Advanced Research in VLSI conference, MassachusettsInstitute of Technology.

BL. (2000). CODA mpx30 Motion Capture System [HTML].B & L Engineering. Retrieved April 27, 2000 from the WorldWide Web: http://www.bleng.com/animation/coda/codamain.htm.

Brown, R. G., & Hwang, P. Y. C. (1992). Introduction toRandom Signals and Applied Kalman Filtering (2nd ed.).New York: John Wiley & Sons, Inc.

Burdea, G., & Coiffet, P. (1994). Virtual Reality Technology(1st ed.). New York: John Wiley & Sons, Inc.

Burton, R. P. (1973). Real-time measurement of multiplethree-dimensional positions. Unpublished doctoral disserta-tion, University of Utah, Salt Lake City.

Burton, R. P., & Sutherland, I. E. (1974). Twinkle Box:Three-dimensional computer-input devices. AFIPS Confer-ence Proceedings, 1974 National Computer Conference, 43,513–520.

Chi, V. L. (1995). Noise model and performance analysis ofoutward-looking optical trackers using lateral effect photo di-odes (Tech. Rep. TR95-012). Chapel Hill: University ofNorth Carolina at Chapel Hill.

Emura, S., & Tachi, S. (1994). Sensor fusion based measure-ment of human head motion. Paper presented at the 3rdIEEE International Workshop on Robot and Human Com-

munication (RO-MAN 94 NAGOYA), Nagoya University,Nagoya, Japan.

Feiner, S., MacIntyre, B., Hollerer, T., & Webster, A. (1997).A touring machine: Prototyping 3D mobile augmented re-ality systems for exploring urban environments. PersonalTechnologies, 1(4), 208–217.

Focus Software. (1995). ZEMAX Optical Design Program Us-er’s Guide, Version 4.5. Tucson, AZ: Focus Software.

Foxlin, E., Harrington, M., & Pfeifer, G. (1998). Constella-tion™: A wide-range wireless motion-tracking system foraugmented reality and virtual set applications. In M. F. Co-hen (Ed.), Computer Graphics (SIGGRAPH 98 ConferenceProceedings ed.) (pp. 371–378). Orlando: ACM Press, Addi-son-Wesley.

Fuchs (Foxlin), E. (1993). Inertial head-tracking (manual).Unpublished master’s thesis, Massachusetts Institute ofTechnology.

Gelb, A. (1974). Applied Optimal Estimation. Cambridge,MA: MIT Press.

Gottschalk, S., & Hughes, J. F. (1993). Autocalibration forvirtual environments tracking hardware. Computer Graphics(SIGGRAPH 93 Conference Proceedings ed.) (pp. 65–72).Anaheim, CA: ACM Press. ACM Press, Addison Wesley.

Hollerer, T., Feiner, S., Terauchi, T., Rashid, G., & Hallaway,D. (1999). Exploring MARS: Developing indoor and out-door user interfaces to a mobile augmented reality system.Computers & Graphics, 23,(6), 779–785.

IGT. (2000). Image Guided Technologies [HTML]. RetrievedSeptember 15, 2000 from the World Wide Web: http://www.imageguided.com/.

Intersense. (2000). Intersense IS-900 [HTML]. RetrievedApril 27, 2000 from the World Wide Web: http://www.isense.com/.

Jacobs, O. L. R. (1993). Introduction to Control Theory (2nded.) New York: Oxford University Press.

Kadaba, M. P., & Stine, R. (2000). Real-Time MovementAnalysis Techniques and Concepts for the New Millennium inSports Medicine [HTML]. Motion Analysis Corporation,Santa Rosa, CA. Retrieved September 15, 2000 from theWorld Wide Web: http://www.motionanalysis.com/appli-cations/movement/rtanalysis.html.

Kalman, R. E. (1960). A new approach to linear filtering andprediction problems. Transaction of the ASME—Journal ofBasic Engineering, 82(series D), 35–45.

Lewis, F. L. (1986). Optimal Estimation with an Introductoryto Stochastic Control Theory. New York: John Wiley & Sons,Inc.

Welch et al. 19

MAC. (2000). HiRes 3D Motion Capture System [HTML].Motion Analysis Corporation. Retrieved September 15,2000 from the World Wide Web: http://www.motionan-alysis.com/applications/movement/gait/3d.html.

Maybeck, P. S. (1979). Stochastic models, estimation, and con-trol (Vol. 141). New York: Academic Press.

Mazuryk, T., & Gervautz, M. (1995). Two-step predictionand image deflection for exact head tracking in virtual envi-ronments. Proceedings of EUROGRAPHICS 95, 14(3), 30–41.

Meyer, K., Applewhite, H., & Biocca, F. (1992). A survey ofposition trackers. Presence, a publication of the Center forResearch in Journalism and Mass Communication, 1(2),173–200.

Mulder, A. (1994a). Human movement tracking technology(Tech. Rep. TR 94-1). School of Kinesiology, Simon FraserUniversity.

———. (1994b, May 8, 1998). Human Movement TrackingTechnology: Resources [HTML]. School of Kinesiology, Si-mon Fraser University. Retrieved September 15, 2000 fromthe World Wide Web: http://www.cs.sfu.ca/people/ResearchStaff/amulder/personal/vmi/HMTT.add.html.

———. (1998, May 8, 1998). Human Movement TrackingTechnology [HTML]. School of Kinesiology, Simon FraserUniversity. Retrieved September 15, 2000 from the WorldWide Web: http://www.cs.sfu.ca/people/ResearchStaff/amulder/personal/vmi/HMTT.pub.html.

Polhemus. (2000). Polhemus [HTML]. Retrieved September15, 2000 from the World Wide Web: http://www.polhemus.com/home.htm.

Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery,B. P. (1990). Numerical Recipes in C: The Art of ScientificComputing (2nd ed.). Cambridge University Press.

Sorenson, H. W. (1970). Least-squares estimation: FromGauss to Kalman. IEEE Spectrum, 7(July), 63–68.

Spohrer, J. (1999a). Information in places. IBM Systems Jour-nal, Pervasive Computing, 38(4).

Spohrer, J. (1999b, June 16). WorldBoard: What Comes Afterthe WWW? [HTML]. Learning Communities Group, ATG,Apple Computer, Inc. Retrieved December 24, 1999 fromthe World Wide Web: http://worldboard.org/pub/spohrer/wbconcept/default.html.

Sutherland, I. E. (1968). A head-mounted three dimensionaldisplay. Proceedings of the 1968 Fall Joint Computer Confer-ence, AFIPS Conference Proceedings (vol. 33, part 1, pp.757–764). Washington, D.C.: Thompson Books.

Thomas, S. W. (1984, December). The Alpha_1 computer-

aided geometric design system in the Unix environment. Paperpresented at the Computer Graphics and Unix Workshop.

UNC Tracker Project. (2000, July 10). Wide-Area Tracking:Navigation Technology for Head-Mounted Displays[HTML]. Retrieved July 18, 2000 from the World WideWeb: http://www.cs.unc.edu/;tracker.

University of Utah Computer Science. (1999). Alpha 1 Publi-cations [HTML]. University of Utah, Department of Com-puter Science. Retrieved May 28, 1999 from the WorldWide Web: http://www.cs.utah.edu/projects/alpha1/a1_publications.html.

Usoh, M., Arthur, K., Whitton, M. C., Bastos, R., Steed, A.,Slater, M., & Brooks, F. P., Jr. (1999). Walking . Walking-in-Place . Flying, in Virtual Environments. In A. Rock-wood (Ed.), Computer Graphics (SIGGRAPH 99 Confer-ence Proceedings ed.) (pp. 359–364). Los Angeles: ACMPress, Addison Wesley.

Van Pabst, J. V. L., & Krekel, P. F. C. (1993, September 20–22). Multi sensor data fusion of points, line segments and sur-face segments in 3D space. Paper presented at the 7th Inter-national Conference on Image Analysis and Processing,Capitolo, Monopoli, Italy.

Wang, J.-F. (1990). A real-time optical 6D tracker for head-mounted display systems. Unpublished doctoral dissertation,University of North Carolina at Chapel Hill, Chapel Hill.

Wang, J.-F., Azuma, R. T., Bishop, G., Chi, V., Eyles, J., &Fuchs, H. (1990, April 16–20). Tracking a head-mounteddisplay in a room-sized environment with head-mounted cam-eras. Paper presented at the SPIE 1990 Technical Sympo-sium on Optical Engineering and Photonics in AerospaceSensing, Orlando, FL.

Wang, J.-F., Chi, V., & Fuchs, H. (1990, March 25–28). Areal-time optical 3D Tracker for head-mounted display sys-tems. Paper presented at the Symposium on Interactive 3DGraphics, Snowbird, UT.

Ward, M., Azuma, R. T., Bennett, R., Gottschalk, S., &Fuchs, H. (1992, March 29–April 1). A demonstrated opti-cal tracker with scalable work area for head-mounted displaysystems. Paper presented at the Symposium on Interactive3D Graphics, Cambridge, MA.

Welch, G. (1995). Hybrid self-tracker: An inertial/optical hy-brid three-dimensional tracking system (Tech. Rep. TR95-048). Chapel Hill: University of North Carolina at ChapelHill, Department of Computer Science.

———. (1996). SCAAT: Incremental tracking with incompleteinformation. Unpublished doctoral dissertation, Universityof North Carolina at Chapel Hill, Chapel Hill.


Welch, G., & Bishop, G. (1995). An introduction to the Kalmanfilter (Tech. Rep. TR95-041). Chapel Hill: University of NorthCarolina at Chapel Hill, Department of Computer Science.

———. (1997). SCAAT: Incremental tracking with incom-plete information. In T. Whitted (Ed.), Computer Graphics(SIGGRAPH 97 Conference Proceedings ed.) (pp. 333–344).Los Angeles: ACM Press, Addison-Wesley.

———. (2000, January 23, 2000). The Kalman Filter[HTML]. University of North Carolina at Chapel Hill. Re-trieved April 29, 2000 from the World Wide Web: http://www.cs.unc.edu/;welch/kalman/index.html.

Welch, G., Bishop, G., Vicci, L., Brumback, S., Keller, K., &Colucci, D. N. (1999). The HiBall tracker: High-perfor-

mance wide-area tracking for virtual and augmented envi-ronments. Proceedings of the ACM Symposium on VirtualReality Software and Technology (pp. 1–11). University Col-lege London, London, United Kingdom (December 20–23): ACM SIGGRAPH, Addison-Wesley.

Welch, P. D. (1967). The use of fast Fourier transformfor the estimation of power spectra: A method basedon time averaging over short, modified periodograms.IEEE Transactions on Audio Electroacoust, AU(15),70–73.

Woltring, H. J. (1974). New possibilities for human motionstudies by real-time light spot position measurement. Biote-lemetry, 1, 132–146.

Welch et al. 21

high-performance wide-area optical trackingwelch/media/pdf/pres101_welch_etal.pdf ·...

Documents