mobile robot localization in indoor environment

Pergamon Pattern Recognition, Vol. 30, No. 1, pp. 99-111, 1997

Pattern Recognition Society. Published by Elsevier Science Ltd Printed in Great Britain. All rights reserved

0031 3203/97 $17.00+.00

P I h S0031-3203(96)00064-7

MOBILE ROBOT LOCALIZATION IN INDOOR ENVIRONMENT 1

HANSYE S. DULIMARTA a and ANIL K. JAIN b'* aDepartment of Computer Science, Bandung Institute of Technology, Bandung 40132, Indonesia bDepartment of Computer Science, Michigan State University, A726 Wells Hall, East Lansing,

MI 48824, U.S.A.

(Received 8 August 1995; in revised form 3 April 1996; received for publication 9 May 1996)

Abstract--This paper describes an indoor mobile robot system with a self-localization capability. This capability can be used, for instance, to recover from failures during navigation. One obvious strategy in determining the self-location is to start the system from a known state and update the current state of the system in accordance with the actions effected by the robot during the course of its activity. However, sensor noise sometimes makes this strategy unreliable. A reasonable approach to overcome this difficulty is to provide feedback and verification mechanisms, for instance, via landmark features as external reference sources for the robot. The proposed system described here uses "ceiling lights" and "door number plates" for self-localization. Experimental results indicate the effectiveness of our approach.

Mobile robots Autonomous navigation Character recognition Self-localization Visual landmarks

l. INTRODUCTION

To effectively navigate in a fairly complex workspace, an autonomous robot system is required to determine the state of its world (including its own location with respect to its immediate surroundings). The ability of a robot to automatically determine the state of the world is known as self-localization. Self-localization capability is critical to reliable performance of an autonomous system. It can be, for instance, used for recovering from

• (1) failures during navigation. In such systems, self- localization plays a crucial role in detecting the occurrence of failures. 2 It is much easier to build a system which can recognize and recover from the failures than a system which never fails at all.

Many different meanings can be associated with the term the state of the world. For instance, a state can be as simple as "the approximate inclination angle of the robot relative to the horizon" or as complicated as "the identity of the object in front of the robot". The choice of the state description is usually dictated by the objectives of the system, the complexity of the environment, sensing and computational resources of the robot. One obvious strategy in determining the self- location is to start the system from a known state and update the current state of the system in accordance with the actions effected by the robot during the course of its activity. However, in practice, sensor noise, artifacts of the processing (numerical inaccuracies, for instance), and discrepancies between the computed and effected robot actions (e.g. due to wheel slippage) often render

* Author to whom correspondence should be addressed. Tel.: (517)353-5150; e-mail: [email protected].

1Supported by NSF grant CDA-8806599. 2In the literature, such detectable failures are referred to as

cognizant failures. (2~

this simple strategy unreliable. More seriously, these errors often accumulate over a period of time to result in absolute chaotic system behavior. A reasonable approach to overcome such practical difficulties is by building feedback and verification mechanisms into the system. One of the simplest and reliable methods of implementing such mechanisms is by providing landmark features of the workspace as external reference sources for the robot. In such systems, the robot uses its sensory data and internal descriptions representing its environment to estimate its location. The estimated location is verified by corresponding the external (landmark) features with the features stored in the robot's internal representation. Some of the mobile robot systems that specifically employ visual landmark recognition are: CARMEL (3) and a number of mobile robots in the AAAI '92 Robot Competition. ca) To a certain extent, this problem is similar to the problem of establishing a correspondence between sensed features and model features in model-based object recognition. ~5) A reliable detection and recognition of the landmark features plays a crucial role in the performance of the mobile robot system. The proposed system described here uses "ceiling fights" and "door number plates" for self-localization.

Which landmark features should be selected for a given autonomous navigation system? Often, this question cannot be answered definitively. We chose landmark features for our system which can be easily detected in real-time using visual sensing• Visual sensing modality has several attractive features: (i) sensors are inexpensive; (ii) the sensor characteristics are well studied; and (iii) visual landmarks are commonly observed in indoor environment. We do not require "installation" of special landmark features (e.g. barcodes) in the existing robot workspace. However,

99

100 H.S. DULIMARTA and A.K. JAIN

robot vision researchers commonly shirk away from using visual features since the resulting image processing techniques to detect and recognize these features are deemed to be computationally demanding. For instance, using a rather primitive hardware, Moravec (6'7~ was able to control his mobile robot to move at an average speed of 0.067 m/min (1 m in 15 min). More recent work on mobile robots that employ sophisticated visual processing in both indoor and outdoor applications include the CMU ALV and NavLab project, (8-1°~ and Mobi robot with its stereo camera. (11~ The most recent work in vision-guided mobile robot is reported in reference (12), where the robot maintains a model of uncertainty and keeps track of the growth of uncertainty as it moves towards the goal. By doing so, the search space used in establishing the correspondence between the robot's internal representation and the landmark features is reduced.

Computer vision techniques will be useful in a robot navigation system only if they are sufficiently simple to be effected in "real-time". Is it feasible to use simple visual landmark features commonly available in the indoor environments for real-time navigation systems? We have demonstrated the feasibility of using simple

and commonly available visual features like "ceiling lights" and "door number plates" for robust indoor navigation. Our landmark feature detection modules illustrate how computational demands of visual processing can often be significantly reduced by prudent design methods. We also determine that despite erroneous individual component behavior, the overall system performance is correct and robust; our system has been tested successfully in more than 100 test runs in a fairly complicated hallway with both static and moving obstacles and variable lighting conditions.

1.1. RoME

Our mobile robot, RoME (Robotic Mobile Experi- ment), is a TRC LABMATE mobile base with 24 ultrasonic sensors and 8 infrared proximity detectors attached to the Proximity Subsystem, two CCD color cameras with 6 mm and 16 mm lenses, two SunVideo image grabbers, and a pan and tilt carousel for mounting the cameras. The onboard processor is a Sun SPARC- stationl running a Solaris 2.3 operating system. This configuration allows us to run RoME autonomously. Fig. 1 shows a picture of ROME.

Fig. I. Front view of ROME, a TRC LABMATE mobile base with 24 ultrasonic sensors, 8 infrared proximity detectors, and two CCD color cameras facing sideways and upward.

Mobile robot localization in indoor environment 101

1.2. Control architecture

As the robot navigation systems become more complex, the management of information and streamlin- ing decision-making become important. Our distributed and modular architecture facilitates system development as well as easier management of information and decision-making. Usually, there are a number of concurrent processes (modules) running on a mobile robot, where each process accomplishes a subtask as part of a "global" task to be achieved by the robot. Some of these concurrently running processes will acquire data from the peripheral devices/sensors available on the robot. We should emphasize that con- currency is an important issue here. For instance, during navigation the robot has to avoid obstacles, detect landmarks, and check its internal sensors (like bumper switch) concurrently. If the system was built in a non- modular fashion, controlling all these activities would be very difficult. It is much easier to develop a separate module for each activity and let these modules run concurrently. However, when there is more than one module competing for resources, a mechanism for controlling resource sharing among these processes is needed. Moreover, since the processes cooperate to achieve a common goal of the robot, they need to interact and exchange information with one another. This situation also dictates the need for a common communication medium for the processes.

The generic components in the proposed architecture can be replaced by any other functionally equivalent modules with relative ease and independence. For instance, the whole system can be ported to another environment by designing appropriate visual landmark feature detection modules. Here, we briefly highlight the features of the architecture; a more complete description

can be found in reference (13). The control architecture is based on a client-server model. Resource access control, resource sharing, information sharing, and process synchronization are delegated to a collection of server modules. We have determined that these facilities are essential and inherently used by all the control architectures for robot navigation system. Thus, our model can be used as a general control architecture. The core of the system consists of a single Data Server and a number of Hardware Servers. The priority scheme used in the architecture allows execution priority of a client module to be controlled regardless of the availability of this facility on the underlying operating system. Hardware Servers can he made small enough to be run on a machine with limited resources whereas the client modules can run on more powerful machines, perhaps at a remote site. The distributed nature of the model allows different hardware platforms and operating systems to be used for controlling the robot.

1.3. Navigation task

Our goal is to command the mobile robot to move from one point to another in the Engineering Building on the MSU campus while avoiding both static and dynamic obstacles. Fig. 2 shows a typical hallway in the MSU Engineering Building. The initial and final locations can be either on the same floor or on different floors of the building. In the latter case, it is necessary for the robot to locate the elevator, enter it, and exit when the destination floor is reached. To determine the actual position of the robot in the map, we have used the ceiling lights and door number plates as landmarks to be recognized by the robot automatically.

We have used the StickRep attributed graph developed by Walsh (1~ as the coarse-level representation of the

Fig. 2. A typical robot workspace; hallways with dynamic obstacles (people, open doors, tables, chairs, etc.).


robot workspace (hallways or corridors). The map contains the locations of doors, corners, ceiling lights, walls, and elevators. An arc in the graph represents either a wall segment or a door. A node stores information about the topological connectivity of the edges. In our experiments, we store two entities in each node: adjacency angle and node name. Physically, nodes in the graph correspond to vertical edges in the hallway such as junctions between wall segments and door jambs. An arc contains information about edge type, length of the edge, attributes of the edge, and the projected position of the ceiling lights in the hallway to the edge. A partial map of the third floor of the MSU Engineering Building where our robot navigated is shown in Fig. 3. The physical size of the entire floor plan is approximately 136×116m 2. The width of the hallways varies in different parts of the building; the narrowest hallway is approximately three times the diameter of our mobile robot.

The rest of this paper is organized as follows. Section 2 describes our approach to landmark detection and recognition. The results of our experiment are given in Section 3. Finally, we conclude the paper by discussing what we have learnt from our experiments and our plans for future research.

2. LANDMARK DETECTION

The overall structure of our indoor navigation system consists of five major components: Path Planner, Navigator, Ceiling Light Tracker, Local Mapper, and Door Number Plate Recognizer. The collaboration of all these modules allows our mobile robot to navigate reliably and robustly.

The Path Planner generates navigation plans from initial and final positions given to the robot. The plans generated by the Path Planner are used by the Navigator to control the robot to reach the final position while avoiding static and dynamic obstacles. The Local Mapper is needed for correcting the accumulated heading and positional errors caused by wheel slippage. The Ceiling Light Tracker counts the number of ceiling lights detected by the robot. Lastly, the Door Number Plate Recognizer recognizes the door numbers (digits) on side walls of the hallways. A more detailed description of the last two modules is given in Sections 2.1-2.3•

2.1. Ceiling light detection

The goal of this operation is to register the robot position with some physical reference points in the robot workspace. We chose to utilize ceiling lights as the

.340.. 340.dbl . 3zlD.corlalO

- ~ - . - . ~ . _ _

w o m e n turning space

m e n •

ceiling light I

men.dbl !

I I

p

346.2

Fig. 3. A partial map of the hallways on the third floor of the MSU Engineering Building. White rectangles represent the ceiling lights in the hallway, while black rectangles represent the door number plate on the

sidewalls.


Fig. 4. Camera setup for detecting ceiling lights (facing up) and door number plates (facing sideways).

reference points because they are easy to detect due to the high contrast between the lights and the ceiling surface.

Our robot is equipped with two color CCD cameras with focal lengths of 6 mm and 16 mm. Fig. 4 shows the imaging setup used in our experiments. The camera facing up (6 mm focal length) is used for detecting ceiling lights and the other camera facing sideways (16 mm focal length) is used for detecting door number plates. With the given setup, the surface area of the ceiling that can be captured by a single image is approximately 1.2x 1.5 m 2. The maximum width of our hallway is approximately 3.66 m and the dimension of a ceiling light is approximately 0 . 6 x l . 2 m 2. So, the camera is able to see the ceiling light even when the robot position is not in the center of the hallway.

For efficiency, the ceiling light tracking module uses only three scan lines from the entire input image (160x120 pixels). These lines are taken at distances associated with 1/4th, 1/2, and 3/4th of the image height. The image is periodically grabbed every 0.5 s by the Camera Server and then the ceiling light detection module requests the aforementioned three scanlines for further processing. The ceiling light detection algorithm simply looks for strong leading edges of the ceiling lights in the three scan lines. An edge is detected if the gradient of a pixel exceeds some threshold. The gradient is computed by taking the first-order difference (as an approximation to the first derivative) between two adjacent pixels in a scanline. A typical threshold value used for the 256-graylevel input images is 50. Fig. 5 shows an image of a ceiling light as seen by the robot. The three lines used in the detection are labeled A, B, and C. Fig. 6 shows a high-level algorithm for detecting the presence of ceiling lights in an input image.

In order to avoid incorrect detections (false positives), the output of the ceiling light tracker is then verified from the information stored in the internal hallway map. Whenever a "ceiling light" is detected in the image, the module checks the distance the robot has traveled since the last detected light. If this distance agrees with the actual distance between the two lights computed from the stored map, then the response is valid. Otherwise, the detection of ceiling light is deemed to be invalid. Using this approach, our mobile robot is able to roughly estimate its actual position within the stored map. False negatives can be easily detected as well by comparing the distance traveled by the robot and the distance between two ceiling lights obtained from the stored map. If the robot has traveled beyond the distance calculated from the map and no image is detected yet, then a false negative has occurred.

Using the algorithm shown in Fig. 6, the time needed for grabbing the three scan lines and detecting the edges in them is approximately 50 ms. The distance between two successive ceiling lights is approximately 6 m. Even if our robot cruises at its maximum speed (1 m/s), it will take 6 s to travel between two ceiling lights. Thus, the above algorithm is able to track the ceiling lights in real- time. The average speed of the robot in our experiments is approximately 0.3--0.4 m/s.

2.2. Door number plate detection

In a typical office building, door plates containing office numbers are commonly observed landmarks. Moreover, the number of these plates is usually higher than the number of ceiling lights. Our robot uses this structured information for both self-localization and verification purposes. Ceiling lights serve only as "coarse" reference points. The uncertainty of the robot


Fig. 5. An image of a ceiling light and the three lines used by the edge detection algorithm.

for i := 0 To NumberOfLines do begin

GameraGrabRow ( i,ScanLine ); for j := ColStart To ColEnd do

begin grad := ScanLine [j + 1]- ScanLine[j]; if grad < -50 t h e n

cnt := 1; exit loop;

fi end

end end

Fig. 6. Algorithm for detecting ceiling lights via edge detection in the three lines shown in Fig. 5.

location increases as the robot passes a ceiling light before it comes to the next ceiling light. This uncertainty can be reduced if some other landmarks are available between any two ceiling lights. In our case, there are approximately 2-3 door number plates between any two successive ceiling lights.

In our domain, door number plates are located at approximately the same height above the floor. There- fore, the height of the side-looking camera is adjusted so that the plates are located near the middle row of the image. Moreover, all the number plates in our domain contain white letters on a black background. In order to "highlight" the number plates, the input image is preprocessed as follows: (i) compute the "negative" image, (ii) stretch the graylevel values to increase the contrast, and (iii) threshold the graylevel image to obtain a binary image. Fig. 7 shows the intermediate results of each processing step.

For a quick response, our plate detection algorithm searches for a closed contour in a thresholded intensity image and calculates the curvature of each contour point. The bounding box of the door number plate is

determined by identifying contour points with a nigh curvature. This bounding box can then be used to further locate the bounding box containing only the door numbers on the plates.

In the binary image, the algorithm searches for dark- to-bright transitions, first along the middle scan line, and continues searching in the upper half of the input image if such a transition could not be found. In binary images, this dark-to-bright transition corresponds to a pixel on the border of a region (i.e. plate). Once such a transition is found, the algorithm will trace the border and store the row and column positions of the pixel in a linked list. At the same time, the chain code of the curve that is constructed by these pixels is determined. If a closed contour is found, then it is stored in a linked list of contours for further consideration. Otherwise, the contour is discarded. For each contour in the linked list, the "curvature" of each pixel on the contour is determined and corner points can be located from the curvature values. The detected corner points are then used for determining the bounding box associated with the contour. We have used the corner detection algorithm from chain-code representation of curves proposed in reference (14). If there are several bounding boxes found, overlapping ones are identified and merged. Finally, the size of the bounding box is used for rejecting or accepting the box as a candidate for door number plate. Fig. 8 shows the high-level algorithm for plate localization.

In order to avoid multiple detection of the same plate from several different positions and orientations of the mobile robot, we need to determine whether any two image frames overlap. Fig. 9 shows two overlapping image frames captured by the robot as it was navigating in the hallway. For this purpose, the geometries shown in Figs 10 and 11 have been developed. Given the heading angle of the robot, Fig. 10 shows how the length of left- and right-projections of the field of view


It) (dl

i (,,)

Fig. 7. Intermediate results of door number plate detection. (a) input image (320×240), (b) "negative" image, (c) contrast enhancement, (d) binary image obtained after thresholding, and (e) bounding box of the plate. The input image is taken at a distance of approximately 1.5 m; the physical width of the plate is approximately 20 cm. With a 16 mm lens, the camera spans a width of approximately 60 cm on the wall

(three times the width of the plate).

r := huageHeight/2; repeat

CollectContour( r , L); MergeBoundingBox (L); PurgelnconsistentBoxes( L ); r := r - DecrementFactor;

until r < 0 or a plate is found; {L contains a list of valid bounding boxes}

procedure CollectContour (r, L) begin

Find y such that a black-to-white transition occurs at (r, y); Trace the contour from (r, y), and

store atl the points in linked list L; if a closed contour is found then

Compute curvature points of each plxel in L; Determine the bounding box of L;

fi end

Fig. 8. Algorithm for plate localization in input graylevel images.

of the camera to the wall are computed. In Fig. 10,/3 is the angle between the optical axis and the normal (measured from the wall normal); a is the angle between the fight ray of the camera field of view and the wall normal (measured from the fight ray); d is the sonar reading of the sensor which is approximately perpendicular to the wall; l and r are the distances

between the point of projection of the robot center and point of projection of the left and fight rays, respec- tively; l<0 when / 3 < - 3 0 °, and r<0 when /3>30 °. The following equations hold for the above entities:

1 = dtan(30° +/3) , r = dtan(30° - / 3 ) .

Since the above equations are valid only when 1/31<60 °, and the error in the computation is very large when 1/31~60 °, the door number plate detector performs no action when 1/31_>50 °.

The amount of overlap between any two successive image frames can be determined using the geometry shown in Fig. 11. In the figure, the first image frame is captured when the robot is at position (x~,yl). The distance between the two wall normals is D. An overlap occurs when D<(rl +12); the quantity ( r l+12-D) denotes the length of the overlapping segment. This length also determines the starting column number for locating the number plate in the second image frame. For a fast computation of D, this value is approximated by the distance between (xl,yl) and (x2,Y2).

2.3. Dig i t recognit ion

Once the number plate is detected by the algorithm given in Fig. 8, the digits corresponding to the door


Fig. 9. Two image frames containing the same door number plate.

~,/optical axis

camera field of ~ . ~ ~ ~ ~"~/

J ~ / wall

Fig. 10. Number plate detection geometry.

c d a

D

Fig. 11. Geometry of two overlapping snapshots.

number are then located within the plate. In our domain, the plates come in two different sizes which can be distinguished by their aspect ratios, The aspect ratio of plates of type 1 is approximately 4, while for type 2 it is approximately 2.5. The digits in plates of type 1 are located in the left-half of the plate, while in type 2 the digits are in the upper-left quadrant of the plate [see Fig. 12(a) and (b)]. This information enables us to correctly extract the door number in any plate. Fig. 12 shows images of three door number plates where the door numbers have been successfully extracted.

The door numbers are recognized using a simple and effective template matching scheme. Our matching strategy is as follows. Let {M1,M2 . . . . . Mk} be the set of stored binary images of door number plates in the current vicinity of the robot. Let I be the binary image of the door number detected by the mobile robot. Every image pair (l,Mi) is normalized to have the same size. If I is an image of size (hl,WO and Mi is of size (h2,w2), then the new dimension of the two images is (max(hl,hz),max(wx,w2)). This operation results in a linear stretching of the two images. By doing so, the detailed information contained in the two images is retained. For recognition purposes, we use the following matching coefficient between I and Mi:

Ci = (moo + m l l - mol Z ml°'~ × 100, \moo + mll + toOl + mloJ

where rnij is the number of matches between pixels with label i in one image and pixels with label j in the other image. In our case, label 0 refers to "black" pixels and label 1 refers to "white" pixels. The binary image I is recognized as image Mr when

C r = m a x G . i:l,...,k

Since the ceiling tracking module provides a coarse localization information, we can limit the door number


(t,) i

(,9

Fig. 12. Localization of number plates and door number extraction in three different input images. Left column: input image (320×240 pixels); right column: detection results (average size of the binary door number image is 70×30

pixels).

matching to only a small number of stored images. In our case, there are approximately 2-3 door number plates between any two successive ceiling lights. Therefore, if the robot is in the vicinity of ceiling light Lk, then the matching can be performed only to the first 5 closest door number plates to Lk.

3. EXPERIMENTAL RESULTS

We have conducted a number of experiments on our mobile robot. The robot navigated autonomously on the third floor of the MSU Engineering Building while capturing images from the two cameras. The average speed of the robot in our experiments is approximately 0.3-0.4 m/s. In this section, we present the result of both the ceiling light tracking and door number recognition.

After testing it on more than 100 autonomous runs involving more than I000 ceiling lights, we have determined that our ceiling light tracking module performs very well. The ceiling light is detected by locating edges in only three scanlines. This processing takes only about 50 ms allowing the robot to have the capability of real-time ceiling light tracking. In some portions of our building, where the sunlight enters the hallway through windows, there is not enough contrast between the ceiling light and its surrounding. Never- theless, by combining the information in the stored map with the output of the visual processing, the ceiling light

tracking module is able to locate the ceiling lights correctly. We have also tested the module when some of the ceiling lights were off. Again, the approach we use proves to be reliable and robust.

Using the algorithm shown in Fig. 6, the time needed for detecting the presence of a ceiling light in an input image is approximately 50 ms. The distance between two successive ceiling lights is approximately 6 m. Even if our robot cruises at its maximum speed (1 m/s), it will take 6 s to travel between two ceiling lights. Thus, the algorithm is able to track the ceiling lights in real-time.

Now, we describe the results of door number plate recognition. Fig. 12 shows three of the input images that were captured while the robot was moving in the hallway. The image was continuously grabbed from the camera at a rate of 4 frames per second while the robot is moving at a speed of approximately 250 mm/s. From over 500 frames captured, approximately 10% of them contained a door number plate. The sample input images shown in Fig. 12 represent all types of plates found in our environment.

Fig. 13 shows a partial model database of the door plate numbers and Fig. 14 shows a partial set of test data extracted from the images captured by the robot while it was navigating. The average size of each binary image (captured door number) is approximately 70x30. The binary image in the model database was created by the following procedure: (i) position the robot in the center of the hallway with the side-looking camera facing the door number plate, (ii) grab a graylevel image of size 320 × 240, (iii) locate and extract the number plate using the algorithm shown in Fig. 8, and (iv) manually select a

=Bi=i Fig. 13. Number plate model database. The size of each image

is approximately 70×30 pixels.

E Fig. 14. Number plate test database. The last row contains an image of an incomplete "349" and five "unknown" plates due

to false positives.


o

(D

o

8

a

8

a ~

"O

&

0~

O

o r,.)

o'x i


threshold value that will give the best binary image of the door number.

Table 1 shows the matching results when each test image was matched to every image in the database. Since during the navigation the camera used for door number plate recognition faces only the left-side wall, the matching can be performed to either the odd-

numbered or even-numbered images only. The table shows the result of matching the odd-numbered images. Table 2 shows the matching scores of the false positives

(the last six images in Fig. 14) to the images in the model database. All entries in this table are less than 50. In these two tables, the database images are shown in

the leftmost column of the table. The matching results in the two tables show that false positive detections can be eliminated by setting a threshold on the matching coefficients. By observing all the entries in Table 1, we

can see that the matching coefficient must be at least 50

for accepting the input binary image as a valid door number plate.

On a SPARCstationl, the time needed to compute the matching coefficient for a pair of images is approximately 83 ms. The average time needed to grab the input

image, locating the plate, and extracting the digits on the plate is approximately 960 ms. In our experiments, the time needed for the robot to travel between two

successive door number plates is at least 1.5 s. There- fore, we can also perform the door number plate matching in real-time.

Using the ceiling lights as the only landmarks in navigation, our mobile robot is able to locate its final destination with a maximum error of 1 m. When both ceiling lights and door number plates are used for landmarks, the error is reduced to approximately 0.6 m.

Table 2. Matching score for the false positives. The last six columns correspond to the images in the last row of Fig. 14. All the scores are less than 50

339-db 8.94 10.08 19.85 14.47 8.03 9.62 341-db 18.71 24.85 18.26 18.33 15.38 12.58 343-db 11.52 18.26 10.30 18.11 13.64 16.44 345-db 7.88 16.44 17.12 19.17 19.70 18.71 347-db 15.23 17.12 22.20 24.09 16.89 19.70 349-db 8.64 8.71 9.55 9.02 10.30 9.47 350-db -0.15 16.44 20.00 16.59 13.64 20.38 351-db 12.65 11.82 25.38 20.15 15.53 19.55 352-db 4.92 12.12 25.38 15.00 11.14 13.03 353-db 6.74 16.21 23.56 13.79 9.17 14.09 354-db 7.50 17.88 39.62 20.00 15.83 21.67 355-db 0.23 14.24 29.77 14.09 15.68 15.6l 356-db -0.76 11.29 22.58 5.23 11.52 10.68 357-db 12.95 13.94 35.53 20.30 13.11 17.12 358-db 0.68 10.00 15.23 5.91 9.77 9.24 359-db 4.77 8.33 22.95 11.36 7.05 10.45 360-db 1.59 4.55 21.59 7.12 9.92 8.18 361-db 10.30 11.74 21.67 19.02 13.48 15.98 363-db 10.30 13.41 20.30 10.98 10.76 10.08 364-db 8.56 12.58 35.23 14.24 14.62 14.55 367-db 5.76 10.68 30.15 13.11 10.30 12.35 368-db 5.76 6.59 18.48 0.53 10.15 2.65 369-db 3.03 0.98 20.00 5.08 7.27 5.83 371-db 13.26 20.15 19.92 19.70 11.89 20.00 372-db 8.33 8.86 11.97 14.32 5.91 4.62 373-db 10.68 16.82 15.08 16.67 11.14 8.33 375-db 7.12 17.05 20.15 19.77 18.33 14.02 376-db 4.32 18.33 19.92 16.67 14.47 13.03 377-db 13.33 21.44 27.58 27.20 14.09 20.23 379-db 8.03 11.14 17.73 15.53 10.30 10.08 380-db 1.67 10.23 19.09 11.74 12.27 14.17 381-db 8.86 5.45 17.80 12.12 8.56 10.76 382-db 1.82 7.35 20.91 7.95 6.36 8.56 384-db 6.89 8.94 35.38 10.00 11.89 13.03 385-db 3.11 8.48 22.35 6.67 10.53 9.24 387-db 5.98 14.24 29.17 17.27 10.38 16.67 388-db 3.33 6.74 17.12 3.41 8.03 7.35 389-db 4.77 2.42 19.62 5.61 5.83 6.52 391-db 8.11 7.42 22.95 14.39 10.08 15.30 393-db 5.53 19.39 13.86 11.82 9.02 10.76 394-db 8.56 16.06 31.14 12.27 13.41 13.48 395-db 6.44 24.39 23.56 20.00 16.59 19.70


Despite the simple technique used in our digit recognition method, the results of our experiments show that the technique is adequate for giving the self- localization capability. Although a number of other character recognition techniques are available, we did not attempt to compare ours with those techniques since our objective is not to come up with the "bes t" character recognition method, but to demonstrate a procedure for self-localization.

4. SUMMARY AND CONCLUSIONS

In this paper, we have presented our experimental results on indoor mobile robot navigation using ceiling lights and door number plates as landmarks. The entire navigation system is implemented using a distributed control architecture that employs a client-server model. To obtain a quicker response from the vision modules, we have employed simple image processing techniques. Using this approach we are able to achieve real-time processing. For instance, the ceiling light detection module processes only three scanlines from the entire input image. The method used for detecting the presence of ceiling lights involves computation of the first-order gradients and detection of strong gradient magnitudes allowing us a total processing time of only 50 ms. This allows us to do real-time ceiling light tracking.

The number plate detection module employs a region boundary tracing algorithm, chain-coding, and comer detection from the chain-codes. Once the number plate is located in the input image, the digits on the plate are extracted and then recognized by matching them with the binary digits stored in the model database. The average time needed for grabbing the input image, locating the number plate, extracting the digits in the plate, and matching it to the model database is approximately 1.3 s. This also allows us to do real-time door number plate recognition. Although individual components in our system may occasionally fail, the overall performance of the system has been consistently dependable. Our experimental results indeed show that the localization accuracy of our mobile robot is

improved (from 1 m to 0.6 m) when both ceiling lights and door number plates are used for landmarks.

REFERENCES

1. Stephen J. Walsh, Indoor robot navigation using a symbolic landmark map, Ph.D. Thesis, Department of Computer science, Michigan State University, East Lansing, Michi- gan (1992).

2. Illah Nourbakhsh et al., The winning robots from the 1993 robot competition, AI Magazine 14(4), 51-62 (1993).

3. David Kortenkamp, Marcus Huber, Charles Cohen, Ulrich Raschke, Clint Bidlack, Clare Bates Congdon, Frank Koss and Terry Weymouth Integrated mobile-robot design, 1EEE Expert 8(4), 61-73 (1993).

4. Thomas Dean and R. Peter Bonasso, 1992 AAAI robot exhibition and competition, AI Magazine 14(4), 35-48 (1993).

5. Anil K. Jain and Patrick J. Flynn, eds, Three-Dimensional Object Recognition System, Vol. 1. Elsevier, Amsterdam (1993).

6. Hans P. Moravec, Robot Rover l/isual Navigation. UMI Research Press, Ann Arbor, Michigan (1981).

7. Hans P. Moravec, The Stanford cart and the CMU rover, Proc. IEEE 71(7), 872-884 (1983).

8. Richard Wallace, Anthony Stentz, Charles Thorpe, Hans Moravec, William Whittaker and Takeo Kanade, First results in robot road-following, in Proc. Ninth Intl. Joint Conf. Artificial Intelligence, pp. 1089-1095. Los Angeles, California (1985).

9. C. Thorpe, M. Herbert, T. Kanade and S. Shafer Toward autonomous driving: The CMU NavLab: II. Architecture and systems, 1EEE Expert 6(4), 31-42 (1991).

10. C. Thorpe, M. Herbert, T. Kanade and S. Shafer Toward autonomous driving: The CMU NavLab: I. Perception, IEEE Expert 6(4), 44-52 (1991).

l 1. David J. Kriegman, Ernst Triendl and Thomas O. Binford Stereo vision and navigation in buildings for mobile robots, IEEE Trans. Robotics Automat. 5(6), 792-803 (1989).

12. Alto Kosaka and Avinash C. Kak, Fast vision-guided mobile robot navigation using model-based reasoning and prediction of uncertainties, CVGIP: Image understanding 56(3), 271-329 (1992).

13. Hansye S. Dulimarta, Client-Server control architecture for robot navigation, Ph.D. Thesis, Department of Com- puter Science, Michigan State University, East Lansing, Michigan (February 1994).

14. H. Lynn Beus and Steven S. H. Tiu, An improved comer detection algorithm based on chain-coded plain curves, Pattern Recognition 20(3), 291-296 (1987).

About the Author--HANS DULIMARTA is an Associate Professor in the Department of lnformatics at the Institute of Technology, Bandung, Indonesia. He received his Ph.D. degree in Computer Science at Michigan State University in 1994. Dr Dulimarta's research interests include computer vision, pattern recognition and mobile robotics.

About the A u t h o r - - ANIL JAIN is a University Distinguished Professor and Chairman of the Department of Computer Science at Michigan State University. His current research interests are computer vision, image processing, neural networks and pattern recognition. He has made significant contributions and published a large number of papers on the following topics: statistical pattern recognition, exploratory pattern analysis, neural networks, Markov random fields, texture analysis, interpretation of range images, and 3D object recognition. Several of his papers have been reprinted in edited volumes on image processing and pattern recognition. He received the best paper awards in 1987 and 1991, and received certificates for outstanding contributions in 1976, 1979 and 1992 from the Pattern Recognition Society. Dr Jain served as the Editor-in-Chief of the IEEE Transactions on Pattern Analysis and Machine Intelligence (1991-1994), and currently serves on the editorial boards of Pattern Recognition journal, Pattern Recognition Letters, Journal of Mathematical Imaging, Journal


of Applied Intelligence and IEEE Transactions of Neural Networks. He is the co-author of the book Algorithms for Clustering Data (Prentice-Hall, 1988), has edited the book Real-Time Object Measurement and Classification (Springer-Verlag, 1988), and has co-edited the books, Analysis and Interpretation of Range Image (Springer-Vedag, 1989), Neural Networks and Statistical Pattern Recognition (North-Holland, 1991), Markov Random Fields: Theory and Applications (Academic Press, 1993), and 3D Object Recognition (Elsevier, 1993). Dr Jain is a Fellow of the IEEE. He was the Co-General Chairman of the llth International Conference on Pattern Recognition, Hague (1992), General Chairman of the IEEE Workshop on Interpretation of 3D Scenes, Austin (1989), Director of the NATO Advanced Research Workshop on Real-time Object Measurement and Classification, Maratea (1987), and co-directed NSF supported Workshop on "Future Research Directions in Computer Vision", Maui (1991), "Theory and Application of Markov Random Fields", San Diego (1989) and "Range Image Understanding", East Lansing (1988). Dr Jain was a member of the IEEE Publication Board (1988-90) and served as the Distinguished Visitor of the IEEE Computer Society (1988-90). He is currently a Distinguished Lecturer of the IEEE Computer Society's Asia-Pacific Lectureship Program.

mobile robot localization in indoor environment

Documents