discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfnetwork (wsn)...

13
Discovery, localization and recognition of smart objects by a mobile robot E. Menegatti M. Danieletto, M. Mina, A. Pretto, A. Bardella, A. Zanella, P. Zanuttigh ? University of Padova, Dep. of Information Engineering (DEI), via Gradenigo 6/B, 35131 Padova, Italy phone: +39 049 827 7840 fax: +39 049 827 7826 mailto: [email protected] Abstract. This paper presents a robotic system that exploits Wireless Sensor Network (WSN) technologies for implementing an ambient intelligence scenario. We address the problems of robot object discovery, localization, and recognition in a fully distributed way. We propose to embed some memory, some computa- tional power, and some communication capability in the objects, by attaching a WSN mote to each object. We called the union of an object and of a mote, a smart object. The robot does not have any information on the number nor on the kind of objects in the environment. The robot discovers the objects through the radio fre- quency communication provided by the WSN motes. The robot roughly locates the motes by performing a range-only SLAM algorithm based on the RSSI-range measurements. A more precise localization and recognition step is performed by processing images acquired by the camera installed on the robot and matching the descriptors extracted from these images with those transmitted by the motes. Experiments with eight smart objects in a cluttered office environment with many dummy objects are reported. The robot was able to correctly locate the motes, to navigate toward them and to correctly recognize the smart objects. Object-Recognition, SIFT, Wireless Sensor Network, mobile robot, ambient intelli- gence. 1 Introduction Several research and industrial projects are focusing on service and personal robotics. At the beginning, most of the efforts were driven by the idea of developing robots with large knowledge and very skilled acting in passive environments. In the last years, a new paradigm gained momentum: the idea of simple robots connected among them and connected to distributed sensors and distributed actuators. Depending on the research field, this paradigm is known also as: ubiquitous robotics or networked robotics or am- bient intelligence [1, 9, 6, 8, 15] . In this approach intelligent and complex behaviors are achieved through the cooperation of many simple robots and sensors. ? This work was partially supported by the University of Padua under the project ”RAMSES2” and partially supported by Fondazione Cassa di Risparmio Padova e Rovigo under the project ”A large scale wireless sensor network for pervasive city-wide ambient intelligence”.

Upload: others

Post on 24-Jun-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

Discovery, localization and recognitionof smart objects by a mobile robot

E. Menegatti M. Danieletto, M. Mina, A. Pretto, A. Bardella, A. Zanella, P. Zanuttigh ?

University of Padova, Dep. of Information Engineering (DEI), via Gradenigo 6/B, 35131Padova, Italy phone: +39 049 827 7840 fax: +39 049 827 7826 mailto:

[email protected]

Abstract. This paper presents a robotic system that exploits Wireless SensorNetwork (WSN) technologies for implementing an ambient intelligence scenario.We address the problems of robot object discovery, localization, and recognitionin a fully distributed way. We propose to embed some memory, some computa-tional power, and some communication capability in the objects, by attaching aWSN mote to each object. We called the union of an object and of a mote, a smartobject. The robot does not have any information on the number nor on the kind ofobjects in the environment. The robot discovers the objects through the radio fre-quency communication provided by the WSN motes. The robot roughly locatesthe motes by performing a range-only SLAM algorithm based on the RSSI-rangemeasurements. A more precise localization and recognition step is performed byprocessing images acquired by the camera installed on the robot and matchingthe descriptors extracted from these images with those transmitted by the motes.Experiments with eight smart objects in a cluttered office environment with manydummy objects are reported. The robot was able to correctly locate the motes, tonavigate toward them and to correctly recognize the smart objects.

Object-Recognition, SIFT, Wireless Sensor Network, mobile robot, ambient intelli-gence.

1 Introduction

Several research and industrial projects are focusing on service and personal robotics.At the beginning, most of the efforts were driven by the idea of developing robots withlarge knowledge and very skilled acting in passive environments. In the last years, anew paradigm gained momentum: the idea of simple robots connected among them andconnected to distributed sensors and distributed actuators. Depending on the researchfield, this paradigm is known also as: ubiquitous robotics or networked robotics or am-bient intelligence [1, 9, 6, 8, 15] . In this approach intelligent and complex behaviors areachieved through the cooperation of many simple robots and sensors.

? This work was partially supported by the University of Padua under the project ”RAMSES2”and partially supported by Fondazione Cassa di Risparmio Padova e Rovigo under the project”A large scale wireless sensor network for pervasive city-wide ambient intelligence”.

Page 2: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

Fig. 1. (Left) An overview of the experiment setup: the robot and the movable smart objects.(Right) The modified Pioneer 2 robot platform. On the top of the robot is a camera with a 90 deg.field of view. On the camera support is latched a TMote Sky to communicate with the WirelessSensor Network constituted by the smart objects.

In this paper, we focus on the problem of object location and recognition by anautonomous mobile robot. A personal or a service robot will need to interact with andmanipulate objects in our houses or in office environments. Thus, it needs to be able torecognize the different objects and locate them in the environment. Several approacheshave been proposed for object detection and recognition, most of which perform visualrecognition of the object by using a dataset of object-models stored on the robot [3, 2,18]. These approaches, however, require the robot to know the object models before-hand, thus limiting the applicability and the scalability of the solution. We propose asystem that overcomes these limits by attaching to the objects small wireless devices(called motes) that provide communication, processing, and sensing capabilities. Anobject equipped with such a device, hence, acquires some intelligence, becoming a so-called smart object. In our solution, the robot does not have any prior knowledge aboutthe shapes of the objects, their positions and their number. Therefore, we can deal withany object from the simplest to the most complex. We can deal with movable (or mov-ing) objects (like the ones in Fig. 1 or chairs, plants, other robots, etc.). We can deal witha very large variety and multiplicity of objects, because of the distributed approach and,most important, fact that the robot does not need to know the objects beforehand.

Unfortunately, to achieve this flexibility, it is not enough to use localization ap-proaches based on the [signal] strength of the radio signals transmitted by the motes.It is well known that the most common localization techniques based on radio signalstrength (RSS) do not provide a precise geographical location of the motes [19]. This isparticularly true in indoor environment, because of interference and multiple reflectionsof the radio signals. Currently, the best algorithm can locate the motes with respect tothe robot with a precision of a meter or so (relying only on standard radio signals emit-ted by standard motes). This is not enough for a reliable localization of the object basedonly on the object ID. We hence propose to store locally, in the memory of the smartobject, the appearance of the object. Thus, the object can transmit its appearance to therobot when the robot requests to interact with that object. The robot can create a map

Page 3: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

with a coarse localization of the smart objects of interest obtained via radio and movetoward them. When the robot is in the room where the object of interest is located, it canseek for its appearance in its camera image. In this way, the robot can locate much moreprecisely the object and navigate toward it in order to perform the desired interaction.

The application scenarios for this type of systems include industrial and home ap-plications. For instance, the system may be used in room storages to make it possiblefor robots to find and retrieve objects on request. The system does not require to storein the robot any information concerning the position or the actual appearance of theobjects. Therefore, the robot may be instructed to find an object and the robot will beable to recognize that object in a group of others by using its vision capabilities in con-junction with the information provided by the object itself upon request. Moreover, theproposed approach is scalable also in the number of robots. Indeed, every time a smartobject transmits its appearance, more than one robot could listen to the description.Distributing the knowledge in the environment (i.e., in the objects) makes it possible toseamlessly work with one single robot or with a team of cooperating robot. In a homescenario, the same system can be used to tied-up the play-room of kids. At the end ofthe day, a robot can locate all toys in the room and store them in the right containers.Both the toys and the containers are smart objects in the sense explained above.

One of the principal sources of inspiration for our work was the PEIS-Ecology(i.e., Ecology of Physically Embedded Intelligent Systems) approach [15]. In fact, in thePEIS framework a large number of sensors are attached to objects and can transmit tothe robot useful information [16] (i.e., the position of objects, the light in the ambient,etc.).

This framework has been already described in rather general terms in our previousseminar works [12] [?] . This paper introduces into our previous framework severalnew elements that increase the reliability and the flexibility of the system. In this workwe presented a unified system for mote localization and object recognition based onthe interaction between autonomous mobile robot and WSN Technology. In Section 2,we explain how we implemented the smart objects. In Section 3 we present the newimplementation of a range-only SLAM performed with measures of distance obtainedfrom the RSSI of the messages sent by the motes in which the initialization problem ofour previous work is solved by introducing a particle filter for implementing a delayedinitialization of the Extended Kalman Filter. In Section 4, we present the new objectappearance descriptors we adopted to solve the problem of motion blur in the cameraimage that cased missed matched in the previous implementation of the recognitionmodule. In Section 5, we present experiments in which the robot performs all the stepsto discover and approach the smart objects. In particular we show new preliminaryexperiments on the robustness to motion blur and on the delayed initialization based onthe particle filter.

2 Smart objects

We propose the use of WSN technologies for implementing the smart objects. Thisallows to relay on hardware and software already developed by the WSN communitydecreasing the cost of the smart tags to be attached at the objects and exploiting already

Page 4: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

found solutions for energy saving and efficient communications. The smart objects andthe robot will also constitute a WSN that can perform (and transmit) measures bothon the internal state of the objects (like object temperature, level of filling, inclination,weigh, deterioration, etc) and on the state of the environment (temperature, pollution,humidity, and so on). Moreover, smart objects may establish a multi-hop communica-tion network to relay messages to the robot in case direct communication is not avail-able. As written above, each mote attached to the object enables the robot to reliablyrecognize and locate the object in the environment and the proposed approach is scal-able to the number of objects. This is achieved by storing on the motes a compactdescription of the appearance of the object. We choose a description that can fit in thesmall memory on board of the motes and that could be created by the manufacturer ofthe object. This is important in our idea of a totally distributed approach. It is not therobot programmer or robot installer that create the descriptions of the smart objects. Itis the object manufacturer that store the object appearance in the mote, so the objectmanufacturer and the robot provider do not need to interact, but just to follow the samestandard.

2.1 The communication protocol

The smart objects and the robot are equipped with motes that shares the same com-munication protocol. In our experiments we used Berkeley TmoteSky which are IEEE802.15.4 compliant. The communication protocol was implemented in NesC exploitingthe TinyOS primitives.

In order to allow the unambiguous identification of each object, each mote is iden-tified by a unique ID, which is also used for labeling each packet transmitted by themote. The communication protocol is connectionless and unreliable, i.e., no acknowl-edgement is required to confirm the correct reception of descriptor data packets. Thischoice is motivated by energy saving and packet traffic management considerations.

3 Object discovery and localization

As said, the number of smart objects and their position in the environment is unknownto the robot. The robot needs to discover and locate the objects in the environment. Therobot can discover the smart objects when their motes are in the communication range ofthe robot. At the beginning, when the robot enters the environment, it sends a HELLOmessages and waits for HELLO packets from the motes attached to the objects. Therobot reads the ID of the replying objects (i.e., of the smart objects in the communicationrange) and starts a SLAM procedure (i.e., the Simultaneous Localization of the robotAnd of the Mapping of the wireless nodes in the environment).

The robot can estimate the distance to nearby nodes of the WSN by measuring theReceived Signal Strength Indicator (RSSI) of the received radio messages. Since theRSSI can be calculated directly by most radio transceiver, it allows for a (rough) rangeestimation without the need for specialized hardware, with an advantage in terms ofdevice’s cost and size. Another advantage of RSSI-based ranging is that it does not re-quire the node to be in line of sight with the robot, since the RF signal passes through

Page 5: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

obstacles as persons, furniture or even walls. Unfortunately, the RSSI measure is verynoisy, especially in an indoor environment due to interference and reflections (mul-tipath). However, our approach proved to be successful in such a challenging envi-ronment. We adopted an Extended Kalman Filter SLAM algorithm to integrate RSSImeasurements from the different nodes over time, while the robot moves in the envi-ronment. In a previous work [12], we realized that the main source of error in the motelocalization is the initial guess of the motes position, which is necessary to initialize theExtended Kalman Filter. In this paper, after introducing the RSSI-based SLAM method,we extended it with a new mote position initialization technique that uses particle filtersand resulted to be much more accurate of the initialization based on trilateration, whichwas used in previous works.

3.1 RSSI-based range-only SLAM

To estimate the distance between two motes by measuring the RSSI, it is required toknow the signal propagation law. Unfortunately, the characteristics of the radio channelare largely affected by the environment and are difficult to capture by a unique model.In this work, we refer to the standard path-loss model, which is generally consideredreasonable for a large number of scenarios [7]. Accordingly, we estimate the distancedi of a node whose radio signal is received with power Pi as:

di = d0 · 10A−Pi10np (1)

where

– Pi radio signal power [dBm]– d0 is the far-field reference distance (d0 = 1.071467 m)– A is the nominal received power at the reference distance d0 (A = −45 dBm).– np is the path loss coefficient (np = 2)

The power measure P[dBm] can be obtained through a linear conversion of the RSSImeasurements returned by the node transceiver. We observe that the distance estimategiven by 1 will be affected by an error due to the noisy RSSI readings. This error istaken into account in the filtering matrices of the KF, as detailed below.

EKF SLAM is a well-known technique that recursively solves the online SLAMproblem where the map is feature-based. It estimates at the same time the position ofthe robot and the position of the features: in our case the features are the motes. Thus,the state vector of the system that should be estimated is:

qk =[xk, yk, θk, xm1 , ym1 , . . . , xmn

, ymn

]>(2)

where [xk, yk, θk]> is the robot position at time k and [xmi , ymi ]> is the location of the

i− th motes.During the EKF SLAM prediction phase, the state (i.e., its mean and covariance) attime k is updated according to the motion model:

µ−k = f (uk, µk−1) (3)

Σ−k = AkΣk−1A>k +BkQkB

>k (4)

Page 6: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

where µk−1 and Σk−1 are the mean and covariance of the state q at time k − 1.When the robot receives a new radio message from the ith node, it reads the RSSI

value and estimates the distance dki using (1) we can perform the correction phase ofthe EKF. Given h the function that maps the current state in an expected measurementdk,i given the mote mi, we can update the mean and covariance of the state:

µk = µ−k +Kk

(dk,i − h

(µ−k , i

))(5)

Σk = (I −KkHk)Σ−t (6)

where Hk is the Jacobian of the function h and Kk is the Kalman gain. Note thatthanks to the unambiguous ID associated to every mote that is embedded in every sentpacket, in our case the data association problem is solved. For a more detailed descrip-tion please refer to [12].

3.2 Delayed Initialization

As shown in [12], the largest part of the residual error in the estimation of the motepositions is due to a wrong initialization of the motes in the Kalman filter at the be-ginning of the SLAM procedure. In this paper, we propose the use of a particle filterfor implementing a delayed initialization in the mote positions. In our previous work[12], the initialization of the Extended Kalman Filter was performed with the trilater-ation method. The robot collected three measures of its distance to a mote, from threenon-aligned positions. Three circumferences centered in each robot position and withradius the corresponding robot-mote distance are traced. The intersection of the circlescorresponds to the mote position. Unfortunately, RSSI range measurements are verynoisy, so the measure of distances are better represented by a mean distance averagedover several measures with an associated standard deviation, hence, an annular ringwith a Gaussian distribution with the peak at the mean distance radius, see Fig. 2. Theestimated mote position now lays somewhere in the intersection of the three annularrings (or better in the intersection of the three Gaussian distributions associated to thethree distances), see the grayish ellipse in Fig. 2. We modeled all this, instancing foreach mote a particle filter in which the samples are hypotheses on the position of themote drawn from a normal distribution. In Fig. 2(Right), the robot received the firstmessage from a mote and calculates the mote-robot distance r from the RSSI using 1.The samples are uniformly distributed around a ring of radius r and standard deviationσ, previously calculated with a calibration table in which at each measured distance isassociated a standard deviation. The weight associated to every particle is given by theGaussian distribution. When the robot receives new messages from the mote and calcu-lates new estimations of the mote distance, these correspond to new annular probabilitydistributions centered in the corresponding robot position. We apply a Sampling Im-portance Resampling (SIR) algorithm each time a new distance estimation is available,making the particle to condensate toward the real mote position, see Fig. 4 in Section 5.To solve the problems of the frequent outliers, that can arise when estimating the dis-tance from an RSSI measure, especially in the first steps of the initialization, we alsodistributed uniformly in the environment a certain percentage of the particles (namely

Page 7: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

Fig. 2. (Left) An example of trilateration. P1, P2, and P3 are robot positions along the path ofthe robot where the corresponding distances robot-node d1, d2, and d3 were estimated using theRSSI. (Right)The anular ring and the samples drawn from the underlying Gaussian distributionassociated to a distance estimated by the robot measuring the RSSI of the messages received bya mote.

the 5 %). This enables the particle filter to recover from a totally wrong estimation asin the case of the well-known “kidnapped robot problem” [17].

4 Object recognition and reaching

Even if, the delayed initialization using the particle filter improves the results of theRSSI-based localization, the residual error still is the order of a meter or so. This is toomuch if we want the robot to be able to approach and manipulate the object, especiallywhen in the same room there are many smart objects. Our proposal is to perform theprecise localization of the smart objects by recognizing their appearance in the robot’scamera. Operatively, when the robot is in the surrounding of the object of interest itswitches to Discovery Mode. A Sleep message is sent to non-interesting motes and theDescriptor Request packets are sent to the mote of interest. This is done to minimize thebattery energy consumption of the sensors and to avoid interference with other activemotes. Received descriptors are passed to the object identification module. The robotcontroller executes the following steps:

– Grab an image with the on-board camera;– Calculate the descriptors of this image;– Match a subset of these descriptors with those arrived from object’s mote;– Return object presence and position to the robot navigation module.

In the last years, many works of visual object recognition was successfully im-plemented exploiting Scale-Invariant Feature Transform (SIFT) descriptors. The SIFTdescriptors, introduced by David Lowe in 1999 [10] [11], extract features from an in-put image to find a match on a set of different reference images. However, in indoor

Page 8: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

environment, most of the time the ambient is dim and the light is not enough for grab-bing clear images. In the case of a mobile robot moving around, seeking for the objectsof interest, this means that the images grabbed by the robot will likely be affected bymotion blur. In our first implementation [12] of a recognition system based on SIFT fea-ture, the system was missing correct matches when the robot moved too fast. We solvedthis problem by using, instead of SIFT, a new feature detector scheme called MoBIF(Motion Blur Invariant Feature detector) we originally developed for humanoid robots[13]. MoBIF descriptors proved to perform similarly to SIFT and SURF on standarddatasets and outperformed SIFT and SURF in images affected by motion blur or in dimimages. Like SIFT, MoBIF descriptors are particularly robust to affine transformationand occlusion. For further details please refer to [13, 14].

4.1 Object appearance description

The MoBIF descriptors provide a compact representation of the appearance of the ob-ject and they can be stored in the tiny mote’s memory. A single MoBIF descriptor,named MoBIF block, occupies approximately 128 bytes. Since the IEEE 802.15.4 pro-tocol data unit (PDU) has a payload of 28 bytes only, each MoBIF block needs to befragments in approximately 7 PDUs for transmission. Each MoBIF block is assigneda 2 bytes signature (MoBIF Identifier, SID) that marks all the fragments of that block,thus making it possible to recognize the fragments of the same original MoBIF block atrobot side. Beside the SID field, each PDU also carries a sequence number (SN) field,of only 3 bits, that specifies in which order the fragments have to be reassembled. Fur-thermore, we dedicated the first two bytes of the payload to carry the SID field, leavingthe remaining 26 bytes for the MoBIF data fragment.

4.2 Descriptor Clouds

Unfortunately, MoBIF descriptor (like SIFT and SURF) are not robust to large perspec-tive transformations nor to large rotations of the object along z-axis that occurs whenthe robot might observe the objects from very different points of view. Another problemregards the scale invariance of SIFT, SURF, or MoBIF. These descriptors show a goodinvariance to the scale only until a certain limit (in the order of a couple of meters forthe size of our objects).

We addressed these two problems taking many pictures of the object from differentpoints of view, separated by 20-30 degrees and at several distances (1 m, 3 m, and 5m). We extracted the descriptors from each image. The descriptors coming from allthe pictures are merged in a single descriptor cloud and the object matching process ismade only once on this cloud. During the building of the cloud, the descriptors comingfrom each picture are added to the cloud in an incremental way. If a descriptor is toosimilar to another descriptor already present in the cloud, it is rejected.Let define Z = {Z1, Z2, ...} as the set of reference pictures. The descriptor cloud isDc.

1. Set Dc = ∅.2. For each reference image Zi in Z do:

Page 9: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

(a) For each descriptor Yj extracted from Zi:– if ∃ Ys ∈ Dc so that Yj is similar to Ys then discard Yj

– else add Yj to Dc

The set of descriptors resulting from this process completely describes the externalsurface of the object. Searching for objects in a frame is hence done by looking forcorrespondences on this set.It should be noted that this union of descriptors potentially similar due the fact that thereference images might share common regions does not compromise the recognitionprocess. In fact the redundant descriptors are joint together in a single sample whichis sufficiently different from any other descriptor of the cloud. Thus its selectivity inpreserved.

On the contrary, considering each reference picture one by one, the time requiredto evaluate the correspondences would drastically increase, because the total number ofdescriptors increases quickly. On the contrary, merging the information coming fromall the pictures and deleting redundant descriptors allows to recognize the object fromany point of view maintaining reasonable computational requirements.

4.3 Object recognition

The object identification process begins by processing the images Im1 taken by theonboard camera in order to extract a list I1 of MoBIF descriptors. This set of descriptorsis compared with the list of descriptor I2 associated to the reference image Im2 of anobject. If there is a minimum number of descriptors of I1 fairly similar to I2, we canassume the object appears in the image.

The recognition of the object in the image acquired by the onboard camera encom-passes three steps, which are repeated for each descriptor x of I1:

1. Find the descriptors y1 and y2 of I2 that are closer to x (using the Euclidean normas distance);

2. If the ratio between the distance of y1 and y2 from x is less than a certain threshold(in this case we used 0.49 as suggested by Lowe [11]) the correspondence betweeny1 and x is accepted;

3. The process is repeated until I1 is empty or the number of correspondences isgreater than a threshold. In this case, it is likely that the target object is presentin Im2.

The object identification process does not require the association of all MoBIF compo-nents of the object with all MoBIF components extracted from the current frame. Thismakes the system robust to packet loss. The proposed method allows to precisely locatethe object without prior knowledge of an approximate position information as requiredby registration techniques like the ICP algorithm that can fall into a local minimum [4].The solution found by our method can be further refined using the ICP algorithm.

Page 10: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

Fig. 3. Example of correct matches obtained with MoBIF descriptors (left) a zoom of an imagegrabbed by the robot without motion blur, (right) an image grabbed with motion blur.

5 Experiments

The robot is a custom-built wheeled differential drive platform based on the Pioneer 2by MobileRobots Inc, see Fig. 1(Right). The robot is equipped with only a camera andthe odometers connected to the two driven wheels. The mote is mounted on top ofthe robot and acts as a gateway to the WSN. The experiment1 in Fig. 4 is performedin outdoor scenario and the robot is equipped with more sensors like encompass andgyroscopes to improve the odometry. Then the experiment is finalized to localize all thesensor in the environment.

5.1 Discovering and Mapping New Objects

When the robot receives from the mote the MoBIF-based object descriptors, it is ableto look for this object in the surrounding environment. Once the object is discovered,the mote is labeled in the map with this descriptor.In our experiments, we use several types of smart objects with well defined appearance(i.e., with distinctive shape and/or distinctive pattern, see for example Fig 1). In all theexperiments we performed, the robot, in addition to correctly localize itself and build amap of the perceived motes, correctly recognized the smart objects and associated themto the right mapped mote. In Fig. 3 an example of correct recognition even in presenceof motion blur is presented. The red dots are the MoBIF descriptors.

Mote ID 2 3 4 5 6xmean (cm) 22.17 -31.11 -18.91 1.29 20.77ymean (cm) 15.93 -48.80 9.01 -39.54 101.92

Table 1. Landmark’s mean error position

1 Watch the simulation on web site: www.dei.unipd.it/danielet/video

Page 11: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

(a) (b) (c)

(e) (f) (g)

Fig. 4. The delayed initialization with the particle filter in action while the robot (the green triangle) is moving in the environment. (First row)(a) The robotreceived the first distance RSSI thus the annular rings are created around the robot and it encompasses the motes; (b) The robot moved; new measures areinserted in the particle filter and the samples cluster around the mote position; (c) the particle filter converged and the node is initialized in the ExtendedKalman Filter. (Second row) More complex situation along the robot path with several showing the convergence of multiple particle filters (d), (e), (f).

Page 12: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

5.2 SLAM with Delayed Initialization

We successfully tested our delayed initialization strategy with some dataset where ourprevious range-only SLAM approach [12] based on initialization with trilateration,failed due to a wrong initial guess of the mote positions. In such a cases the ExtendedKalman Filter may quickly diverge, making impossible to reconstruct the right map ofthe mote positions in the environment and the robot path. With the proposed delayedstrategy based on a particle filter, the initial mote position is estimated over severalconsecutive range measures, filtering out in a probabilistic way the outliers. An exam-ple of SLAM convergence with the proposed approach is depicted in Fig. 4. Beforeincorporating a new mote inside the map, the robot maintains and updates for somesteps a set of particles for every new detected mote: the mote is incorporated once itslocation uncertainty decreases below a threshold. Despite the wide noise in the RSSIrange measurements, our approach make it possible to well recover the robot trajectoryand roughly estimate the motes positions (Fig. 4 (g)). The accuracy of the proposedalgorithm in localizing the different smart objects is shown in Table 1 .

6 Conclusions and Future work

In this paper, we presented the implementation in a single framework of a system oflocalization and recognition of objects for home or service robots, based on the conceptof Intelligent Ambient and Wireless Sensor Network. The robot does not have any a pri-ori information on the objects it needs to locate. It is equipped with the same mote thatis attached to the object to make them “smart”. We presented preliminary experimentsin which the robot is able to discover, locate, and recognize the object by exploitingWSN technologies, a range-only SLAM algorithm, and appearance descriptors robustto motion blur. In the future, we want to integrate in the SLAM filter also the localiza-tion information one can extract from the robot’s camera images, once the camera isprecisely calibrated. The next step will be to map not only the motes, but also the restof the environment by integrating, with the system presented in this paper, the VisualSLAM algorithm we are developing for humanoid robots. Thus, the smart objects willbe located in a 3D visual map of the environment. The final step will be to integratethis complete system with a Robotics Brain Computer Interface we are developing incollaboration with IRCCS San Camillo of Venice (Italy) and the University of Palermo(Italy) [5]. The BCI system will enable to select just “by thinking” the smart objectwe want the robot to interact with. This will open, also to people with severe disabil-ities (like people affected by ALS - Amyotrophic Lateral Sclerosis), the possibility tointeract with a domotic house or an intelligent ambient.

References

1. Network robot forum.2. T. Asfour, P. Azad, N. Vahrenkamp, K. Regenstein, A. Bierbaum, K. Welke, J. Schroder,

and R. Dillmann. Toward humanoid manipulation in human-centred environments. Robot.Auton. Syst., 56(1):54–65, 2008.

Page 13: Discovery, localization and recognition of smart …pretto/papers/simpar2010.pdfNetwork (WSN) technologies for implementing an ambient intelligence scenario. We address the problems

3. P. Azad, T. Asfour, and R. Dillmann. Combining apperance-based and model-based methodsfor real-time object recognition and 6d localization. In Submitted to International Conferenceon Intelligent Robots and Systems (IROS), Beijing, China, 2006. IEEE Institute of Electricaland Electronics Engineers.

4. P. J. Besl and N. D. McKay. A method for registration of 3-d shapes. IEEE Trans. PatternAnal. Mach. Intell., 14(2):239–256, 1992.

5. A. Chella, E. Pagello, E. Menegatti, R. Sorbello, S. Anzalone, F. Cinquegrani, L. Tonin,F. Piccione, K. Prifitis, C. Blanda, E. Buttita, and E. Tranchina. A bci teleoperated mu-seum robotic guide. Complex, Intelligent and Software Intensive Systems, 2009. CISIS ’09.International Conference on, pages 783 – 788, Mar 2009.

6. F. Dressler. Self-organization in autonomous sensor and actuator networks. 2006.7. A. Goldsmith. Wireless Communications. Cambridge University Press, 2005.8. J. Kim, Y. Kim, and K. Lee. The third generation of robotics: Ubiquitous robot. Palmerston

North, New Zealand, 2004.9. J. Lee and H. Hashimoto. Intelligent space - concept and contents. Advanced Robotics,

16(3):265–280, 2002.10. D. Lowe. Object recognition from local scale-invariant features. In Proceedings of Interna-

tional Conference on Computer Vision, pages 1150–1157, 1999.11. D. G. Lowe. Distinctive image features from scale-invariant keypoints. International Journal

of Computer Vision, 60:91–110, 2004.12. E. Menegatti, A. Zanella, S. Zilli, F. Zorzi, and E. Pagello. Range-only slam with a mobile

robot and a wireless sensor networks. Robotics and Automation, 2009. ICRA ’09. IEEEInternational Conference on, pages 8 – 14, May 2009.

13. A. Pretto, E. Menegatti, M. Bennewitz, W. Burgard, and E. Pagello. A visual odometryframework robust to motion blur. Robotics and Automation, 2009. ICRA ’09. IEEE Interna-tional Conference on, pages 2250 – 2257, May 2009.

14. A. Pretto, E. Menegatti, and E. Pagello. Reliable features matching for humanoid robots.Humanoid Robots, 2007 7th IEEE-RAS International Conference on, pages 532 – 538, Jan2007.

15. A. Saffiotti, M. Broxvall, M. Gritti, K. LeBlanc, R. Lundh, J. Rashid, B. Seo, and Y. Cho. Thepeis-ecology project: Vision and results. In IROS 2008: Intelligent Robots and Systems, pages2329–2335, Amsterdam, The Netherlands, The Netherlands, 2008. North-Holland Publish-ing Co.

16. A. Sanfeliu, N. Hagita, and A. Saffiotti. Network robot systems. Robot. Auton. Syst.,56(10):793–797, 2008.

17. S. Thrun, W. Burgard, and D. Fox. Probabilistic Robotics (Intelligent Robotics and Au-tonomous Agents). The MIT Press, 2005.

18. M. Vincze, M. Schlemmer, P. Gemeiner, and M. Ayromlou. Vision for robotics: a tool formodel-based object tracking. Robotics and Automation Magazine, IEEE, 12(4):53–64, Dec.2005.

19. G. Zanca, F. Zorzi, A. Zanella, and M. Zorzi. Experimental comparison of rssi-based lo-calization algorithms for indoor wireless sensor networks. In REALWSN ’08: Proceedingsof the workshop on Real-world wireless sensor networks, pages 1–5, New York, NY, USA,2008. ACM.