Indoor localization using controlled ambient sounds ?· Indoor localization using controlled ambient…

Download Indoor localization using controlled ambient sounds ?· Indoor localization using controlled ambient…

Post on 02-Aug-2018




0 download

Embed Size (px)


<ul><li><p>Indoor localization using controlled ambient soundsIsh Rishabh</p><p>University of California, IrvineIrvine, CA, USA</p><p>Email:</p><p>Don KimberFX Palo Alto Laboratory</p><p>Palo Alto, CA, USAEmail:</p><p>John AdcockFX Palo Alto Laboratory</p><p>Palo Alto, CA, USAEmail:</p><p>AbstractAudio-based receiver localization in indoor environ-ments has multiple applications including indoor navigation, loca-tion tagging, and tracking. Public places like shopping malls andconsumer stores often have loudspeakers installed to play musicfor public entertainment. Similarly, office spaces may have soundconditioning speakers installed to soften other environmentalnoises. We discuss an approach to leverage this infrastructureto perform audio-based localization of devices requesting local-ization in such environments, by playing barely audible controlledsounds from multiple speakers at known positions. Our approachcan be used to localize devices such as smart-phones, tabletsand laptops to sub-meter accuracy. The user does not needto carry any specialized hardware. Unlike acoustic approacheswhich use high-energy ultrasound waves, the use of barelyaudible (low energy) signals in our approach poses very differentchallenges. We discuss these challenges, how we addressed those,and experimental results on two prototypical implementations:a request-play-record localizer, and a continuous tracker. Weevaluated our approach in a real world meeting room and reportpromising initial results with localization accuracy within half ameter 94% of the time. The system has been deployed in multiplezones of our office building and is now part of a location servicein constant operation in our lab.</p><p>I. INTRODUCTIONIndoor localization has several applications such as indoor</p><p>navigation, location tagging, and tracking. Due to the obstruc-tion of direct line-of-sight to satellites in indoor environments,Global Positioning System (GPS) does not work well indoors.Researchers have actively explored alternative mechanismsfor indoor localization. Many systems have been proposedthat utilize different modalities like wi-fi, RFID or camera-networks. These systems often employ equipment which isexpensive or deployed solely for the purpose of localization.Many of these systems (like Wi-Fi and RFID-based) fail toachieve a sub-meter accuracy which is often required forapplications like indoor navigation. Some systems have thepotential to infringe upon user privacy (such as camera-basedtracking systems or systems using microphones deployed inthe environment).</p><p>In developing our approach to indoor navigation, we havefocused on several guiding principles. We want accuracy onthe order of a meter or better. The setup should not requireany dedicated expensive equipment to be installed in the en-vironment. Users of our system should not require specializedgadgets or procedures; an app running on a smartphone shouldbe enough. Moreover, our system should respect user privacyand should not require potentially invasive microphones orcameras to be installed in the environment.</p><p>Fig. 1. GUI for the tracking server. The top panel shows cross-correlations (inblack) of the recorded signal with 6 reference signals played from speakers,with estimated delays indicated as small green bars. The tall green lineindicates the estimated time when the signals were played. The bottom paneldepicts a map of the room, showing speaker positions as red circles. The greendot indicates estimated microphone position, and green lines show which 4speakers out of the 6 were used for the estimate.</p><p>Large indoor locations such as malls, consumer stores andmuseums are usually equipped with loudspeakers for publicaddress, or to play music for customer entertainment. Indoorworkspaces often include sound conditioning speakers to playnoise or ambient sounds to soften other environmental noise.With little modification, these systems can be leveraged toprovide additional functionality to allow users to determinetheir location. This work introduces an approach that usescontrolled ambient sounds played through multiple speakers,which can be recorded by a smart-device (smart-phone, tablet</p></li><li><p>2012 International Conference on Indoor Positioning and Indoor Navigation, 13-15th November 2012</p><p>or laptop). The recorded audio can then be used to determinea users location.</p><p>The audio played from each speaker contains a low energypseudo-random white noise sequence, possibly mixed withmusic or other sounds for human consumption. The audio isplayed using any multi-track audio system, such as commonlyavailable 5.1 or 7.1 surround systems, so that the source tracksare known to be synchronized with each other. No a priorisynchronization between the source and recording device isrequired. The recorded audio is analyzed to estimate the arrivaltimes of the signals from each speaker. These estimates arethen used to determine both the microphone position, and thesynchronization between speakers and microphone. Our con-tributions lie in proposing an audio-based receiver localizationsystem that uses low-energy (barely audible) audio signalsfor indoor localization. We use windowed cross-correlationof long duration signals to improve signal-to-noise ratio andto account for mismatched speeds between the playback andreceiver system clocks, i.e. the clock drift.</p><p>To demonstrate our approach, we have developed a systemwhich provides localization services in three zones of ourresearch center - the main meeting room, the kitchen area,and a lab area. A user interface to the localization system isshown in Figure 1. It may be run as a standalone client on alaptop, or as a server back end, providing estimates to Androidclients as shown in Figure 2.</p><p>II. RELATED WORK</p><p>Many systems attempt to address the problem of indoorlocalization using different modalities. Systems using wifi [1],[2] report a median localization error of more than a meter, andtypically involve extensive setup and tuning. Localization errorfor systems based on GSM [3], IR [4] and RFID [5] also fallin similar range. In order to attain sub-meter accuracy, opticaland acoustic based systems have to be used.</p><p>Localization systems can conceptually be classified intosource localization systems and receiver localization systems.Camera-based (optical) systems [6], [7] and most acoustic sys-tems using microphones [8], [9], [10], [11], [12] try to localizea source by sensing its signal through multiple receivers withknown positions. Such systems can potentially violate userprivacy by recording their video or voices without explicitapproval. Another undesirable property of audio-based sourcelocalization systems is that the device to be localized needs tocontinuously make audible sounds, which may be a cause ofirritation to the user [12].</p><p>On the other hand, in receiver localization systems, areceiver senses the ambient signal which is then processed todetermine the receivers location. The ambient signal can ei-ther be uncontrolled or controlled. Systems using uncontrolledsignals either employ multiple known receivers [13] as part ofthe system (potentially violating privacy) or use collaborativesensing. Collaborative sensing [14] requires multiple unknownreceivers that try to simultaneously localize themselves. Somesystems like [15] take a hybrid approach by using both asource and receiver in the client device. Such collaborative</p><p>Fig. 2. Android App showing map of building. The App is streaming audiobuffers to a processing server, which returns position estimates in real time.Red dots show positions of loudspeakers in three zones covered. The blue pinindicates estimated position, which is updated in real time. The size of thedots in that zone indicate peak strength, and red lines indicate which speakerswere used for estimation.</p><p>systems cannot be used to localize a single receiver. In othersystems like [16] the receiver senses multiple characteristicspatial signatures in an environment to identify which part ofthe environment it is in. Systems such as [17], [18] model theacoustic background spectrum to produce location dependentsignatures. These signature based fingerprinting methodshave the advantage of not requiring installed infrastructure,but require training, provide only coarse level (e.g. room level)localization, and are subject to sporadic changes in backgroundacoustics.</p><p>The proposed system differs from these systems as it isa receiver localization system using controlled audio sig-nals. Many systems employing ultrasonic waves [19], [20],[21], [22], [23], [24] usually have a better ranging accuracycompared to those that use audible sound, but these systemshave several limitations. They require dedicated special in-frastructure - ultrasound transducers and wireless coordinationnetworks that are not used for any other purpose [25]. Mostlow-end and PC speakers have a frequency response limitedto 20kHz, and many smartphones microphones are unsuitablefor ultrasonic signals. Also, ultrasound has a limited rangecompared to audible sound due to greater attenuation whilepropagating through the air. Another difference between ap-</p></li><li><p>2012 International Conference on Indoor Positioning and Indoor Navigation, 13-15th November 2012</p><p>proaches using ultrasound and ours is that ultrasonic systemscan afford to use high-energy signals to boost signal-to-noiseratio. We, on the other hand, cannot use high-energy soundsas it might be disturbing to the people in the environment.Hence, our approach is tailored toward localization in low SNRconditions.</p><p>We differentiate our work in the following ways.1) We use low-energy sounds as our reference signals.2) We improve effective SNR in the cross-correlations by</p><p>using long duration reference signals and recordings.3) We assume no explicit prior synchronization between the</p><p>speakers and the recording device. This avoids overheadslike using a separate RF channel for synchronization.</p><p>We also introduce the following ideas in this paper.1) Use of windowed cross-correlation and its modification</p><p>to address clock drift (Section IV-C) between playingand recording devices.</p><p>2) A simple correlation quality heuristic to select a subsetof speakers so that only those speakers with good cross-correlation peaks are selected for localization. This helpsin avoiding speakers for which correlation is very noisy.</p><p>3) Use of a solution fitness (residue in Section IV-E)to decide whether the estimated value of t0, to beintroduced later, is accurate. This value of t0 can bere-used later for better estimation.</p><p>III. ARCHITECTURE &amp; SYSTEMWe have experimented with several variations of the pro-</p><p>posed system, and have built a flexible architecture to supportdifferent types of experiments and modes of usage. Thearchitecture includes three primary elements: (1) audio playersresponsible for playing known synchronized tracks from mul-tiple speakers, (2) clients equipped with microphones, runningon laptops, tablets or smart phones, and possibly (3) processingservers, which could be used by underpowered clients toperform localization. Depending on usage requirements, audioplayers may be controllable servers or standalone audio decksplaying continuously and autonomously.</p><p>The first mode we experimented with is the request-play-record mode. A client requests that the server play knownsignals from each speaker, and the client then records itsmicrophone signal. The recorded signal is processed, eitheron the client, or in the case of an Android client, is uploadedto a processing server. Finally, the result is returned to theclient, for display on a map. This usage scenario is shownin Figure 3. In this case, although there is a rough level ofsynchronization in the sense that the latency between clientrequests and audio playing, may be on the order of .1 sec orless, we do not assume precise sample level synchronization.Synchronization is discussed in more detail below. For somescenarios, it may be appealing for clients to be able to requestwhen sounds are to be played, but it has some disadvantages.One is the requirement of additional infrastructure and systemcomplexity. Another is contention, since in a busy environ-ment, different clients may request sounds to be played atsimilar times.</p><p>Fig. 3. Request-play-record client-server mode. Clients make play requeststo audio server. The server plays audio, and clients record audio. Clients thenprocess audio locally, or upload to a server which processes recording andreturns location estimates. The audio server and processing server may be thesame machine.</p><p>A second mode, with many advantages in practice, iscontinuous mode. In this mode, the signals are played con-tinuously, independently of any client requests. The signalsplayed contain periodic pseudo random sequences, with periodof about .5sec. In some cases we mix those signals with othersounds, such as music, which seem to have relatively littleeffect on the performance of the system. Note that for thismode, there is no a priori synchronization between playerand client, and in fact the player does not require a server.We have successfully used an Oppo BDP-93 Blue-ray discplayer for this purpose. Also, in this mode, the clients arerunning independently of the server and of each other. A clientsimply records some audio, and processes it, when it wants alocalization. Note that in this mode, there is no single discretesignal. Although the processing may be performed on a singlefixed length recording, it may also be performed continuouslyas a sort of position tracker. In the continuous case, successiveaudio buffers are used to estimate arrival delays, as peaks in arunning average windowed correlation. Position estimates arerecomputed each frame. An implementation of this continuoustracker will be discussed in Section IV-E.</p><p>Several important issues of synchronization and timingmust be addressed in our system. All clients and servershave software accessible system clocks. In addition, thereare sampling clocks used by all playing and recording audiodevices. It is not easy to ensure synchronization of systemclocks, because Android devices do not directly support anaccurate time protocol such as NTP without unlocking thedevices to gain root privileges. It is also difficult to ensure syn-chronization between system clocks and audio device clocks,because the audio device interfaces typically do not providethe system clock time associated with the first sample of eachaudio buffer. We have found, and others have reported [11],unpredictable variable delays between system time and truesample time on the client side for various portable devices.On the server side, even using a low latency audio interface</p></li><li><p>2012 International Conference on Indoor Positioning and Indoor Navigation, 13-15th November 2012</p><p>Fig. 4. Continuous play mode. The audio system continually plays the signal,and various clients record independently, and process locally or stream toserver for processing.</p><p>such as PortAudio [26] with ASIO drivers on Windows wefound unpr...</p></li></ul>


View more >