towards a cloud robotics platform for distributed visual slam · roboearth [6] aims to develop a...

13
Towards a Cloud Robotics Platform for Distributed Visual SLAM Peng Yun 1 , Jianhao Jiao 2 , and Ming Liu 2(B ) 1 Department of Computer Science Engineering, HKUST, Kowloon, Hong Kong [email protected] 2 Department of Electronic and Computer Engineering, HKUST, Kowloon, Hong Kong {jjiao,eelium}@ust.hk Abstract. Cloud computing allows robots to offload computation and share information as well as skills. Visual SLAM is one of the intensively computational tasks for mobile robots. It can benefit from the cloud. In this paper, we propose a novel cloud robotics platform named RSE-PF for distributed visual SLAM with close attention to the infrastructure of the cloud. We implement it with Amazon Web Services and OpenResty. We demonstrate the feasibility, robustness, and elasticity of the proposed platform with a use case of perspective-n-point solution. In this use case, the average round-trip delay is 153ms, which meets the near real-time requirement of mobile robots. Keywords: Cloud robotics · Mobile robots · Visual SLAM · Perspective-n-point 1 Introduction 1.1 Motivation Visual SLAM is one of the intensively computational tasks for mobile robots [1, 2]. It requires powerful processors to estimate the pose from each incoming frame in real time [3]. Cloud computing is a promising method to make up for the deficiency of local computational ability. Traditional robotics, which combined with cloud computing, evolves into cloud robotics in recent years. The cloud allows robots to offload intensively computational tasks and large-scale storage, so that both cost and power consumption of robots get reduced [4]. Besides, This work was sponsored by the Research Grant Council of Hong Kong SAR Government, China, under project No. 16212815, 21202816 and National Nat- ural Science Foundation of China No. 6140021318 and 61640305; Shenzhen Sci- ence, Technology and Innovation Comission (SZSTI) JCYJ20160428154842603 and JCYJ20160401100022706; partially supported by the HKUST Project IGN16EG12. All rewarded to Prof. Ming Liu. c Springer International Publishing AG 2017 M. Liu et al. (Eds.): ICVS 2017, LNCS 10528, pp. 3–15, 2017. DOI: 10.1007/978-3-319-68345-4_1

Upload: others

Post on 13-Sep-2020

6 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platformfor Distributed Visual SLAM

Peng Yun1, Jianhao Jiao2, and Ming Liu2(B)

1 Department of Computer Science Engineering,HKUST, Kowloon, Hong Kong

[email protected] Department of Electronic and Computer Engineering,

HKUST, Kowloon, Hong Kong{jjiao,eelium}@ust.hk

Abstract. Cloud computing allows robots to offload computation andshare information as well as skills. Visual SLAM is one of the intensivelycomputational tasks for mobile robots. It can benefit from the cloud. Inthis paper, we propose a novel cloud robotics platform named RSE-PFfor distributed visual SLAM with close attention to the infrastructure ofthe cloud. We implement it with Amazon Web Services and OpenResty.We demonstrate the feasibility, robustness, and elasticity of the proposedplatform with a use case of perspective-n-point solution. In this use case,the average round-trip delay is 153 ms, which meets the near real-timerequirement of mobile robots.

Keywords: Cloud robotics · Mobile robots · Visual SLAM ·Perspective-n-point

1 Introduction

1.1 Motivation

Visual SLAM is one of the intensively computational tasks for mobile robots[1,2]. It requires powerful processors to estimate the pose from each incomingframe in real time [3]. Cloud computing is a promising method to make up for thedeficiency of local computational ability. Traditional robotics, which combinedwith cloud computing, evolves into cloud robotics in recent years. The cloudallows robots to offload intensively computational tasks and large-scale storage,so that both cost and power consumption of robots get reduced [4]. Besides,

This work was sponsored by the Research Grant Council of Hong Kong SARGovernment, China, under project No. 16212815, 21202816 and National Nat-ural Science Foundation of China No. 6140021318 and 61640305; Shenzhen Sci-ence, Technology and Innovation Comission (SZSTI) JCYJ20160428154842603 andJCYJ20160401100022706; partially supported by the HKUST Project IGN16EG12.All rewarded to Prof. Ming Liu.

c© Springer International Publishing AG 2017M. Liu et al. (Eds.): ICVS 2017, LNCS 10528, pp. 3–15, 2017.DOI: 10.1007/978-3-319-68345-4_1

Page 2: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

4 P. Yun et al.

robots have an access to a shared repository on the cloud [5–7]. As a result,data sharing and skills reusing can be achieved. A robot is eventually not onlya machine but also a bridge between its user and heterogeneous services.

Although the cloud benefits robots, some drawbacks and challenges still existand must be further addressed. Existing cloud robotics platforms enable datatransmission and executing computation parallelly [4,8] and optimizedly [9,10].However, few of them pay attention to the infrastructure of the cloud. Similar toother cloud platforms, the platform for distributed visual SLAM is open to thepublic and required to run stably in a long time. Under a long-term operation,some modules of the cloud tend to crash occasionally. In this paper, we pay closeattention to the robustness, security, and elasticity of the cloud. We propose anovel cloud robotics platform named RSE-PF for distributed visual SLAM.

Besides, most of the previous work constructed their cloud platforms withROS1. ROS provides researchers a middleware to design their robot systemsin a modular way. The loosely coupled modules can be re-used, which helpsresearchers implement their robot systems conveniently. However, there are stillsome shortcomings with ROS, like its huge overhead and platform dependencies.Therefore, different with [5], RSE-PF does not treat ROS as a necessary part.

1.2 Contribution

We stress the following contributions in this paper.

– We clarify the tasks of each cloud service type for distributed visual SLAMand propose a novel cloud robotics platform named RSE-PF.

– We demonstrate the feasibility, robustness, and elasticity of RSE-PF withimplementing the perspective-n-point solution for autonomous robots creat-ing point cloud maps in a distributed way.

The features of RSE-PF are as follows.

Robust: The broken servers are detected and replaced automatically.Secure: Only authorized users are allowed to send requests to the cloud.Elastic: The cloud scales outward when it is under huge pressure. It scales inwardwhen it keeps idle for a while.Multiple programming languages supported: Visual SLAM services could beimplemented with C/CPP and Lua scripts. Researchers are allowed to imple-ment their application with/without ROS.

1.3 Organization

In Sect. 2, a brief introduction to cloud computing and previous work isdescribed. In Sect. 3, we clarify the tasks of three cloud service types for dis-tributed visual SLAM. The RSE-PF model is presented in Sect. 4 along with theimplementation in Sect. 5. In Sect. 6, we demonstrate the feasibility and featuresof RSE-PF via implementing the perspective-n-point solution based on proposedcloud platform. Finally, we conclude with a discussion of our future work.1 http://www.ros.org/.

Page 3: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platform for Distributed Visual SLAM 5

2 Related Work

2.1 Cloud Computing

NIST defined cloud computing and three service types of it [11], i.e. Software asa Service (SaaS), Platform as a Service (PaaS), and Infrastructure as a Service(IaaS). In brief, SaaS applications are exposed to users. The users have an accessto the functional applications via various devices (e.g. mobile phones, laptops,tablets) and do not manage or control hardware resources or runtime environ-ments directly. The users of PaaS are provided a platform which includes pro-gramming languages, dependencies, and management tools. They are allowed todeploy their applications on the platform without consideration on the infrastruc-ture of the cloud. As for IaaS, the users are able to manage or control the cloudinfrastructure in the way provided by IaaS providers. IaaS, PaaS, and SaaS offerincreasing abstraction. The relationship of these three cloud service types isconsidered as Fig. 1.

Infrastructure as a Service

Platform as a Service

Software as a Service

Fig. 1. The relationship of IaaS, PaaS, and SaaS: IaaS provides basic computing, stor-age as well as network components to the platform. PaaS provides necessary runtimeenvironment to applications. SaaS is the functional applications which are exposed tothe users.

2.2 Cloud-Based Visual SLAM

As for visual SLAM, robots are required to compute the camera pose from eachincoming frame. Limited by on-board computational ability, robots are hardto locate themselves and map the scene in real time. Enti et al. [4] proposedDAvinCi framework which offloaded intensively computational workload fromonboard resources to a backend cluster system. They implemented such a soft-ware framework around Hadoop cluster with ROS message. They proved thefeasibility of their system and performance gains in execution times through

Page 4: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

6 P. Yun et al.

implementing the FastSLAM based on DAvinCi. RoboEarth [6] aims to developa giant repository and achieve a World Wide Web for robots so that robots areable to share data and reuse skills. As a part of RoboEarth project, Repyuta[5] provided an access to RoboEarth knowledge repository which enables robotsto share data and skills. Based on Rapyuta, [7] implemented a dense visualodometry algorithm on a smartphone-class ARM multi-core CPU.

A common feature of previous work is that they built a bridge between clientsand servers with ROS message or WebSocket. Clients send data to the cloud, andservers finally send results back. In this way, offloading can be achieved. However,the infrastructure of the cloud does not get a full use. In this paper, we proposea cloud robotics platform for distributed visual SLAM with consideration onrobustness, security, and elasticity. Besides, in order to show the feasibility andfeatures of RSE-PF, we demonstrate the use case that multiple robots createpoint cloud maps simultaneously and distributedly with the PnP solution servicebased on RSE-PF.

3 Cloud Services for Distributed Visual SLAM

In general, cloud services can be grouped into three categories: IaaS, PaaS, andSaaS [11]. Similar to general cloud services, cloud services for distributed visualSLAM includes these three types as follows.

3.1 IaaS

IaaS provides basic computation, storage, and network resources. Since the cloudis open to the public, the security of the cloud should get attention. Besides, thecloud may be under high concurrent connection at some point, and it may alsokeep idle for a while. Therefore, it should be adjusted elastically according tosome metrics like CPU utilization or robot counts. In addition, the infrastruc-ture must be fault-tolerant. Since there are hundreds of servers in the cloud,some crashes or errors may happen occasionally. There should be a robustnessmechanism to handle such situations, like replacing the crashed server with anew one.

3.2 PaaS

PaaS provides prerequisites to SaaS applications, like dependencies, program-ming language supports, etc. It also provides each robot client with secure andisolated runtime environments. Besides, the communication protocol betweenrobots and the cloud should be well-designed to save bandwidth resources andreduce network delay. In addition, the platform should be with low overhead sothat the cloud are able to handle high concurrent connections.

Page 5: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platform for Distributed Visual SLAM 7

3.3 SaaS

SaaS provides visual SLAM services in a stateless way. The stateless way meansthe received requests contain all necessary information for SaaS applications,and the cloud does not need to retain information for computation. In Sect. 6,the PnP solution is one of the stateless applications.

As mentioned in IaaS, one robustness mechanism is to replace the crashedserver with a new one. With this mechanism, the new server can hardly provideservices to the past users if the SaaS application is stateful. It is because thestate information has gone with the crashed server. Take ORB-SLAM [3] foran example. ORB-SLAM is classified as a stateful algorithm in this paper, forthe keyframes of ORB-SLAM could be considered as the state of this algorithm.When the server running ORB-SLAM crashes, another one can hardly replaceit and continue to provide services because the keyframes have gone with thatbroken server.

Besides, it is important to consider the influence of network delay in somevisual SLAM applications which are sensitive to network delay. At t + Δt1, thecloud receives the message which was sent at t. The result of this message willbe received by the robot at t + Δt1 + Δt2. Δt1 and Δt2 are the network delayof uploading and downloading respectively.

4 RSE-PF Model

Based on three cloud service types, we propose RSE-PF for distributed visualSLAM. As depicted in Fig. 2, it consists of a firewall, a load balancer, a monitor,a scale modifier, multiple backend servers, and multiple data servers. The mainfunctions of each part are as follows.

Load Balancer

MonitorScale Modifier

Firewall

IaaS

PaaS

SaaS

Backend Server Backend Server Backend Server

Data Server Data Server

Fig. 2. The architecture of RSE-PF model (Orange: infrastructure, blue: platform,green: software) (Color figure online)

Page 6: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

8 P. Yun et al.

firewall : Since the cloud provides services to the public, a firewall is a necessarypart of the cloud. Developers design the security rules, open ports, and accesscontrol lists to protect the cloud from unauthorized visits.load balancer : It distributes requests to backend and data servers. It also executeshealth detection towards backend and data servers periodically.monitor : It monitors the states of the cloud, including the result of the healthdetection, CPU and bandwidth usage of backend server, etc. Once some eventsget detected, the monitor will inform the scale modifier.scale modifier : It modifies the scale of the cloud after getting informed by themonitor. It launches servers when the cloud is under big pressure. It also killsservers when the cloud keeps idle for a while. Besides, it replaces the brokenserver with a new one according to the result of health detection.backend server : Developers design the transmission method between robots andthe cloud. They also implement their visual SLAM applications like PnP, placerecognition in a stateless way.data server : It stores data which needs long-term storage or to be shared amongmultiple robots. Images, key frames, and maps can be stored in visual SLAMapplications.

In Fig. 2, IaaS, PaaS, and SaaS are labeled accordingly. The mechanism ofIaaS originates from Amazon Web Services2, which is as follows. After requestsreach the load balancer through a firewall, the load balancer distributes therequests to backend and data servers. Backend servers make computation andexchange data with data servers. In addition, when the monitor detects eventslike unhealthy servers or some metrics exceeding warning lines, it will inform thescale modifier. Subsequently, the scale modifier will adjust the cloud scale.

5 RSE-PF Implementation

In this part, we will detail the implementation of RSE-PF model from IaaS,PaaS, and SaaS.

5.1 IaaS

As proposed above, RSE-PF consists of a load balancer, a scale modifier, amonitor, data servers and backend servers. Multiple companies provide their IaaSproductions, such as Amazon Web Services(AWS) and Google Compute Engine3.AWS provides some cloud services infrastructures, covering computing, storingand network. We implement the IaaS of RSE-PF based on AWS as follows.

Computing: We implement backend servers with Amazon EC2 instances. Ama-zon EC2 allows users to exploit cloud resources on demand. Via Amazon MachineImages or executing pre-prepared scripts, EC2 instances are armed with quali-fied environments immediately after they get created.2 https://aws.amazon.com/.3 https://cloud.google.com/compute/.

Page 7: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platform for Distributed Visual SLAM 9

Storage: AWS provides multiple types of storage, including S3, EBS, and Glacier,etc. S3 allows long-time storing, just like network disks. A little difference tonetwork disks is that S3 naturally serves machines instead of human beings.API is provided to help developers manage and develop the cloud automatically.EBS provides high IOPS, just like hard disk, while Glacier provides cheap andpermanent storing service, which suits for storing log files and raw data.Network: Amazon Virtual Private Cloud can be used to construct the cloudarchitecture and improve the security of the cloud. Besides, main parts can bedoubled with VPC to add redundancy and achieve high availability.Security: Besides VPC, each EC2 instance is protected by a firewall. Users canlimit unauthorized visits with secure strategies of this firewall.Management: Elastic Load Balancer of AWS can be used as a load balancer.It performs request distribution and health detection. In particular, HTTP,HTTPS, and WebSocket protocols are all supported in AWS Elastic Load Bal-ancer. CloudWatch of AWS helps monitor the whole cloud. It alarms when somemetrics of the cloud exceeding their thresholds. In addition, AWS Auto Scalinghelps modify the scale of cloud system elastically after being alarmed by Cloud-Watch.

5.2 PaaS

The PaaS of RSE-PF is implemented based on OpenRestyTM 4. It is a powerfulweb platform which integrates Nginx core, Lua libraries and LuaJIT. Researchersare allowed to implement web services on it with Lua scripts, Nginx C mod-ules, and C/CPP programming languages. Compared to huge overhead of ROS,OpenRestyTM is capable of handling 10 K to 1000 K connections in a singleserver.

The flowchart RSE-PF PaaS is shown in Fig. 3. Subsequently, we will definethe communication protocol and explain how to build an isolated computingenvironment for each robot.

Communication Protocol: In general, messages of visual SLAM transmittedbetween robots and the cloud could be classified as three types: command, data,and result messages. In most applications of visual SLAM, both command andresult messages are small-scale, like place recognition and pose estimation. There-fore, we encode command and result messages into JavaScript Object Notation(JSON). JSON is a lightweight data exchange format which is easy for machinesto parse and generate. The JSON format string can be easily decoded in thecloud via OpenRestyTM cjson Lua model. Typical messages are shown in Fig. 3.

To maintain a bi-directional full duplex connection between robots and thecloud, WebSocket is used to transmit messages, especially command and resultmessages. WebSocket is built on the top of HTTP. It creates open commu-nication channels between clients and servers. As a consequence, the cloud isable to publish messages actively without robots asking results periodically.

4 https://OpenResty.org/.

Page 8: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

10 P. Yun et al.

NginX Lua CoreFunction

File

Command

DataResult

Lua VM

Command Messages: "robot_id": "...", "type": "command(...)", "parameters": "..."

Result Messages:"robot_id":"...","type": "result","result":"..."

Fig. 3. The flowchart of RSE-PF PaaS (Robots send command and data messagesto the cloud, while the cloud publishes results back to robots. In the cloud, whenthe connection between the robot and the cloud is established, a Lua VM is created.Following computation work will be completely processed in this Lua VM.)

Besides, compared with HTTP, WebSocket-based transmission helps save band-width resources hundreds of times under high concurrent connection. It alsohelps reduce 70% network delay compared with HTTP long polling [12].

Payload length of messages is one of the key metrics because large scalemessages consume plenty of bandwidth resources. Some results demonstratedthat the delay and packet loss will increase if payload length of messages increases[5]. Therefore, we compress JSON format string with Zip before transmission.

In applications of visual SLAM, robots might need to offload some data likepoint clouds, maps, and RGB images to the cloud for storage or computation.These data might be larger than 100 Kbytes. Converting images or point cloudsto JSON format string would result in an even larger message size. When thepayload length goes up, the performance goes down. As a result, we adopt HTTPto post files to data servers instead of encoding them to JSON string so thatlarger message size caused by converting is avoided.

Isolated Computing Environment: When the connection between the robotand the cloud is established, a Lua VM is created. Further computational taskswill totally be processed in this Lua VM. After loading necessary Lua models orC dynamic link libraries, the computational environment specific for this robot

ConnectionEstablished

(Lua VM Created)

Connection Closed(Lua VM Released)

Request

Response

Fig. 4. The state diagram of the cloud (When the connection is established, the LuaVM is created. The robotic services will be provided in a request-response way. Oncethe connection is closed, the Lua VM will be released)

Page 9: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platform for Distributed Visual SLAM 11

is prepared. Besides, the memory of each Lua VM is also isolated from others.After the Lua VM having been created, the robotic services start. The robotsends command messages to the cloud, and the cloud will publish result messagesactively towards the robot. If the robot does not need services any more, thisconnection will be closed, and the Lua VM will be released simultaneously. Thelife length of Lua VM is as the same to the connection, which is shown in Fig. 4.

5.3 SaaS

In RSE-PF, we develop visual SLAM services with Lua and C/CPP. Lua is alightweight script language, which could help implement some basic processing,like encoding and decoding of JSON format string, recording logs, etc. C/CPPcould help implement complicated logics in visual SLAM algorithms. In RSE-PF, developers are allowed to compile their C/CPP codes of robotic services intodynamic link libraries, which will be called by LuaJIT FFI.

As shown in Fig. 3, The message received by Nginx are processed by Luabasically, like file operations, encoding and decoding. It implements compli-cated logics of visual SLAM by calling core functions which is dynamic linklibraries coded by C/CPP. Finally, the returned value from core functions willbe published actively to the robot. Furthermore, in order to detail the methodto implement SaaS of RSE-PF, we take perspective-n-point as an example innext section.

6 RSE-PF Use Case

In this section, we implement perspective-n-point (PnP) solution [13] based onRSE-PF to demonstrate its feasibility and features. With PnP robotic service,robots create point cloud maps from the NYU Depth V2 dataset [14].

The perspective-n-point is the problem of estimating the position and ori-entation of the camera given n correspondent points. It originates from cameracalibration and is widely used in many fields, especially location and navigationof autonomous robots.

6.1 Implementation

Since both matched key points and transformation matrix are in small scale inPnP solution service, all necessary information are sent by command or resultmessages via WebSocket. Besides, we compress JSON-format string with libz toreduce bandwidth costs.

The division of computation between robots and the cloud is shown in Fig. 5.In the cloud, Lua modules process requests basically, including encoding, decod-ing, compressing and decompressing. We implement PnP solution based on thesolvePnPRansac function of OpenCV. It makes the final solution more robust tooutliers with RANSAC. We encapsulate the PnP solution into a core function.The process is shown in Fig. 6. The robot computes descriptors and matches

Page 10: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

12 P. Yun et al.

Obtain FramesCompute Descriptors

Match Key PointsJoin Point Cloud

Solve Perceptive-n-Point

Matched Key Points

Transform Matrix

Command Messages: "robot_id": "1001", "type": "command(pnp)", "pts_obj": "0.0128909,-0.378263,2.671;...", "pts_img": "329,182;334,110;..."

Result Messages:"robot_id":"1001","type":"result","matrix":"1,0,0,0; 0,1,0,0; 0,0,1,0; 0,0,0,1;","inliers":22

Fig. 5. Computation division between robots and the cloud (PnP)

server_initialization()while true{ data = receive() data = uncompress(data) if the data contains close command close the connection else { result = ffi.pnp_solution(data) result = compress(result) send(result) }}

char* pnp_solution(char* data){ parameters = decode_json(data) result = solvePnPRansac(parameters) result = encode_json(result) return result}

Lua Scripts C/CPP core function

Fig. 6. The pseudocode of PnP robotic service.

keypoints locally. After received transform matrix, it will join current frame intothe point cloud. Finally, the point cloud of the whole scene will be created.

As proposed in Sect. 5, we implement the infrastructure of RSE-PF withAWS. Data servers are not used in this case because necessary information isin small scale (less than 10 KB) and can be carried in command messages. Wechoose AWS in Seoul because the delay from our lab in Shenzhen to Seoul EC2server is the lowest among all AWS regions. AWS ELB, Cloud Watch, and AutoScaling are used to construct the IaaS of RSE-PF. A hundred Docker containersact as robots and request PnP services. Docker containers run on a Core i7processor with 16 GB RAM (Fig. 7).

Page 11: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platform for Distributed Visual SLAM 13

Top Left

FrontPerspective

Fig. 7. The point cloud map created by one of Docker containers

6.2 Result

We record the real-time performance of all these robots. Robots perform near5 fps in this use case, and the average time from sending a command message toreceiving a result message is 156.36 ms. Specifically, the average time for server

13:15 13:20 13:25 13:30 13:35 13:40 13:45 13:50 13:55 14:00 14:05 14:10 14:150

10

20

30

40

50

60

70

80

90

100

Time of Day

Ave

rag

e C

PU

Uti

lizat

ion

(%

)

0

1

2

3

4

5

6H

ealt

hy

Bac

ken

d S

erve

r C

ou

nt

Fig. 8. The average CPU utilization and the healthy server count of the cloud whenthe cloud faces an increasing number of requests. (Every 20 s, a docker container islaunched, so that a hundred docker containers sent requests to the cloud at the sametime. Finally, we kill all docker containers after 13:52.)

Page 12: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

14 P. Yun et al.

computing is 29.82 ms. In other words, the transmission delay is near 130 mswhich is in the normal range for cloud computing.

From the results, the transmission delay is a bottleneck for real-time distrib-uted visual SLAM services. If better real-time performances are expected, cloudresources close to robots are preferred. Besides, developers are recommended toassign the computation tasks carefully. The tasks which are not sensitive to delayare recommended to be assigned to the cloud, like map merging and global opti-mization. In addition, the saved time should be more than transmission delay sothat real-time performance gains from cloud computing.

Besides, Fig. 8 demonstrates the elasticity and robustness of the cloud. Thecloud scales up when the average CPU utilization exceeds the threshold (70% inthis case). It scales down when keeping idle for a while (5 min in this case). At13:50, one backend server crashed. Another backend server took its place andcontinued to serve robots soon after that.

7 Conclusion and Future Work

In this paper, we firstly defined the tasks of three cloud service types for dis-tributed visual SLAM. IaaS provides basic computation, storage, and networkinfrastructures. It is also equipped with mechanisms of security, elasticity, androbustness. PaaS defines the transmission protocols between robots and thecloud. It also provides isolated computational environments to each robot. SaaSprovides visual SLAM applications for robots in a stateless way.

Besides, we proposed RSE-PF based on three cloud service types anddescribed the implementation method of RSE-PF based on AWS andOpenRestyTM . With the firewall around EC2 instances, the cloud is under secureprotection. With the load balancer, monitor, and scale modifier, the cloud runsrobustly and elastically. The Lua VM created in Nginx provides each robot withan isolated runtime environment. In addition, the visual SLAM applicationsshould be stateless in order to help the cloud replace the broken server withoutshooting troubles.

Moreover, we demonstrated perspective-n-point as a use case to prove the fea-sibility, elasticity, and robustness of our proposed framework. In this use case, theaverage round trip delay is 153 ms, which meets the near real-time requirementof autonomous robots. According to the round-trip delay, we found one bottle-neck of the cloud-based algorithms was the transmission delay and proposed tworecommendations for developers to improve the real-time performance of theiralgorithms.

From this work, we find the stateless requirement of SaaS for cloud roboticsis a little harsh. For visual SLAM, functions like solutions of perspective-n-point,iterative closest point, and place recognition could be processed in a statelessway, but some algorithms like ORB-SLAM are complicated to implement state-lessly. In the future, we plan to implement the PaaS part of RSE-PF with Dockerto handle such complex robotic services. Besides vision, robots are capable oflocating themselves based on WiFi signal strength [15]. It can be implementedas a SaaS application and helps robots locate themselves in indoor scenes.

Page 13: Towards a Cloud Robotics Platform for Distributed Visual SLAM · RoboEarth [6] aims to develop a giant repository and achieve a World Wide Web for robots so that robots are able to

Towards a Cloud Robotics Platform for Distributed Visual SLAM 15

References

1. Liu, M.: Robotic online path planning on point cloud. IEEE Trans. Cybern. 46(5),1217–1228 (2016)

2. Gianni, M., Papadakis, P., Pirri, F., Liu, M., Pomerleau, F., Colas, F., Zimmer-mann, K., Svoboda, T., Petricek, T., Kruijff, G.J.M.: A unified framework for plan-ning and execution-monitoring of mobile robots. In: AAAI Conference on Auto-mated Action Planning for Autonomous Mobile Robots, pp. 39–44 (2011)

3. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accu-rate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)

4. Arumugam, R., Enti, V.R., Bingbing, L., Xiaojun, W., Baskaran, K., Kong, F.F.,Kumar, A.S., Meng, K.D., Kit, G.W.: DAvinCi: a cloud computing framework forservice robots. In: 2010 IEEE International Conference on Robotics and Automa-tion (ICRA), pp. 3084–3089. IEEE (2010)

5. Hunziker, D., Gajamohan, M., Waibel, M., D’Andrea, R.: Rapyuta: the roboearthcloud engine. In: 2013 IEEE International Conference on Robotics and Automation(ICRA), pp. 438–444. IEEE (2013)

6. Waibel, M., Beetz, M., Civera, J., d’Andrea, R., Elfring, J., Galvez-Lopez, D.,Haussermann, K., Janssen, R., Montiel, J.M.M., Perzylo, A., et al.: Roboearth.IEEE Robot. Autom. Mag. 18(2), 69–82 (2011)

7. Mohanarajah, G., Usenko, V., Singh, M., D’Andrea, R., Waibel, M.: Cloud-basedcollaborative 3D mapping in real-time with low-cost robots. IEEE Trans. Autom.Sci. Eng. 12(2), 423–431 (2015)

8. Beksi, W.J., Spruth, J., Papanikolopoulos, N.: CORE: a cloud-based object recog-nition engine for robotics. In: 2015 IEEE/RSJ International Conference on Intelli-gent Robots and Systems (IROS), pp. 4512–4517. IEEE (2015)

9. Wang, L., Liu, M., Meng, M.Q.-H.: A hierarchical auction-based mechanism forreal-time resource allocation in cloud robotic systems. IEEE Trans. Cybern. 47(2),473–484 (2017)

10. Rahman, A., Jin, J., Cricenti, A., Rahman, A., Yuan, D.: A cloud robotics frame-work of optimal task offloading for smart city applications. In: 2016 IEEE GlobalCommunications Conference (GLOBECOM), pp. 1–7. IEEE (2016)

11. Mell, P., Grance, T., et al.: The NIST Definition of Cloud Computing (2011)12. Lubbers, P., Albers, B., Salim, F., Pye, T.: Pro HTML5 Programming. Springer,

Heidelberg (2011). doi:10.1007/978-1-4302-3865-213. Gao, X.-S., Hou, X.-R., Tang, J., Cheng, H.-F.: Complete solution classification

for the perspective-three-point problem. IEEE Trans. Pattern Anal. Mach. Intell.25(8), 930–943 (2003)

14. Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and supportinference from RGBD images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y.,Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7576, pp. 746–760. Springer, Heidelberg(2012). doi:10.1007/978-3-642-33715-4 54

15. Sun, Y., Liu, M., Meng, Q.H.: WiFi signal strength-based robot indoor localization.In: IEEE International Conference on Information and Automation, pp. 250–256(2014)