matlab real-time workshop-based internet accessible robot

Matlab Real-Time Workshop-based Internet Accessible Robot Control Laboratory

REV 2007 - www.rev-conference.org 1


Zdenko Kovačić, Nenad Smolić-Ročak, Siniša Dujmović, Ranko Munk University of Zagreb, Faculty of Electrical Engineering and Computing

Unska 3, 10000 Zagreb, Croatia

Abstract—In this paper an implementation of the Internet accessible robot control laboratory based on the use of the Matlab Real Time Workshop is described. The functionality of the developed system is illustrated by Internet-based remote control of a four degrees of freedom SCARA robot.

Index Terms— Remote control, robot control, Internet, Matlab Real Time Workshop

I. INTRODUCTION The aim of remotely controlled robotic systems is to

execute a task given to a robot without a physical contact of a robot operator with an environment the robot works in. There are numerous applications of such robot systems, some of them dating from early days of robotics, including manipulations in radioactive surroundings, explosive and poisoned atmospheres, and other locations unsafe for human beings [1]. Nowadays applications of remotely controlled robotic systems include, for example, remotely operated robots for surgical operations, robots - home assistants, nursery robots and very recently, entertainment and therapeutic robots [2].

With the development of advanced e-learning systems, more and more attention has been paid to remotely operated robot systems used in education. The purchase of rather expensive robotic equipment is always a problem for educational robotic laboratories. Eventually, this becomes a serious constraint in organization of a practical work with a large number of students enrolled in the robotics curriculum. The use of server-client communication via the Internet enables a completely different organization of laboratory activities allowing a whole day long access to the robotic laboratory equipment [3, 4]. By providing feedback information from the sensors mounted on the real robot via the same Internet channel, a full visualization of a robot arm motion becomes possible (including 3D motion of a virtual 3D model of a robot or a picture of robot motion acquired from a remote supervisory camera) [5].

One of advanced features of modern simulation tools is that they allow users to generate a real-time executable code directly from the simulation model. This code can be further downloaded into dedicated hardware systems. This enables fast validation and shortens product development time. In addition, by introducing new control solutions in the form of ready-to-use function blocks prepared for

popular software packages such as the Matlab+Simulink, they can be brought to a wide academic and industrial community in a very convincing and representative way. In this paper the Matlab Real Time Workshop (RTW) is used as a backbone of the Internet accessible robot control laboratory. For this purpose server and client function blocks for Simulink were built as a connection to the Internet and target robot control hardware, respectively.

This paper is organized in the following way. First we describe a concept of an implemented Internet accessible robot control system. Then we describe the way how server and client applications exchange information including reference data, sensor data and a picture of a robot from a remote supervisory camera. Implementation on a standard PC platform is presented. We conclude the paper with some ideas and directives for a future work.

II. CONCEPT OF THE REMOTE CONTROLLED ROBOT SYSTEM

The implemented remote control robot system shown in Fig. 1 consists of the following hardware:

1. One server PC 2. One or more client PCs connected through TCP/IP

with Matlab-Simulink server. 3. A 4 DOF SCARA robot arm Kiwi driven by the

modular motion control system Ultimodule. 4. One web camera connected to the server. Each client is connected with the Matlab-Simulink

server function block via the Internet providing that a client PC supports socket communication. According to a first come-first serve policy, one client becomes a master that can receive current robot positions and set new robot positions, while other clients can only observe. On the other hand, the Matlab-Simulink client function block is connected through the TCP/IP to the server running in the robot controller.

Feedback signals from robot joint position sensors (encoders) are fed to the Matlab Real Time Workshop through the TCP/IP and they become available for any connected PC client. New robot positions are acquired either from the master client or inside the Matlab-Simulink control scheme. A camera connected to the server is used for monitoring of a tele-operated robot Kiwi, and its picture is also sent through the Internet to all connected clients.



Figure 1. The concept of the Internet accessible robot control laboratory

Figure 2. The web page of the master PC client.

The software part of the implemented system consists of a client web page (see Fig. 2) with Java applet for setting joint positions or tool center positions. The client web page also contains an interactive VRML-based GUI and the picture from the supervisory web camera. Matlab Real Time Workshop (RTW) encompasses the TCP/IP server application implemented as a Simulink function block, the model of a 4-axes robot motion controller, and a client application implemented also as a Simulink function block. The client function block communicates with the Ultimodule robot controller via the TCP/IP protocol, while data is exchanged using Windows Sockets.

III. ULTIMODULE ROBOT CONTROLLER Ultimodule platform based on the Smart Controller

Module (SCM240) executing real-time operating system eCos was used to develop the Ultimodule robot controller. Key features of the Ultimodule platform are:

1. Support of a large number of different IO modules, 2. 64-bit 200 MHz RISC processor, 3. Real-time operating system eCos 4. Different communication devices (Ethernet, CAN,

RS232)



The implemented Ultimodule robot controller, shown in Fig. 3, consists of the following components:

1. Robot controller, 2. Communication module. The robot controller executes joints control algorithms

and does periphery information processing. Due to the joints control algorithms execution, it is necessary to execute robot controller with a fixed control time interval Td. The real time operating system eCos makes implementation of such thread possible.

There are two robot controller operation modes: 1. Local robot control mode, 2. Remote robot control mode. In the local robot control mode, position and speed

feedback paths are closed inside the robot controller, and set-points are received from a client. Main advantage of this mode is that there are no problems with communication delays, but implementation of new control algorithms is more demanding and time consuming.

In the remote robot control mode, position and speed feedback paths are closed by Matlab-Simulink through TCP/IP network. In this mode the robot controller is only used to read inputs and set outputs. The main advantage of this mode is easy implementation of new control algorithms in Matlab-Simulink. The main disadvantages are caused by communication delays in the network. A solution, implemented here, is to use two nodes network (PC equipped with Matlab RTW+Simulink and Ultimodule robot controller) since this avoids packets collisions and interferences with other network traffic.

The communication module executes the TCP/IP communication routines. The main component of the communication module is a server, which accepts connections from the clients and creates separate client threads for each connected client. A maximal number of connected clients is limited due to hardware limitations and performance considerations. Each client thread is processing client requests. The client sends requests as string commands (see Table 1).

Client privileges are determined according to the robot controller operation mode. In the local robot control mode privileges are determined according to the connection order – only the first client to connect has privileges to move the robot (e.g. set commands) and all other clients cannot do that. In the remote robot control mode, only the Matlab client has privileges to move the robot. Regardless of the control mode, all connected clients can read current robot controller states.

TABLE I. BASIC CLIENT COMMANDS

Client commands Server answer Description GetPos(); Pos(q1,q2,q3,q4) Reads current joints

positions GetSpd(); Spd(w1,w2,w3,w4) Reads current joints

speeds GetDigIN(); DigIN(w) Reads digital inputs SetPos(q1,q2,q3,q4); error(error_code); Sets new set-points

for joint controllers SetDigOUT(w) error(error_code); Sets new values to

digital outputs SetAnaOUT(w1,…); error(error_code); Sets new values to

analog outputs

Figure 3. Ultimodule robot controller organization



In the remotely controlled systems, safety is one of the main concerns. In the Ultimodule robot controller two aspects of system safety are considered - software and hardware.

The first level of software safety is a protection in case of communication problems. The client who has control, sends command in every client control interval (e.g. command GetPos()). If that command is not received within a defined time-out period, all robot joints are stopped paying attention to joints acceleration and speed limits. This level of safety is implemented in such way because it is assumed that if command was not received in the defined time-out period, either the client was deadlocked or the connection was not terminated in a proper way. The consequence is that the client is not able to control the robot any more, so the system should go to the safe state (the robot is stopped).

The second level of software safety is achieved by defining software-based position limits for each robot joint. In case that the user tries to send the robot outside of the defined working area, the robot controller will keep new joint set-points within the defined position limits.

The purpose of hardware safety is to prevent accidents in case of software malfunction. Hardware safety is ensured by mounting mechanical limit switches to each robot joint. Limit switches disable joint drives if joints tend to leave a permitted working area.

From the user’s point of view, the main difference between software and hardware safety activation is that in the software safety activation case the user can still move robot remotely, while during hardware safety activation the intervention of the laboratory operator is needed.

IV. SERVER AND CLIENT FUNCTION BLOCKS FOR THE MATLAB-SIMULINK RTW

In order to achieve the connectivity of Matlab with the Internet on one side and with the robot controller on the other side, server and client function blocks for Simulink were developed.

The client block connects Matlab-Simulink with the robot controller. In order to establish successful communication, it is necessary to initialize the system. As the first step, it is necessary to execute a hard-home (HH) procedure. This is because all robot joints are equipped with incremental optical encoders and on the robot controller startup, current positions of robot joints are unknown. After a successful HH procedure, the client sets the robot controller to the remote robot control mode. In this mode, it is important to stop all robot joints in case of communications problems. Otherwise robot could exit a working area, without activating software limits, and that could lead to an accident. In that order to ensure the safety the Ultimodule robot controller requires that the Matlab client sends commands in each control interval.

The server block connects Matlab-Simulink with other clients. Inside the server block, user authorization, user privileges determination, and interaction with Matlab-

Simulink model are implemented. It is important to mention that communication delays between the Matlab server and other clients do not affect control quality of the robot.

The masks of the client and server blocks (see Figs. 4 and 5, respectively) show the parameters which must be set by the user. In case of the client block, this includes the IP address of the Ultimodule robot controller, a port number, a control interval Td, a number of analog inputs sent to the server and the number of joint positions and velocities acquired from the server (Fig. 4). In case of the server block, its mask includes a port number of the server, a control interval Td, a number of expected reference joint positions from the PC client, and the number of joint positions and velocities to be sent from the server (Fig. 5).

The layout of the server-client based control scheme implemented in the Matlab RTW is shown in Fig. 6. Such a configuration allows the user to generate own references for the robot joints and to use all available features of Matlab presentation of results. Changes of controller parameters can be done in an on-line manner without interrupting the closed loop control of the robot. This is especially convenient while carrying on various experiments with different types of servo control algorithms (e.g. robust, adaptive, and intelligent).

Figure 4. Client function block for Matlab-Simulink.

Figure 5. Server function block for Matlab-Simulink.

u_a

u_d

E_c

y_a

y_d

S_c

client_f

Client

u_a

u_d

E_s

y_a

y_d

T_r

S_s

server_f

Server



Figure 6. Control scheme implemented in the Matlab-Simulink Real Time Workshop

V. SOLUTION OF THE SYNCHRONIZATION PROBLEM Since execution of the Real Time Workshop (RTW) is

closely related to the Windows operating system, which by default is not a real-time system, synchronization of the Matlab RTW-based robot controller and the rest of the system hardware becomes an important issue. In order to be able to use RTW with the selected Ultimodule robot controller trough TCP/IP network, instead of running RTW in the Real Time Windows Target (RTWT) mode, the implemented system works in the so called Generic Real Time Target (GRTT) mode. Namely, the fact is that the implemented Simulink server and client blocks use Windows operating system calls for the TCP/IP communication. On the other hand, the Matlab RTWT is running a Real Time Windows Kernel, which intercepts messages form Matlab thus disallowing any system call from the Simulink function blocks.

Usage of the GRTT mode eventually resulted in the variable time of control algorithm execution (Ta) due to inevitable interrupts of the control algorithm caused by concurrent Windows applications. This problem would lead to a significant drift of the control interval duration which is not permitted. The problem of synchronization was solved by recording the time needed to process the control algorithm in every control interval and by estimating the remaining time (Tsleep) until the end of the control interval Td. Closely tied to the tick frequency, each control interval is reset at the start, thus preventing the accumulation of time drift (see Fig. 7).

Figure 7. Solution of the RTW GRTT synchronization problem.

VI. EXPERIMENTAL RESULTS The described remote robot control system was tested

in the Laboratory for Robotics and Intelligent Control Systems (LARICS) at the University of Zagreb, Faculty of Electrical Engineering and Computing. The results of one such experiment with proportional (P) controllers of joint positions and proportional-integral (PI) controllers of joint velocities are shown in Fig. 8. As one can see, the system fully exploits the advantages of Matlab (e.g. to show the joint positions and joint velocities responses), as well as the advantages of the Internet-based approach (e.g. by displaying the robot motion captured by a remotely installed web camera).



Figure 8. Experiments with the Internet accessible robot control laboratory based on the employment of the Matlab-Simulink Real Time Workshop

VII. SYSTEM IMPROVEMENTS The presented Internet-based robot control system using

Matlab-Simulink and RTW has been designed to enable only one user to control the robot. Thinking of an Internet accessible laboratory which should enable different users (for example, a group of students) to control robot and keep records about their work, the developed system should be improved by implementing a procedure to add new users. This should also include changes of user access privileges and logging of user actions. All these elements should be integrated during the setup of some Web-based laboratory [6]. In order to raise the performance of the developed system, its concept shown in Fig. 1 should be modified in a way shown in Fig. 9. Because of easier system maintenance, the system is divided in three main components - Web application, database and controllers in the Matlab environment.

Web application is the central part of a user handling system, which is processing user's actions and dispatching them to proper subsystems. Web Application utilizes two interfaces: SQL API and Matlab API. SQL API establishes a connection to the database, while Matlab API translates user actions to proper commands according to the user privileges. It can also generate short knowledge tests, with questions attained from the database, which could be a prerequisite for getting permission to control the robot.

The database stores all user data and keeps record of user privileges. Also, the aim of database is to keep records about users’ actions and system results. Under such organization the user management becomes easier.

By adding the database and the Web application, the user management is removed from the Matlab RTW controller, and tasks of the Matlab RTW controller are only to control the system and collect data from the system. Additional advantage of the modified Web-lab concept is that it would be possible to add different control systems besides the described remote robot control, and thus make those other systems also accessible via Internet.

Matlab

Kiwi

Matlab

Pendulum

Internet

Browser

TCP/IP

HTTP

Database

JConnector

Web ServerWeb Application

Figure 9. Extended concept of Internet accessible robot control

laboratory



VIII. CONCLUSIONS New information and communication technologies

enable development of remote controlled robotic systems in a completely new way. The World Wide Web, i.e., Internet, is connecting millions of users worldwide. This can be used in a highly effective way for building educational laboratories accessible via Internet. In the presented robot control laboratory, the Internet-based network structure allows control and supervision of a remotely placed robot to an arbitrary number of users.

There are two categories of users (clients) – the master or robot operator that is authorized to control a robot (this status is in the presented system deserved on the first come-first serve principle) and observers, which can only monitor a robot motion. Such a concept allows further developments in several directions – towards implementation of new control configurations in the Matlab RTW, towards better assessment of user's knowledge, and to better organization of laboratory exercises when a large number of users are involved.

The safety issues are also very important, as well as the consideration of many side effects that come with the use of Internet (the influence of the speed of Internet connection, how to enforce the rules of a time-distributed use of the system etc).

ACKNOWLEDGEMENT The work described in this paper was performed within

the framework of the project entitled "VIREMCO – Virtual and Remote Motion Control Laboratory“ which was supported by a grant from the Sitek S.p.A., Verona, Italy, and by grants from the Ministry of Science, Education and Sports, Republic of Croatia.

LITERATURE [1] T.B. Sheridan, "Telerobotics, Automation and Human Supervisory

Control.", Cambridge, MA: MIT Press, 1992, p.107. [2] C. Preusche, T. Ortmaier and G. Hirzinger, "Teleoperation

concepts in minimal invasive surgery", Control Engineering Practice, Volume 10, Issue 11, November 2002, pp. 1245-1250.

[3] K. Goldberg, M. Maschna, S. Gentner, et al., "Desktop Teleoperation Via the WWW", Proc. of the IEEE International Conference on Robotics and Automation, pp. 654-659, Japan, 1995.

[4] N. Xi and T. J. Tarn, "Stability analysis of non-time referenced Internet-based telerobotic systems", Robotics and Autonomous Systems, Volume 32, Issues 2-3, 31 August 2000, pp. 173-178.

[5] G. Genter, R. Munk, Z. Kovačić, "Internet-Based Teleoperation of a Robot Arm by Using a Camera and a 3D Robot Model for Guidance and Visualization", Proceedings of the 12th Mediterranean Conference on Control and Automation, Kusadasi, 2004.

[6] J. Garcia-Zubia, D. Lopez-de-Ipina, U. Hernandez, P. Orduna, I. Treba, " An Approach for WebLabs Analysis", International Journal of Online Engineering, Volume 3, No. 2, 2007

AUTHORS Zdenko Kovačić is with the University of Zagreb,

Faculty of EE&C, Unska 3, 10000 Zagreb, Croatia (E-mail: [email protected]).

Nenad Smolić-Ročak, is with the University of Zagreb, Faculty of EE&C, Unska 3, 10000 Zagreb, Croatia (E-mail: [email protected]).

Siniša Dujmović is with the University of Zagreb, Faculty of EE&C, Unska 3, 10000 Zagreb, Croatia (E-mail: [email protected]).

Ranko Munk is with the University of Zagreb, Faculty of EE&C, Unska 3, 10000 Zagreb, Croatia (E-mail: [email protected]).

matlab real-time workshop-based internet accessible robot

Documents