[ieee oceans 2009-europe (oceans) - bremen, germany (2009.05.11-2009.05.14)] oceans 2009-europe -...

10
Portable control console for autonomous ocean-going vehicles David Hlaváč, João Tasso Borges de Sousa, Fernando Lobo Pereira Underwater Systems and Technology Laboratory (LSTS) Faculdade de Engenharia da Universidade do Porto (FEUP) Rua Dr. Roberto Frias s/n, 4200-465 Porto, Portugal {hlavac, jtasso, flp}@fe.up.pt Abstract-This paper describes the design and implementation of a portable control console for autonomous ocean-going vehicles and discusses lessons learned from operational deployments. I. INTRODUCTION First we describe the operational requirements behind our design of the portable console named pdaConsole. Second, we discuss the use cases, the statement of design patterns for the development (with special emphasis on the peculiarities of mobile devices) and briefly describe COTS mobile devices that suit the demands. Finally we discuss the actual development results of our pdaConsole and its impact on field deployments. A. C3I infrastructure and field deployments PdaConsole is portable in the sense that it runs on handheld devices, providing an interface and at the same time complementing a larger C3I (Command, Control, Communication and Information) infrastructure – namely the Neptus framework [1]. This mixed-initiative environment, which is being developed at the Underwater Systems and Technology Laboratory (LSTS, [2]) at the Faculty of Engineering of University of Porto (Portugal), is designed to support the coordinated operation of heterogeneous teams of vehicles, sensors and human operators: These include autonomous and remotely operated underwater, surface, land and air vehicles, drifting and moored sensors, as well as humans. LSTS's operations are conducted outdoors, sometimes away from urban areas, thus requiring simple and user-friendly control consoles with long power autonomy (typically there is no AC power available). B. Role of the PDA within a C3I infrastructure The description of a typical operation of an autonomous underwater vehicle (AUV) makes the description of the requirements more tangible. The planning stage includes not only the design of mission plans, but also the specification of the operational environment. For example, when we use acoustic localization devices, the planner has to specify the locations of the acoustic transponders (for the localization of the vehicle). In this case, the first step of the operational deployment concerns dropping the transponders (with a simple mooring scheme) close to their planned positions. Finding these positions can be greatly assisted by a GPS enabled console which has the planned positions. The GPS position of the moored transponders can be easily recorded in the console and transmitted to all mission participants afterwards. The AUV mission starts with the deployment of the AUV in the water. The mission specification is then uploaded to the AUV from the console, which is also used to send the start command. The console stays over the whole mission time in the hands of the supervisors who follow the AUV during its mission in a boat. During the mission execution, the console offers the possibility to send an abort signal to the AUV with the help of an acoustic transponder. The console should provide information on the supervised vehicle state, the mission execution state, the current network connectivity and the availability of other consoles. The console should provide information on the planned future behavior through an interactive map where the parameters of every mission are stored. Even more important than having a roughly control over the vehicle's behaviors (select and upload mission to vehicle; send start and stop command) through the portable console - which is expected to be close to the vehicle most of the time - is having a stable connection to the vehicle while other consoles might not. C. Requirements From the description above we can deduce the requirements we impose on the hard- and software for the portable console: - Wireless connectivity - GPS enabled - Pocket sized for easy stowage - Long lasting autonomous power supply - Touch screen for interaction with software - Highly luminous screen even with sunlight - GUI capabilities - Memory for storing data - Customized software running on the device Any COTS PDA or Smartphone satisfies the stated requirements at least partially. Nowadays, such devices are affordable and easily customizable from the software point of view. Smartphones (with the integrated telephony capability) do not only have a Wi-Fi connection, but also can transmit over GPRS and UMTS/3G. Especially under harsh conditions 1-4244-2523-5/09/$20.00 ©2009 IEEE

Upload: fernando-lobo

Post on 10-Mar-2017

216 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

Portable control console for autonomous ocean-going vehicles

David Hlaváč, João Tasso Borges de Sousa, Fernando Lobo Pereira

Underwater Systems and Technology Laboratory (LSTS) Faculdade de Engenharia da Universidade do Porto (FEUP)

Rua Dr. Roberto Frias s/n, 4200-465 Porto, Portugal {hlavac, jtasso, flp}@fe.up.pt

Abstract-This paper describes the design and implementation

of a portable control console for autonomous ocean-going vehicles and discusses lessons learned from operational deployments.

I. INTRODUCTION

First we describe the operational requirements behind our design of the portable console named pdaConsole. Second, we discuss the use cases, the statement of design patterns for the development (with special emphasis on the peculiarities of mobile devices) and briefly describe COTS mobile devices that suit the demands. Finally we discuss the actual development results of our pdaConsole and its impact on field deployments.

A. C3I infrastructure and field deployments PdaConsole is portable in the sense that it runs on handheld

devices, providing an interface and at the same time complementing a larger C3I (Command, Control, Communication and Information) infrastructure – namely the Neptus framework [1]. This mixed-initiative environment, which is being developed at the Underwater Systems and Technology Laboratory (LSTS, [2]) at the Faculty of Engineering of University of Porto (Portugal), is designed to support the coordinated operation of heterogeneous teams of vehicles, sensors and human operators: These include autonomous and remotely operated underwater, surface, land and air vehicles, drifting and moored sensors, as well as humans.

LSTS's operations are conducted outdoors, sometimes away from urban areas, thus requiring simple and user-friendly control consoles with long power autonomy (typically there is no AC power available).

B. Role of the PDA within a C3I infrastructure The description of a typical operation of an autonomous

underwater vehicle (AUV) makes the description of the requirements more tangible.

The planning stage includes not only the design of mission plans, but also the specification of the operational environment. For example, when we use acoustic localization devices, the planner has to specify the locations of the acoustic transponders (for the localization of the vehicle). In this case, the first step of the operational deployment concerns dropping the transponders (with a simple mooring scheme) close to

their planned positions. Finding these positions can be greatly assisted by a GPS enabled console which has the planned positions. The GPS position of the moored transponders can be easily recorded in the console and transmitted to all mission participants afterwards. The AUV mission starts with the deployment of the AUV in the water. The mission specification is then uploaded to the AUV from the console, which is also used to send the start command. The console stays over the whole mission time in the hands of the supervisors who follow the AUV during its mission in a boat. During the mission execution, the console offers the possibility to send an abort signal to the AUV with the help of an acoustic transponder. The console should provide information on the supervised vehicle state, the mission execution state, the current network connectivity and the availability of other consoles. The console should provide information on the planned future behavior through an interactive map where the parameters of every mission are stored. Even more important than having a roughly control over the vehicle's behaviors (select and upload mission to vehicle; send start and stop command) through the portable console - which is expected to be close to the vehicle most of the time - is having a stable connection to the vehicle while other consoles might not.

C. Requirements From the description above we can deduce the requirements

we impose on the hard- and software for the portable console: - Wireless connectivity - GPS enabled - Pocket sized for easy stowage - Long lasting autonomous power supply - Touch screen for interaction with software - Highly luminous screen even with sunlight - GUI capabilities - Memory for storing data - Customized software running on the device

Any COTS PDA or Smartphone satisfies the stated

requirements at least partially. Nowadays, such devices are affordable and easily customizable from the software point of view. Smartphones (with the integrated telephony capability) do not only have a Wi-Fi connection, but also can transmit over GPRS and UMTS/3G. Especially under harsh conditions

1-4244-2523-5/09/$20.00 ©2009 IEEE

Page 2: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

like on board of a boat, PDAs (in contrast to netbooks or subnotebooks) unveil their competitive edge: the small size is extremely practical – ready to be stuffed into your pocket at any time!

II. DESIGN PATTERNS

All the limitations, imposed by the environment or the device itself, imply special design patterns for outdoor PDA software that is optimized to work with fingertip interaction solely.

A. Hardware Every software works accordingly to the IPO (Input-

Process-Output) model – software for handheld devices is not an exception. Even if we have nowadays the possibilities to input data through communication channels (e.g. Wi-Fi) without user interaction, still we need a user to interact with the software. Every device must have means on the hardware side to feed data or instructions into the software layer in order to process and store them (IPO+S model).

Figure 1 shows a schematic representation of a PDA and Smartphone. Both possess a screen (predominantly touch sensitive) and a joystick for navigation purposes. In the very middle of the joystick there is an additional button which acts as the Enter key. Smartphones have buttons for initiating and terminating a phone call. The remaining space is taken up by multiple hardware buttons which can execute user-defined programs or actions on the device.

Fig. 1: A schematic of a PDA and a Smartphone

Handheld devices as well have a built-in microphone and

even though speech recognition and voice command applications are available for portable devices, the technology doesn’t offer any comfortable way to control the device than to scroll through the list of contacts. Especially in harsh environments (with wind etc.) voice command will likely not to be an option very soon.

A very intuitive and highly useful input method is the touch screen. With its help, devices can provide a so-called soft input panel (SIP) where the user can tap with the included pen on the displayed micro keyboard to send keystrokes to the underlying application.

Fig. 2: Soft Input Panel on a Windows Mobile 5 device

Yet this is a very cumbersome way to type – not only

because the keys are tiny (in order to fit a complete keyboard on the screen), but also because of the virtual necessity to use the touch screen pen. Further down we present an alternative input method for pdaConsole.

B. GUI adaptation Great user-experience and handiness is contradictory to the

necessity to use the pen in order to interact with the program, especially for outdoor applications. Our highest goal from the interaction point of view was to eliminate the necessity to use the pen in order to control the application. This includes the special adaptation of the application's graphical user interface (GUI) featuring GUI controls that allow rapid, easy and innovative forms of user input. The result is a software that works with fingertip interaction solely.

1. Screen orientation By default, PDA and Smartphone screens are in portrait

mode. Smartphones have to be seen as successors of mobile phones which in the beginning had just a numeric pad of oblong shape. By adding a display, this vertical appearance was even underlined. Over time, the displays became bigger or rather longer - extending the height stronger than the width. Displays turned into screens and the numeric pad has almost disappeared by now – giving way to even bigger touch screens. The format of PDA screen has to be considered analogically.

Nevertheless, the landscape mode appears to be more natural to humans, not only because we are used to work with monitors with a width/height ration of greater than 1. The eyes of humans are aligned in a horizontal layer: The field of view is approximately 120° in the vertical direction and almost 180° in horizontal direction [3]: Without eye movement humans can capture a wider range horizontally than vertically. Therefore, the landscape mode is more natural to humans. These are the reasons why we adopted the landscape mode throughout the pdaConsole application.

Page 3: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

On startup, the application rotates the screen into left-hand-landscape mode. The buttons and the joystick are now operable with the left hand; the right hand is untied for other tasks like grabbing of objects, gripping for your own steadfastness or interacting by fingertip with the software.

Holding the device in the left hand and from the left side rather than having the device in portrait mode and holding it with the left hand - frontally at the bottom end - is more comfortable for the user. The muscles are relaxed since all joints and tendons are in their most natural position (the so-called neutral zero position) with the exception of the elbow joint which is flexed. Relaxed muscles consume less energy and therefore add up to longer endurance which signifies that the device can be operated with for a longer time.

2. GUI in general All the windows that are part of pdaConsole are displayed

in full screen (maximized mode). This is favorable, because on the one hand, the windows task bar (which is positioned on Windows Mobile devices at the very top in contrast to regular Windows at the bottom) is not visible and can therefore not be hit or clicked by mistake which would deactivate pdaConsole and unsettle the user. On the other hand, it is an advantage in the sense that each and every window of the console becomes higher by the height of the windows task bar which is a gain by approx. 22 pixels.

Each and every window can define independently functions or actions for every hardware button that is available on the device. Thus, all easily accessible hardware buttons can be the actuators for the most needed functionalities of every window. It is important to give the user the possibility to utilize all available input mechanisms, since mobile devices do not have the rich variety of interaction and input possibilities desktop PCs and notebooks offer with mouse, keyboard and shortcuts.

In order to enable the user to interact with the software by tapping with the fingertip on the screen, the standard font size had to be increased. While Windows Mobile uses 8 pt, the smallest font size used in pdaConsole is 14 pt, headers and lists at least 20 pt.

Buttons and all other click- or selectable components are - not only because of the raised font size - at least 44 pixels wide and high which equals 10 mm on a screen with a resolution of 111 dpi (screen of 72 x 54mm corresponds to 320 x 240 pixels on the Acer n50 PDA). This is roughly the size of a fingertip.

The display is the weakest part of all: With bright sunlight outdoors, it becomes hard to read the screen due to high reflection on the display. Unfortunately, COTS devices which are not designed to be used primarily outside, only allow increasing the brightness of the screen to certain extend. For outdoor purposes high luminosity is desirable.

With weak luminosity or high reflection, colors are almost impossible to distinguish. Therefore only a limited set of colors and high contrasts between back- and foreground (like texts) are used. This helps to retain visibility and readability of the screen even under lurid lighting conditions.

3. GUI controls The Windows Mobile operating system provides menu bars,

context menus and so-called message boxes in which alerts or inquiries are presented to the user who is prompted to respond to these by clicking a button. These GUI controls were primarily designed to take up as little space as possible on the screen – under the assumption that the user selects his choice with the pen. The font size is the regular system font size and therefore compared to what above stated design patterns require in any case too little. The menu items and message box buttons are too thin to be easily hit with a fingertip. Menus often have a blue text color instead of black.

Design patterns for portable outdoor software demand menus with clearly readable labels und easily hittable menu items. The same accounts for message boxes: The message text has to be easily readable even under unfavorable conditions.

As you can see in the following screen shots, menus (including submenus) and message boxes were re-implemented:

The choice buttons of the message box can contain any

user-defined text. It can act therefore as a mini context menu.

The menu window contains a header and at most 3 menu

items and above all a quick access bar. This is a set of small menu items that appear equally on any menu window. They provide access to the most often needed functions. Should the menu window contain more than 3 items, an indicator is shown in the right upper corner and you can use the joystick to scroll through all menu items available. The indicator changes

Page 4: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

depending on whether the user needs to scroll up or down in order to access the remaining menu items. Any menu item can be set to be checkable (for Boolean settings) or contain a submenu: On hit, another menu – the submenu – opens and shows e.g. a more diversified set of options.

Furthermore, a list box control had to be reimplemented in order to comply with stated design patterns. List boxes are used to enable the user to make a choice or to display list-like data. When the number of items in the list box exceeds the number of items which can be visible (due to the set size of the list box), the native list box control displays a scroll bar. Unfortunately this scroll bar has by default a width of 12 pixels (since it was designed to be operated with the mobile device’s pen) which is not wide enough to safely scroll with a fingertip. The pdaConsole list box now has a 44 pixel wide scrollbar and allows scrolling through the list with the joystick’s up and down keys. Pressing the Enter key, confirms the selection of the highlighted item and the program continues with the user’s choice. Additionally every item can contain a second column that displays additional information, e.g. a distance (between the mission participant mentioned in the first column and the PDA) like depicted in the following figure.

C. Input Methods Beside the default SIP (on-screen keyboard) there is no

other possibility to input text. Some recent Smartphone still have a numeric pad but on the other hand, these are commonly not equipped with a touch screen and they are not of our interest.

Even though there is seldom the need in pdaConsole to enter text or data, an input method that is consistent with stated design patterns has to be implemented, since the existing SIP isn’t satisfactory.

PdaConsole’s co-called input box which prompts the user to

make an input has two modes. The following figure depicts the numeric mode.

The user can enter digits, a minus sign and a decimal separator. This mode is used to enter the depth of the moored transponder (for acoustic localization). It can be used to ask for a latitude/longitude value or an IP address of a mission participant (in order to send data over Wi-Fi).

The other mode called “slide-and-append” is the mode that is of higher interest: The input box displays now instead of 12 keys in the numeric mode, a slightly modified version of a horizontal scrollbar and some few control buttons.

The user uses his right index finger to slide the moveable

part of the scrollbar. With every move, the selected character in the textbox changes as well as the text of the Append button where always the currently selected character is displayed. When the volitional character has appeared, he taps the Append button or simply actuates the Enter key of the joystick with his left thumb and the cursors moves on. By default the last added character appears as well as the new currently selected character: Words containing two or multiple subsequent same characters can be entered quickly.

During the whole input process he holds the device like described before and is barely moving his thumb on the joystick with which he can also capitalize or un-capitalize (with the up and down button) the currently selected letter and move the cursor back- and forward.

Depending on the length of the alphabet the input box was set to allow, the accuracy is better or worse. With the default value of “a” through “z” (capital letters can be generated from

Page 5: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

the lower case letter by actuating the joystick) plus digits from 0 to 9, with a width of the scrollbar of 300 pixels and a moveable block of 44 pixels (the minimal size for clickable objects like stated above) an accuracy of (300 - 44) / 36 = 7 pixels per character which is quite agreeable. It goes without saying that this input method takes longer than the traditional SIP and requires some practice, but its concept is straightforward and especially in unfavorable environments it is a valuable alternative.

Frankly, an alphabet of length 36 is practically the upper bound in terms of user-experience and friendliness. Another problem is that the user cannot loop through the alphabet in the sense that after “z” the character “a” reappears. To get from “z” back to “a” he has to scroll back to the very beginning; however this can be achieved very quickly with one wipe.

All these issues could be solved by the following design: Instead of a scrollbar we have a wheel control. (The images are in grayscale to show that this is not a currently existing feature of pdaConsole.)

No matter how many elements our alphabet would contain,

the accuracy would be fixed, e.g. 1 character per 30 degrees. We could scroll through the alphabet forward or backwards by moving the right index finger in circles on the wheel clockwise or counterclockwise. This design has not yet been implemented in pdaConsole, because it is difficult to incorporate wisely: A disadvantage is the relative great amount of screen area that is consumed by the wheel which may of course not be too little. Compared to a simple scrollbar with which still a header and a large text box (to view the entered value) can be positioned, there would be the need to relinquish the header and position the text box awkwardly. An elegant approach is to make the wheel virtual and invisible so it would not waste any valuable space on the screen, however the actual user-experience remains uncertain.

III. PDACONSOLE IN ACTION

The development of pdaConsole was started in May 2008 and the first tested version was released 7 months later. PdaConsole is written C# (version 3.0) with Microsoft .NET Compact Framework version 3.5. The software runs on any Windows Mobile Pocket PC 2003 (or later, which includes as well Windows Mobile 6 - the latest version of Microsoft’s handheld operating system).

A. Why developing a portable console? In LSTS’s mixed-initiative operation scenarios, the most

integral part is Neptus. This console is written in Java and therefore can be executed with any operating system for which a Java Runtime Library is available. Neptus is used for designing, supervising and reviewing/analyzing missions. Before pdaConsole was developed, Neptus was the only source from where to obtain information on the vehicle’s state before and during mission execution. This is difficult especially for the supervision crew members who follow the vehicle during mission execution with a boat and who have their hands on the AUV the remaining time, but do not have any information about its state. Therefore they were bound to talk back to the base over radio. A portable console which they would take along into the boat was a promising idea.

This console would communicate directly with the vehicle and therefore would not be dependent of the presence of Neptus. Before and while the vehicle is executing, it is of greater importance to keep the supervising personnel informed about the state of the vehicle, especially in the case of partial system failure: All information that can be provided by the vehicle at that time can be valuable in order to track the issue and possibly to regain full functionality. Just then, the base and therefore Neptus might be out of reach and would not have any up-to-date information of value. Only a PDA close enough to the vehicle could communicate directly with the vehicle by an ad-hoc or a mesh network and provide the information needed.

Page 6: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

Fig. 3: C3I infrastructure with PDA communication schematic

The following graph shows the distance between PDA and

an AUV over time:

The PDA accompanied the mission supervisors who follow

the AUV during its mission execution in a boat. The following picture shows the supervision crew in the boat together with an AUV (LSTS’s LAUV blue, see [2]).

The maximal distance that was logged over a series of missions (that were run during one day in the field with a total net execution time of 6456 seconds; average mission execution duration was approx. 4 minutes) was 259 meters; the average distance was 67.07 meters.

The PDA is closer to the vehicle than 55 meters over 50% of the total mission execution time as the blue dotted line indicates. 80% of the time the PDA is closer than 100 meters from the AUV. With no high obstacles like buildings between PDA and access point, Wi-Fi connections can be maintained active even over a distance of 100 meters.

A PDA is therefore an adequate and affordable device at the same time to run a portable console.

B. Devices used An Acer n50 PDA was already part of the LSTS lab’s

inventory for a long time before the pdaConsole project was initiated. This PDA is running Windows CE (2003 version). Even though this PDA was introduced to the market in 2005, it is still sufficient to run pdaConsole. Therefore a backward compatibility to Windows Pocket PC 2003 is of great importance. The Acer PDA has a large 3.5 inches display (72 x 54 mm) which is favorable for outdoor applications. With a height of 120 mm, a width of 70 mm, a depth of 17.4 mm and the weight of 150 grams, the PDA is still handy and very robust.

With its built-in Wi-Fi connectivity it can communicate with other mission participants, like Neptus or any LSTS vehicle, over IMC (see [4] and [5]), a protocol that is used to exchange data (so-called messages) between all members of LSTS’s C3I infrastructure.

An external GPS receiver can be connected by Bluetooth or plugged into the existing Compact Flash slot which is more advantageous option in terms of power consumption. On the down side, a Compact Flash GPS card would increase not only the height of the whole gadget by at least 40 mm but also the weight. Moreover, the balance points gets shifted further to the right side (in left handed landscape mode like described above), making it more exhausting to hold the device with the left hand. A Bluetooth GPS device could be placed somewhere close to the steering wheel of the boat – no increase in size, weight and no decrease in handiness, user-experience and reception quality.

A SD (Secure Digital) card slot offers the possibility to insert a SD memory card for data storage, if necessary.

The more recently acquired handheld device is the

Smartphone Asus P535 which is running Windows Mobile 6 Professional. This device was selected specifically based on the needs for pdaConsole. Being a Smartphone, it can establish a GSM connection. Moreover it has Wi-Fi connectivity and an internal GPS receiver. The GPS signal strength and accuracy can be improved by connecting an external GPS antenna (which are also available in sizes smaller than the Smartphone itself) to the provided GPS antenna connector in the back casing.

Page 7: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

The display is slightly smaller than the Asus screen: only 2.8 inches (60 x 45 mm). Both devices have the same amount of pixels, namely 320 x 240 pixels. Nevertheless, pdaConsole is designed to work with any screen size and automatically adapts the size of all GUI control appropriately.

The dimensions of this device are 109 x 59 x 19 mm, weighing 145 grams. Thus it is smaller by 1 cm at least from all sides, but slightly deeper. The weight difference is not noticeable.

The internal power supply can persist approximately 3 hours with GPS and Wi-Fi turned on which is sufficient for our needs. Nevertheless a spare battery is recommended to be charged (at the base where AC power is available) while the other one is used in the device in order to allow swapping.

A Micro SD card slot allows data storage on a Micro SD memory card.

The built-in capability to vibrate turns out to be very useful when the application prompts the user for interaction, e.g. with message and input boxes.

C. Have a look at pdaConsole PdaConsole provides information about the vehicles, their

state and the mission map. The supervision personnel can start and abort a mission directly from within the application without an interaction needed from the side of Neptus. This is in particular very useful if an unforeseen situation occurs. PdaConsole implements a mission management interface that is capable of receiving mission specifications from and sending them to other mission participants. It is therefore considered a light weight console due to the lack of possibilities to design missions and perform mission review and analysis. Nevertheless it is an important part of LSTS's C3I infrastructure and adds value to the whole concept by serving mission related data in a pocket sized format. The application was thoroughly tested and is being used now with every mission. The following screen shots depict pdaConsole’s functionalities. This is how pdaConsole presents itself at application startup:

After startup the map view is displayed. The map shows the

supervised vehicle (solid triangle shape; with multi-vehicle operations all other vehicles are displayed only as an outline), your own position, the home reference, locations of the

transponders, a north arrow, a scale and the areas that are not accessible by the AUV (grey shapes called surroundings).

By default, the north arrow points up, and the map has a

scale of 100 pixels equal 100 meters. The map can be rotated and panned by dragging with the fingertip. The map can be zoomed in until the scale is of 100 pixels equals 12 meters which is 12 cm equals 1 pixel. A more detailed map than this is not required. The map permits zooming out to such extent that practically all seven continents can be displayed.

This is the mission interface with no mission loaded. A mission can be loaded by clicking the button at the bottom or on transmission of mission data from another mission participants by Wi-Fi.

After the mission was loaded, pdaConsole displays the

properties of each maneuver the mission contains.

Page 8: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

From the mission interface menu (which is accessible like on all screens by pressing the enter key of the device’s joystick), the user can start a wizard that helps him in finding the locations where transponders should be dropped as well as saving the actual locations of the placed transponders after dropping and reporting back the relocation to all mission participants.

Returning back to the map view, where now the mission plan has been drawn after mission load, the user now can use the menu of the map window to send the mission start command to the vehicle. (If the vehicle doesn’t have the up-to-date mission specification available, it can be uploaded from the PDA to the vehicle beforehand.)

The AUV now performs the mission. The AUV vehicle tail

displays always the last 100 locations of the vehicle and imparts a notion where the vehicle passed by and how well it follows the planned track and maneuvers.

Every maneuver is represented on the map by a dot that is

labeled with the abbreviation for the maneuver that is executed at that location. If there is a tolerance radius attached to a maneuver, this radius is drawn as a dotted circle around the corresponding maneuver. On click, most objects display an info box with at least the location as a latitude/longitude value pair. Maneuvers display their partial specification.

At any time, the user can consult the state screen which

shows the state of the vehicle’s sensors.

By pressing on a field, all the data that is available for the

selected sensor group is displayed. An indication when data was received the last time from the sensor gives the user an idea how up-to-date the displayed data is.

PdaConsole can guide you to any location as well as to the

current vehicle location and important points that are part of the mission like the navigation startup point. There, the vehicle is deployed into the water from where it starts its mission. The “Guide-To” screen shows the distance to destination and the abbreviation of the type of destination. The compass uses the GPS’s course over ground value to point the arrow into the direction you have to follow in order to reach the destination. Of course, distance and direction get updated automatically on destination or the device relocation. The button is used to change the destination manually or enter a latitude-longitude value pair.

Page 9: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

By default, depending on the current mission state pdaConsole guides to the navigation startup point (before mission execution) and the vehicle’s location (during and after mission execution or on abort) in order to recover the vehicle afterwards.

At times, when the device is not in movement (when the

supervision boat is floating), the GPS device cannot provide accurate course over ground angles. Therefore the compass offers a “fixed” mode: The user can tap on the screen in order to fix the position of North to the point selected – the course over ground value is not taken into account until the user taps the screen for a second time to unlock the compass and thus return to the standard mode.

Those are the main functionalities that pdaConsole

provides. Behind the main windows there are many little more helpful functions that provide a great user-experience to the user.

D. Impact on field deployments With the help of pdaConsole, mixed initiative operations

became - especially for the supervision personnel – more comfortable. Far less often, the crew has to talk back to the base in order to obtain information about the vehicle: All data that is important for their work can be found on the display of pdaConsole. Everybody who is working directly or indirectly with pdaConsole is happy and convinced that developing pdaConsole was a step into the right direction and has to be continued in order to make field deployments increasingly effective.

E. Future development PdaConsole was primarily developed to be used in AUV

missions. Yet, with the IMC communication layer, it is completely transparent to pdaConsole with which vehicle the operation is run and therefore can be used in any mixed initiative operation. Still, there are minor parts that are specific to AUV or present data that can only be retrieved from AUVs like CTD or salinity data. The aim is to provide specific data as well for e.g. ASVs and UAVs.

PdaConsole currently can display a set of vehicles on the map, if they are part of the mission, but it supports only one vehicle whose state data is displayed. For all other vehicle than this supervised vehicle, only the up-to-date location is displayed on the screen in order to get a rough idea where other mission participants are located. For the future there is the need to enhance pdaConsole in order to monitor all data from all vehicles in range.

Currently a Wi-Fi network with one access point is used for communication between mission participants. This access point is located next to the base at the shore or on the quay. Even though the distance between pdaConsole and vehicle is just a few meters, both the PDA and the vehicle have to communicate through the access point with each other. The PDA being the device with the weaker Wi-Fi antenna, this communication schema becomes unfavorable as soon as the distance between PDA and the base is greater than the distance between PDA and vehicle. The PDA loses connection to the access point and no more can receive or send data until the connection is reestablished, even though the vehicle, whose data should be displayed, is close by. This issue can be solved by using mesh networking instead of access point based network. This change would be affect all LSTS’s C3I infrastructure members and is a step that will be concluded soon.

In the open sea, for now we can use an ad-hoc network between vehicle and PDA but with multiple vehicles this is not an option. A mesh is the only satisfying solution for all situations and is therefore a high goal of all.

Beyond that, there is the idea to communicate to the base by GSM. When vehicle and PDA are far away from the base where neither PDA nor vehicle has a connection to the base, the PDA could (if there were still a GSM network available) report sporadically e.g. the position or some other user defined data of the vehicle back to the base using GSM.

Currently, pdaConsole can be run only on Windows Mobile devices. There is the idea to make the application platform independent. Thanks to the Mono project (see [6]), which enables to run C# on all platforms, we could soon have running pdaConsole on Linux handheld devices as well. Even more interesting is the possibility to deploy pdaConsole to Apple’s iPhone with Mono. An iPhone has an extra ordinary large screen and is a very handy device. With its built-in GPS antenna, Wi-Fi connection and 3G is suits perfectly the requirements and promises sustainability for the future. The iPhone’s integrated acceleration sensors would improve the

Page 10: [IEEE OCEANS 2009-EUROPE (OCEANS) - Bremen, Germany (2009.05.11-2009.05.14)] OCEANS 2009-EUROPE - Portable control console for autonomous ocean-going vehicles

accuracy of the device’s course over ground shown on the map and used for the computation of the “Guide-To” indicator.

ACKNOWLEDGMENT

I would like to thank Paulo Dias, José Pinto and Ricardo Martins for their support and their highly valuable suggestions; Luís Madureira and Alexandre Sousa for enabling me to be as close as possible to the AUV during test missions; and Bruno Terra for continuing the pdaConsole project.

REFERENCES [1] Paulo Sousa Dias, Rui M. F. Gomes, José Carlos Pinto, Sérgio Loureiro

Fraga, Gil M. Gonçalves, João Borges Sousa, Fernando Lobo Pereira, “Neptus - A Framework to Support Multiple Vehicle Operation”, Oceans05Europe, Brest, France, 20-23 of June, 2005, (ISBN: 0-7803-9104-7, IEEE Catalog number: 05EX1042C), 2005

[2] Underwater Systems and Technology Web Page, http://whale.fe.up.pt, 28 of March, 2009

[3] Klaus Golenhofen, Basics of Physiology, 2nd edition, Urban & Fischer, ISBN: 978-3437424816, p. 420

[4] Eduardo R. B. Marques, Gil M. Gonçalves and João B. Sousa, "The use of real-time publish-subscribe middleware in networked vehicle systems", 1st IFAC Workshop on Multivehicle Systems (MVS'06), Salvador da Baia/Brazil, October 2-3, 2006

[5] Eduardo R. B. Marques, Gil M. Gonçalves and João B. Sousa, "Seaware: A Publish/Subscribe Communications Middleware for Networked Vehicle Systems", 7th IFAC Conference on Manoeuvring and Control of Marine Craft (MCMC'06), Lisboa, Portugal, 20-22 Sept., 2006

[6] Mono Project Web Page, www.mono-project.com, 28 of March, 2009