Coordinating Communication in Human-Robot Task Collaborations

Aaron St. Clair and Maja Matarić

Interaction Lab, Computer Science Department, Viterbi School of Engineering University of Southern California

Los Angeles, California, USA {astclair, mataric}@usc.edu

Abstract—In this paper we study how coordinating communication during human-robot task collaboration can be used to improve in situ decision-making and team performance. The problem of generating communication actions is formulated as a planning problem compatible with a class of pair-wise loosely-coupled tasks and uses the notion of role assignment to guide the robot's communication actions. We developed an approach for planning three different types of verbal feedback using a Markov decision process representation of the task environment and a set of policies for representing agent roles. We conducted a pilot experiment that compared human-human and human-robot collaborative task performance. A study designed to experimentally validate the approach is currently being conducted with n=20 users to determine if the generated communication can quantitatively improve team performance and qualitatively improve user experience.

Keywords- human-robot collaboration; communication; human-robot teamwork;

I. INTRODUCTION We consider the human-robot interaction (HRI) problem of

a robot generating useful communicative feedback in the course of human-robot task collaboration. As personal service robots become increasingly more competent, capable of performing broader varieties of tasks, and working in human-centered environments, there will be an expanded need for effective methods for collaboration. Rather than requiring users to gain competence in operating autonomous robots via a screen-based interface, we seek to enable robots to use human-like coordination mechanisms, specifically embodied communication. We hypothesize that a robot teammate will make a more effective work partner and improve quantitative measures of team performance when supporting the natural coordination mechanisms people use in working with each other. Further, allowing users to interact with the robot as a teammate rather than an operator will partially offload the burden of coordination from the user, hopefully resulting in quantitative improvements in performance.

We consider the problem of the robot producing communication actions to a person with whom it is interacting, in order to provide feedback and support in situ decision-making. First, we describe design goals for the robot's communication actions, and the formulation of the problem.

Next, we summarize implementation details and describe the pseudo-herding task used to validate the system. We present results from a pilot data collection involving person-person and person-robot teams that provided domain knowledge and were used to motivate the types of communication actions the robot uses in the task. Finally, we discuss initial results from an on-going user study validating the approach with human-robot teams.

II. PROBLEM Social science literature indicates many types of human

communication behavior used during collaborative tasks, including attentional cues to indicate an area of focus, staging actions to maximize shared visual information, gestural and speech cues indicating intentional goals or instructions, and coaching actions such as feedback, encouragement, and empathetic displays to build team rapport. Effectively producing all of these communication actions on robots, in real-world task environments, is not currently feasible and would be difficult to generalize across different robot embodiments; for example, indicating attentional focus is different with humanoid and non-humanoid robots. Based on a review of the relevant social science literature covering human-human collaborations, and on observations of person-person task collaboration in our experimental setting, we focus in this work on using speech. Speech works well across different embodiments and is effective in communicating intent. On the other hand, speech has obvious limitations in noisy environments, with users with hearing or linguistic limitations, and in certain scenarios, such as disambiguating many similar objects. Nonetheless, speech is a natural human communication modality that addresses a range of use cases in home and work environments.

Our approach involves the combination of three robot communication components: 1) robot’s self-narration of its activities, 2) role allocation suggestions for the user, and 3) empathetic displays when positive and negative events occur. The combination provides a balance of information aimed at improving the human collaborator's situational awareness. The second component, offering suggestions to the user, is particularly interesting because it informs the user that the robot is monitoring their progress and evaluating the world

This work was supported in part by National Science Foundation (NSF) grants CNS- 0709296 and IIS-1117279, and the ONR MURI program (N00014-09-1-1031).

from their perspective and allows the robot to potentially influence the joint decision making of the team.

A. Related Work Prior work has demonstrated that robots that account for the

actions of their collaborators when deciding what to do are preferred and perceived as more intelligent [6] and that anticipatory action can play an important role in increasing team fluency [3]. Other work has treated the collaborative process as a dialog, supporting verbal turn-taking and sub-task assignment [1], [8]. Speech has been demonstrated to be an effective input modality for commanding robots [6, 4]. Existing work in assistive robotics on robot speech production has been limited to specific conversational structures, such as turn-taking [1]. Our aim is to develop a methodology allowing the robot to issue task-relevant speech-based communication to a teammate during a dynamic collaboration.

III. APPROACH The task control and communication approach consists of

three constituent components: 1) the task control system, 2) the human activity model and recognizer, and 3) the communication planner and executive. Planning with a human partner in the environment is distinct from planning in multi-robot scenarios mainly in how communication takes place between teammates. Similarly, there are existing approaches to segmenting, recognizing, and modeling human activity in various contexts. We have developed a simplified methodology to serve these purposes in our experimental task, although this work could be integrated with and perhaps improved by state-of-the-art recognition and task planning systems. The main contribution of this paper is in the area of planning and execution of communication actions during collaborative task execution in HRI.

We formulate the coordination problem in two parts, as follows. First, the robot represents the task as an MDP to plan its actions in the presence of noise. We define a Markov decision process (MDP) for the task 𝑀 = {𝑆,𝐴, 𝑇,𝑅}, where 𝑆 is the finite state of the environment, 𝐴 is the set of task actions the robot can execute, 𝑇 is a function giving a probability distribution over states for executing a given action in a given state, and 𝑅 is a reward for each state. We assume that the robot can perform the task and has a policy 𝜋 𝑠 = 𝑎 that allows it to perform the task. In this case, the transition function captures the robot's uncertainty due to environmental sources, such as sensor or motor noise, and also due to changes the human collaborator might make in the environment. This formulation has been successfully used previously in human-robot collaboration [5] to coordinate the task-actions of the robot without any communicative feedback.

In addition to ensuring that the robot can jointly perform the task in the presence of a human partner, the robot needs to provide verbal feedback to its partner. We focus on communicating intended action via the mechanism of roles. People use explicit and implicit roles in team activity and other organized behavior. Roles have been used previously to inform human-computer interfaces for collaboration of multiple users [7] and extensively studied in multi-agent

systems [9]. Our pilot human-human experiments demonstrated that people tended to assign roles to others relative to common task-related objects and locations. Some roles consisted of multiple discrete activities, such as navigating to an object, picking it up, and moving it to a target destination. To capture users’ preferential action selection, we model roles using a set of assignable policies, 𝜋𝑟𝑜𝑙𝑒𝑠 ={𝜋!, 𝜋!,… 𝜋𝑛}. This set of policies is domain-dependent and not assumed to be optimal or otherwise sufficient for solving the task, when executed individually. Rather, the policies in the set are a means of quantifying patterns of user behavior over the course of the task, and of grouping similar actions according to the roles people typically assign.

To track role use over time, we assume the robot receives a stream of recognized actions and the agent (human or robot) responsible for performing them. To accomplish this in our test task, we developed a heuristic action recognition system that monitors state transitions and agent positions. The system maintains a multinomial distribution over the set of user roles based on the likelihood that the user is executing each policy. On each update, policies are reweighted based on their agreement or disagreement with the recognized action. Based on this recognized action and the robot’s own policy, three types of verbal feedback are generated: self-narrative, role-allocative, and empathetic. To generate narrative feedback, such as “I'll go take care of this,” the robot monitors its planned action and issues verbal feedback when the selected action changes. To generate role allocation suggestions for the user, such as “can you take care of the painting?”, the inferred policy of the user is queried to retrieve the user’s likely next action given the model. Since we are assuming the robot has a single-agent model of the task, it does not know the best pair-wise allocation of actions to each user. Instead, we provide the system with a list of action pairs that would conflict if performed by the person and robot at the same time. If the robot and user are expected to take conflicting actions, the robot suggests a different role for the user, one that would not result in a conflict. Finally, to generate empathetic feedback, the algorithm monitors for specific state transitions that are associated with especially good and bad outcomes. When these state transitions occur, the robot expresses empathy using a non-domain dependent positive or negative empathy (e.g., “Oh no” or “Great”), as appropriate.

Appropriate phrases for each type of verbal feedback were determined by having people perform the task in a small pilot experiment. For each action, we stored a set of phrases and randomly selected one such phrase when executing the communication. This avoided repetitive speech, which has been shown to be both annoying to users and to make the robot be perceived as less smart [10]. The phrases for role-allocative feedback can be readily adjusted to be more or less polite using methods from Politeness Theory [2]. We used baseline phrases collected from the pilot study, with no attempt to make them more polite. At this step there is also the possibility for the robot to proactively perform the better action instead of recommending that the user do it. For our

initial experiments, we assume that the robot's task control policy is static.

Figure 1. The experimental setup with person and a Pioneer 2-AT robot.

IV. EVALUATION AND RESULTS To validate the approach, we developed an augmented

reality environment that allows for testing a variety of tasks. The environment consists of a set of overhead projectors that project a merged display on the floor. Users' positions in the room are tracked with a pair of Microsoft Kinects. We tested our framework with a Pioneer 2AT mobile robot and a person. The virtual display data are provided by a task simulator that updates the positions of virtual objects in response to the actions of the physical agents overtime. This environment allows for simplification of agent-environment interaction dynamics and rapid prototyping of the task experience, as well as specifying the difficulty of the task by varying environment parameters. We implemented a pseudo-herding task in which many virtual sheep appear and roam around the environment. The collaborative team's goal is to herd all the sheep one-by-one, and bring them to a centralized collection area as quickly as possible. In addition, there are two timed objects, a lock and a light, that must be activated periodically to avoid penalties. These timing elements were incorporated to encourage teamwork and make the task and interaction more engaging. The set of policies representing user’s roles and the sets of specific phrases were obtained through an initial data collection with 6 person-person teams.

We are currently conducting a full experimental validation of the communication system with human-robot teams in a pseudo-herding task. The experiment is a within-subject design, with each participant seeing both the silent robot and communicating robot, with order counterbalanced. The robot’s task controller is the same in both conditions. Participants are first introduced to the task by the experimenter and then asked to do two trials with a robot teammate with the goal of finishing as quickly as possible. We collect audio, video, tracking data, the simulated state information, and administer a post-experiment survey asking about the robot’s value as a teammate and other demographic information. Preliminary results from 8 (2 female, 6 male) of n=20 participants demonstrate that the mean total duration to complete the task is lower (i.e., faster) when users collaborate with the talking robot, with a mean time to completion of 151 seconds (SD=88) compared to the silent robot with a mean of 169 seconds (SD=100), although this difference is not significant, likely due

to an entrainment effect in which most users’ times improve markedly during their second performance of the task. The survey responses are promising, with all but one user specifying strong agreement that the talking robot is a better teammate than the silent robot, among other positive attributes (see Table 1).

In addition to completing the full evaluation of the task communication system, we plan to apply and evaluate the approach on a real-world assistive task involving physical object manipulation with older adults. We also plan to address methods for learning the set of role policies and verbal references from a guided interaction and develop methods to adapt the robot's communication policy based on factors such as the user’s compliance and preference for different amounts of communication.

TABLE I. MEAN SURVEY RESULTS

Question 0 (disagree) - 6 (agree) The things the robot said made sense. 6.0 The talking robot was a better teammate than the silent robot. 5.88

The robot’s talking helped me understand what it was going to do next. 5.43

I tried to do what the robot told me to do. 5.25 The talking robot was more fun. 5.0 The things the robot said helped me decide what to do. 4.75

REFERENCES [1] Cynthia Breazeal, Cory D Kidd, Andrea Lockerd Thomaz, Guy

Hoffman, and Matt Berlin. Effects of nonverbal com- munication on efficiency and robustness in human-robot teamwork. In Intelligent Robots and Systems, 2005.(IROS 2005). 2005 IEEE/RSJ International Conference on, pages 708–713. IEEE, 2005.

[2] Penelope Brown. Politeness: Some universals in language usage, volume 4. Cambridge University Press, 1987.

[3] Guy Hoffman and Cynthia Breazeal. Cost-based anticipatory action selection for human–robot fluency. Robotics, IEEE Transactions on, 23(5):952–961, 2007.

[4] Thomas Kollar, Stefanie Tellex, Deb Roy, and Nicholas Roy. Toward understanding natural language directions. In Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on, pages 259–266. IEEE, 2010.

[5] Stefanos Nikolaidis and Julie Shah. Human-robot cross- training: computational formulation, modeling and evaluation of a human team training strategy. In Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction, pages 33–40. IEEE Press, 2013.

[6] Julie Shah, James Wiken, Brian Williams, and Cynthia Breazeal. Improved human-robot team performance using chaski, a human-inspired plan execution system. In Proceedings of the 6th international conference on Human-robot interaction, pages 29–36. ACM, 2011.

[7] Randall B Smith, Ronald Hixon, and Bernard Horan.. In Collaborative Virtual Environments, pages 160–176. Springer, 2001.

[8] J Gregory Trafton, Nicholas L Cassimatis, Magdalena D Bugajska, Derek P Brock, Farilee E Mintz, and Alan C Schultz. Enabling effective human-robot interaction using perspective-taking in robots. Systems, Man and Cybernetics, Part A: Systems and Humans, IEEE Transactions on, 35(4):460–470, 2005.

[9] P. J. Gmytrasiewicz and P. Doshi, “A framework for sequential planning in multi-agent settings.,” In Journal of Artificial Intelligence Research (JAIR), vol. 24, pp. 49–79, 2005

[10] C. Torrey, S. R. Fussell, and S. Kiesler, “How a robot should give advice,” in Human-Robot Interaction (HRI), 2013 8th ACM/IEEE International Conference on, pp. 275–282, IEEE, 2013.