CognitiveScience

1

Kartik TalamadupulaSubbarao

KambhampatiJ. Benton

Dept. of Computer ScienceArizona State University

Paul SchermerhornMatthias Scheutz

Cognitive Science Program

Indiana University

Planning for Human-Robot

Teaming

CognitiveScienceMotivationMotivation

2

• Early motivation of AI– Autonomous control for

robotic agents

• Plenty of applications– Household Assistance– Search and Rescue– Military Drones and Mules

• All scenarios involve humans giving orders

• Planning must co-opt this area

CognitiveScience

Human-Robot Teaming

• Teaming– Share the same

goal(s)– Autonomous behavior– Communication

• Role of Planning– Plan generation– Feedback

acceptance– Model resolution

HUMAN

ROBOT PLANNERPlanning and Execution

Monitoring

Human Robot Interaction (HRI)

Mixed Initiative Planning (MIP)

What are the factors that planners must take into account?

CognitiveScience

Dimensions

Scenario / Environment

• Inspired by the real world• Large amounts of domain knowledge from

– Humans with experience– Technical documents and manuals

• New knowledge may arrive during execution– Planner must handle such contingencies

• Planner and Robot Features– Determined by the needs of the scenario– E.g.: NASA needs temporal planning

CognitiveScience

Dimensions

Robotic Agent• Central Actor

– Execute actions– Gather sensory feedback

• Different types of robots– Various capabilities

Gripper

Humanoid Mobile Combined

CognitiveScience

Dimensions

Human User

• Specifies and updates:– Scenario goals– Model (in some cases)

• Must be in communication with robot/system

NoviceUses the robot merely as an

assistant

Domain ExpertAuthority on the execution

environment

System ExpertAuthority on the

integrated AI system

CognitiveScience

Planning

Goal Management

• Human-Robot Teaming– Utility stems from delegation of goals

• Support different types of goals– Temporal Goals: Deadlines– Priorities: Rewards and Penalties

• Bonus Goals: Partial Satisfaction

– Trajectory Goals– Conditional Goals

• Changes to goals on the fly– Open World Quantified Goals

[Talamadupula et al., AAAI 2010]

CognitiveScience

• One true model of the world– Robot

• High + Low Level models

– Human User• Symbolic model + Add’l knowledge

– Planner must take this gap into account

• Model Maintenance v. Model Revision– Usability v. Consistency issues– Use the human user’s deep knowledge

• Distinct Models– Using two (or more) models

• Higher level: Task-oriented model• Lower level: Robot’s capabilities

Planning

Model Management

MODEL

Robot Human

CognitiveScience

HRT Tasks: Examples

SEARCH AND REPORT

RECONNAISSANCE

KITCHEN ROBOT

ROBOT Mobile MobileMobile and Manipulator

HUMAN (USER) Domain ExpertSystem Expert

Novice

MODEL Less Dynamic DynamicHighly

Dynamic

GOALS Evolving Static Evolving

COMMUNICATIONNatural

LanguageAPIs

Natural Language

Feature

Task

CognitiveScience

Case Study

Urban Search and Rescue

• Human-Robot Team in Urban Setting– Find and report location of critical assets– Human: Domain expert; removed from the

sceneSEARCH AND REPORT

• Deliver medical supplies

• Bonus Goal: Find and report injured humans

• Requirements– Updates to knowledge

base– Goal changes

[Talamadupula et. al., AAAI 2010]

RECONNAISSANCE

• Gather information• High risk to humans

– E.g. Bomb defusal

• Requirements– Support model changes– New capabilities

• E.g.: Zoom camera

CognitiveScience

Goal Manager

Goal Manager MonitorMonitor PlannerPlanner

Plan

Plan

Problem Updates

Updated State

Information InitialModel

Information

Sensory Informati

on

Actions

System Integration

Additional Capabilities

Model Update

CognitiveScience

Model Update: Demo Model Update: Demo RunRun

12

• Initial GoalEnd of hallway

• During Execution Injured humans

(boxes) in rooms behind doors

• New action / effect during execution Push doors to get inside rooms

CognitiveScience

Conclusions

• Human-Robot Teaming from a planning perspective

• Planning Challenges– Framework for Human-Robot Teaming Problems– Model and Goal Management

• Need to define the scope of planning for these tasks

– What are the main technical problems

• Huge potential for novel P&S applications• Companion Robots• Military and Service Drones• Household Assistants

CognitiveScience

Future Work

• Multiple Models– Use two (or more) models to direct the planning

• Task v. Motion Level (BTAMP Workshops)• Classical v. More Expressive

• Robotic Proactiveness– “Ask” for help– Many sources of knowledge in the real world– Putting the “teaming” in HRT

• More Application Scenarios– Design planners sensitive to HRT issues

System DemoTuesday 5:30pm

Main Conference