Architecture for Autonomous Assembly1

Architecture for Autonomous AssemblyArchitecture for Autonomous Assembly

Reid SimmonsRobotics Institute

Carnegie Mellon University

Architecture for Autonomous Assembly2

Prior Work: Multi-Robot AssemblyPrior Work: Multi-Robot Assembly

• Heterogeneous

• Coordinated

• Sliding Autonomy

Architecture for Autonomous Assembly3

Overall ApproachOverall Approach

• Utilize our previous work in robot autonomy– Multi-layered software architecture

– Hierarchical, task-level description of assembly

– Robust, low-level behaviors

– Distributed visual servoing

– Force sensing

– Exception detection and recovery

Architecture for Autonomous Assembly4

Architectural FrameworkArchitectural Framework

Deals with goals and resource interactions

Task decomposition; Task synchronization; Monitoring; Exception handling

Deals with sensors and actuators

• Three-Tiered Architecture

• Reactive & Deliberative• Modular• Control loops at multiplelevels of abstraction

Executive

BehavioralControl

Planning

Architecture for Autonomous Assembly5

Syndicate: Multi-Robot ArchitectureSyndicate: Multi-Robot ArchitectureSynchronization / Coordination

Gra

nula

tity

Executive

BehavioralControl

Planning

Executive

BehavioralControl

Executive

BehavioralControl

Planning Planning

Architecture for Autonomous Assembly6

Advantages of Tiered ApproachAdvantages of Tiered Approach

• Control loops at multiple levels of abstraction

• Integrates reactive and deliberative mechanisms

• Managed complexity

• Representations tuned to specific objectives

• “Plans as Advice”

Architecture for Autonomous Assembly7

Why Multi-Robot Architecture?Why Multi-Robot Architecture?

• Modularity– Mobile manipulator agent

– Perception agent

– Coordinator agent

• Efficiency– Promotes concurrency

– Can spread over multiple computers

• Reuse Existing Algorithms and Software

Architecture for Autonomous Assembly8

Behavioral LayerBehavioral Layer• “Skill Manager”

– Skills are concurrent behaviors, including perceptual behaviors– Dynamic creation of real-time feedback loops

• Higher tier (executive) connects sensing and action modules and enables subsets of skills

• Skills indicate status by passing signals back to executive– Enables flexible (and distributed) feedback control loops

(e.g., visual servoing)

Virtual Sensors Action Behaviors

SignalsConfigure

Action

s

Arb

iter

Architecture for Autonomous Assembly9

Distributed Visual ServoingDistributed Visual Servoing• Mast Eye tracks fiducials

– Uses ARTag software package to detect fiducials– Provides 6-DOF transform between fiducials

• Mobile Manipulator uses information to plan how to achieve goal– Use data base describing positions of fiducials on objects

• Behavioral layer enables dynamic, transparent inter-agent connections

Mobile Manipulator

VisualServoRelativepositions

ArmControl

End effector delta

Mast Eye

Tracking

Images (via cameras)

The World

Manipulate environment (via arm)

Architecture for Autonomous Assembly10

Distributed Visual ServoingDistributed Visual Servoing

• Fairly precise– millimeter resolution at one meter

• Relatively fast– 3-4 Hz

• Basically unchanged from Trestle code

• Operates in relative frame– Poses of one object relative to another

– Controller continually tries to reduce pose difference

– Cameras do not need to be precisely calibrated with respect to base or arm

Architecture for Autonomous Assembly11

Executive LayerExecutive Layer• Forms a Bridge Between Planning and Behaviors

– Discrete vs. continuous control

– Symbolic vs. numeric representations

– Real-time considerations

• Basic Roles– Decompose task into subtasks and dispatch tasks

– Monitor execution for contingencies and opportunities

– Reschedule tasks (or schedule new tasks) upon failure

Architecture for Autonomous Assembly12

TDL (Task Description Language)TDL (Task Description Language)• High-Level Language Tailored to Task-Level Control

– Extension of C++ with explicit syntax for task-level control constructs

– Syntax for task decomposition, sequencing, monitoring, error handling

– Compiles into pure C++ with calls to task management library

• Requirements– Simple concepts should be expressible in simple terms

– Do not preclude expression of complex control constructs

– Natural integration with existing code

Architecture for Autonomous Assembly13

Task Trees – DeliberativeTask Trees – Deliberative• TDL Maintains and Coordinates Task Trees

– Execution trace of hierarchical plans• Created dynamically at run time• Can be conditional and recursive

– Temporal constraints (partially) order task execution

– Planning and sensing treated as schedulable activities;Concurrent planning, sensing, and execution

Architecture for Autonomous Assembly14

Task Trees – ReactiveTask Trees – Reactive• Task Trees Augmented with Reactive Elements

– Task-specific execution monitors– Context-dependent, hierarchical exception handlers

• Replan by Analyzing and Manipulating Task Trees– Terminate subtrees– Add new nodes and/or temporal constraints

Architecture for Autonomous Assembly15

Ace Task DecompositionAce Task Decomposition

Child link

Serial Constraint

Architecture for Autonomous Assembly16

Example TDL Code (somewhat simplified)Example TDL Code (somewhat simplified)

Goal ClipInsertion ( ) {

loadPlugArmPose : spawn ArmMove (loadPose);

stowArmPose: spawn ArmMove (stowPose) WITHSERIAL loadPlugArmPose;

roughBaseMove: spawn RoughBaseMove (roughBaseWaypoint)

WITH SERIAL loadPlugArmPose;

spawn RoughArmMove (roughArmWaypoint)WITH SERIAL roughBaseMove, SERIAL stowArmPose;

…}

A keyword that says this is not supposed to be a “leaf” in the task tree

Reusing task with different parameters

In the tree, this is the task name, but this is the actual function being executed

Tells the system to execute this task

after LoadPlugArmPose completes

Wait until both tasks have completed before starting RoughArmMove

Architecture for Autonomous Assembly17

Monitoring & Error RecoveryMonitoring & Error Recovery

• Monitoring– Detect exceptions: significant differences between

expectations and the state of the world

• External Events (contingencies, opportunities)

• Internal Events (sensors, actuators, computers)

• Task Status (preconditions, progress, postconditions)

• Diagnosis– Determine cause of exception

• Error Recovery– Reconcile expectations and world state

• Change world state (repair)

• Alter expectations (replan)

Architecture for Autonomous Assembly18

MonitoringMonitoring

• Detect Completion of Tasks– Success conditions

• Detect Failure Conditions and Lack of Progress

• Need to Consider All Possible Situations– Important to try to cover all failure scenarios

– Can handle hierarchically

• Monitor for very general conditions as well as specific conditions

Architecture for Autonomous Assembly19

Failure RecoveryFailure RecoveryExecutive

ArmPlaceBeam

GraspBeam DockBeam

Spawn

Serialize

Behavioral

VisualServo

Activate

LostBeam

Activate Failure

Failure

Architecture for Autonomous Assembly20

What About the Planning Layer?What About the Planning Layer?

• Planning layer creates task trees online– Based on knowledge of the system

– Example: New task to be done. Planner knows which robots are available/capable to achieve it.Spawn the task on the desired robot.

• Our tasks are straightforward– Number and type of robots is known

• We hard-code our own task tree– One super-task spawns the main tree