Mobile Manufacturing of Large Structures

David Bourne, Howie Choset, Humphrey Hu, George Kantor,Chris Niessl, Zachary Rubinstein, Reid Simmons, Stephen Smith

Robotics Institute, Carnegie Mellon University

Abstract— Assembly of large structures requires large fix-tures, often referred to as monuments. Their cost and massivesize limit flexibility and scalability of the manufacturing process.Numerous small mobile robots can replace these large struc-tures and, therefore, replicate the efficiency of the assemblyline with far more flexibility. An assembly line made up ofmobile manipulators can easily and rapidly be reconfigured tosupport scalability and a varied product mix, while allowing fornear optimal resource assignment. The challenge to using smallrobots in place of monuments is making their joint behaviorprecise enough to accomplish the task and efficient enough toexecute subtasks in a reasonable period of time. In this paper,we describe a set of techniques that we combine to achievethe necessary precision and overall efficiency to build a largestructure. We describe and demonstrate these techniques inthe context of a testbed we implemented for assembling a wingladder.

I. INTRODUCTION

A popular approach in industry for assembling large-scale structures is to establish assembly lines that use large,mounted fixtures to provide precise positioning of parts,relative both to other parts and to the tools used in theassembly. While these lines are efficient, the permanenceof these fixtures requires costly infrastructure and resultsin an assembly space that is not adaptable to changes indemand for different product lines. In addition, the lines arevulnerable to issues that arise during assembly and frequentlyresult in the entire line being delayed until they can beresolved. To address these drawbacks, we are investigating anew approach to assembly where the fixtures and the toolingare all onboard coordinated mobile robots. The mobility ofthe robots allows for rapid reconfiguration of the assemblyspace, which means better adaptability to meet changingproduct demand and the ability to work around line issues.

The primary challenge to replacing mounted fixtures withtooling on small mobile robots is achieving the precisionrequired to reliably assemble the products. Positioning themobile robots and the parts to millimeter accuracy is difficultdue to the uncertainty in both the environment and thecontrollers of the robots. In addition, complex coordinationis needed to get the robots, the parts, and the tools at theright place at the right time for each step in the assembly.

We have built a demonstration facility to explore theefficacy of assembling large structures with a number ofheterogeneous mobile robots. We have developed a sequenceof coordinated behaviors that accomplishes an exampleassembly task by combining four component techniques:i) mobile manipulator hardware design, ii) fixture-free po-sitioning, iii) multi-robot coordination, and iv) real-time

dynamic scheduling. In this paper, we present each of thesetechniques, describe how they contribute to the precision andefficiency of assembly, and discuss how the techniques canbe sequenced to achieve precise, efficient assembly of large-scale structures.

The long-term goal of this work is to create a rigorousframework for rationally sequencing these techniques toachieve the necessary precision for assembly in the mostefficient sequence (see Figure ??) . After working throughthe details of our current multi-robot assembly example, wepropose a generalized framework to accomplish similar tasksthat employs a “funnel” [12] or sequential composition [1]concept in which multiple controllers are sequenced togetherto mitigate uncertainty.

II. RELATED WORK

Assembly planners create a graph [26], [27] that encodesthe order of motion operations and the geometric constraintsensuring that each motion can be executed. Typically, the partmust move in contact with other parts in the subassembly,so an assembly can be viewed as a sequence of compliantmotions [13] where a transition from one compliant motionto another occurs at a change in contact state. Often, aplanner uses a graph, called a contact state graph, as wellas knowledge of the parts’ geometries to determine thesequence. [11], [14], [6]. Much effort has been dedicatedto automatically generating the contact state graph [7], [15],[28].

Regardless of the choice of sequence planner, the overallsystem must be able to accurately align the part for finalassembly, which is often quite difficult when the part’s posehas uncertainty. Here, compliant motions can be used tominimize this uncertainty through a sequence of compliantmotions [12], [17]. In our work, we use compliance (i.e.,springs) to achieve compliant and force-controlled executionand assist in part localization. Previous work has used passivecompliance to account for misalignment between a robot andtask constraints [24]. Series elastic actuators [18] introducedcompliance to reduce shock loads and enhance force controlaccuracy and stability. We have found from our previousresearch that passively compliant mechanisms can improveprecision without losing performance. If a part contacts astructure prematurely, it will comply to the shape of thecontact area without causing damage. By taking advantageof known features in a given structure, passive complianceis both a precise and fast operation to position a part.

2015 IEEE International Conference on Robotics and Automation (ICRA)Washington State Convention CenterSeattle, Washington, May 26-30, 2015

978-1-4799-6922-7/15/$31.00 ©2015 IEEE 1565

Our work to date has assumed the existence of a hierar-chical assembly planning graph, and this process description,which dictates both the order in which parts should arrivefor assembly and the coordinated sequence of actions thatmust be carried out to successfully join specific parts, hasdriven the design characteristics of the robots required toexecute assembly subsequences. The robots that we createdare sometimes referred to as mobile manipulators becausethey combine aspects of both mobile robots and manipulatormechanisms [22]. While some systems [9] use multiplemobile manipulators to manipulate a single object, we arenot aware of a multi-mobile manipulator system that wasdesigned for assembly. As such, prior multi-robot mobilemanipulators typically assume that all poses of the robot isknown at all times, at least with respect the mobile bases.

We believe new mobile manipulators must be createdbecause existing systems either contain more degrees offreedom than necessary or lack the payload to carry out theassembly task. This is because mobile manipulators werenot originally conceived to carry out coordinated assembly.To design these new systems, our work is inspired by i)the tool-design for assembly work [25], which developeda tool representation that includes the tool’s use-volume,i.e., the minimum space that must be free in an assemblyto apply the tool, and ii) [5], which presented a detailedapproach for modeling tools, selecting tools, and generatingtool movements. Their work allowed tools to be modeled asarticulated devices.

III. MODULAR MOBILE MANIPULATOR ROBOT SYSTEM

The work described in this paper uses assembly of airplanewings as a motivating application, but we believe the ideasare general to assembly of many large structures such asships, large construction vehicles, and trains. To date, wehave focused on two tasks: coordination of two robots toinsert ribs into a partially-assembled wing ladder and sensor-based coordinated motion of two robots to carry a flexibleskin panel. The rib-insertion task and larger ladder assemblyprocess (the focus of this paper) involve moving ribs (thecross-wise pieces) into place between two spars (the longlength-wise pieces). In our testbed, the spars are 10 feet longand accommodate 5 ribs; the ribs vary in length (the spars arenot parallel) but average about 3.5 feet and weigh about 20pounds. The spars have L-brackets spaced every two feet, towhich the ribs need to be attached. There are three 3/8 inchdiameter holes on either end of the rib that must be alignedwith corresponding holes on the L-bracket. The alignmenttolerance is about a millimeter.

The robots are 30-inch diameter, 3-wheeled omnidirec-tional bases, designed at CMU. Each robot carries a manip-ulator module specialized for the role it plays in the assemblytask (examples of these mobile manipulators can be seen inFigures 1, 2, 3, and 4). The bases, which have payloadsof several hundred pounds, all have on-board computation,communications, sensors, and power to support themselvesand the manipulator modules they carry. Sensors used in

Fig. 1. The yellow spars support the black ribs forming the ladder. Here,a rib-carrying robot is bringing a rib into its final position.

Fig. 2. Omnibases

the current system include accelerometers, cameras, lasers,potentiometers and limit switches.

The behaviors, algorithms, and framework developed tosupport the rib-insertion task is the focus of the rest ofthis paper. The task begins by having the rib-carrier robotapproach the end of the wing ladder assembly, carrying a riblength-wise (Figure 1). The robot rotates to enable it to movebetween the spars with sufficient clearance. Concurrently,the spar-catcher robot aligns a temporary fixture (Figure 3)with the spar cap and then slides along the spar cap untilit detects the L-bracket (via a limit switch), at which pointit engages a brake to provide a rigid target for the rib. Therib carrier then approaches the target using a guarded move,i.e., a move that stops when the “guard” is triggered. The“guard” sensor is an accelerometer attached to the target.The accelerometer signal is processed by the spar-catcherrobot, which communicates with the rib carrier to triggerthe “guard.” Passive compliance is used to handle the smallamount of movement after contact, due to latency caused bythe signal processing and communications.

Having hit the target, the system uses a camera mounted onthe rib carrier to determine where the rib is along the target(by calculating the percentage of the target not obscured bythe rib). The rib carrier backs up a few centimeters, rotates topoint at the corner where the L-bracket meets the spar, andthen drives towards that point until, once again, the target-mounted accelerometer indicates that contact has been made.

1566

Fig. 3. Spar catcher, on the other occluded side of the yellow spar, isabout to engage with the sparcap in order to register the black and greenfeature against the L-bracket in the spar.

The rib carrier then moves in an arc to “close the door,”stopping when a limit switch on the target is contacted, whichindicates that both ends of the rib are now flush with the L-brackets. Again, the passive compliance in the rib carriermodule (aluminum disks below the rib carrying mechanismin Figure 1) enables the system to compensate for smallamounts of overshoot.

Finally, the system aligns the holes in the rib and L-bracket. Two cameras mounted on the spar catcher headprovide high-resolution images of two of the holes in theL-bracket. The system collects background images of theholes prior to the arrival of the rib carrier and compares themagainst images taken with the rib in place to determine thedegree and direction of overlap between the bracket holesand the rib holes. The data from both images are usedto determine how to move a 4-bar linkage that supportsthe rib (Figure 4) in order to reduce the overlap. Thisservoing continues until the holes are adequately aligned,at which point a person can insert cleco fasteners withoutany resistance. The robot then lowers the tooling that holdsthe rib (which mechanically unlocks the fingers that hold therib in place), and it drives out of the wing ladder to receiveanother rib and begin the task anew.

The broader problem of moving parts (ribs and spars) byrobots from storage areas to assembly stations to achievean efficient overall ladder assembly process is consideredin simulation (Figure 5). Movement tasks are dynamicallyallocated to robots as assembly of wing ladders proceeds atmultiple assembly stations. Robot movements are scheduledso that parts arrive just in time to support their insertion, thusminimizing overall assembly time while also minimizingtraffic congestion.

The rib insertion assembly sequence and broader ladderassembly process combines the four component techniquespresented in the introduction. Global and relative positioningis used to move the robots into positions where they are readyto begin the assembly task. The temporary fixture and therobot compliance that are used in the guarded move step em-ploy both passively compliant hardware and actively compli-ant sensing and control. The execution of the individual stepsand the logic to switch between them represent intelligentbehavior assembly strategies. Component behavior assembly

Fig. 4. Final visual servoing step to align holes.

Fig. 5. Simulation of the broader ladder assembly process.

strategies are assigned to robot teams and coordinated inspace and time using real-time, dynamic scheduling. Startingwith the mechanical components, each of these techniquesis described in more detail in the following sections.

A. Mechanism Design: Mechanical Features and PassiveCompliant Devices

1) Omni Robot Bases (PF3-30): Each mobile manipulatoris built upon the same mobile base. The bases are 30 inchesin diameter and are driven by three omni-directional wheelsthat can currently carry up to 300 pounds each. Omni-directional robot are needed because the mobile manipulatorsrequire enhanced maneuverability in tight spaces as theassembly is being formed. Each robot base is divided intofour distinct layers: (1) the motor controllers and powermodules including lithium batteries, (2) the computationallayer, which is based on high speed Intel quad-i7 CPUs, (3)network connections and sensor preprocessing modules, and(4) task-specific tooling. The first two layers are commonamong all of the robots, while the third and fourth layersare customized according to the assigned role of a particularrobot.

In addition to this layered robot architecture, there are sev-eral hardware extensions that are used for global localization.This localization hardware consists of a sensor package anda fiducial harness designed to be easily sensed by a matching

1567

Fig. 6. Spar catcher.

sensor package on the other robots. The current localizationhardware is identical between robots, allowing us to use asingle harness design, but the system can be extended tosupport any number of different harnesses. The details ofthe harness and sensor package are the following:

a) Fiducial Harness: The fiducial harness is a set of panelsmounted to the robot to form a hexagonal profile. Ahexagonal profile was chosen as a compromise betweenmaximizing the number of distinct corners and flat edgesviewable at a time, while minimizing wasted space on theround PF3-30. A unique AprilTag [16] fiducial is mountedon each flat segment of the harness for each robot.

b) Sensor Package: The sensor package comprises planarlaser rangefinders and cameras. We use one HokuyoUTM-30LX (270◦ FOV) planar laser rangefinders andone Microsoft LifeCam USB webcam (70◦ FOV), thoughthis number could be increased for greater sensing cov-erage. The sensors are mounted at the corners of thefiducial harness hexagon, with unfilled sensor mountpoints occupied by sensor dummies to avoid gaps in theprofile. Assuming that the environment has relatively flatfloors, the sensor package senses the horizontal profileof each robot at the elevation of the fiducial harness.Feedback from the PF3-30’s wheel encoders are also usedin the localization system.

2) Spar Catcher: The spar catcher is a PF3-30 base onwhich is mounted a long compliant arm and a compliantgripper coupled with a complex fixture and sensor package(Figure 3). In essence, this robot is an intelligent hook thatcan blindly attach itself to a spar and then align itself byseveral simple discrete motions: (1) pull back to square upgripper under compliance, (2) move left to contact assemblyfeature, and (3) lock position with a braking mechanism.

The main role of the spar catcher is to emplace a tempo-rary fixture (called a “rib target”) that is used to help positionthe rib. The spar catcher also acts as a sensor platform todetect ribs when they approach and assist in the final visual-servoing alignment step.

3) Rib Carrier: The rib carrier is a PF3-30 base on whichis mounted a compliant rotary table that provides softness inthe yaw dimension of a gripped rib. In turn, the table isconnected by four compliant springs to provide softness inthe two remaining degrees of orientation (pitch and roll).

With this arrangement of compliant members, the ribcarrier can both manage and account for uncertainties in theflatness of the manufacturing floor and the relative flatness ofthe spars and the to-be assembled rib. However, the greatestrange of compliant motion is in the rotary table, whichmanages and accounts for potentially gross uncertainty inthe robot’s orientation relative to the targeted assembly. Ifthe robot is misoriented, the rotary table simply winds upunder contact and measures the mis-orientation for correctivemotions.

The rib carrier also includes an active four-bar linkage,driven by three motors (Figure 4) to do very fine positioningof the ribs. Two of the motors lift and orient the rib (in theplane of the rib), while the third motor is used to move therib side-to-side.

B. Fixture-free Positioning: Relative and Global Positioning

Robots cooperatively carrying large components cannot belocalized with overhead vision systems due to occlusion, andthe density of robots near the workspace may make it difficultfor robots to localize using environmental landmarks. In-stead, we opt for an “environment-agnostic” approach whererobots sense only other robots. In our approach, robots collectsensor data and process it onboard to identify and trackneighboring robots. The resulting relative pose estimates aresent to a central server, where they are assembled into a posegraph that is optimized to generate a consistent estimate ofall robot poses in a common global coordinate frame.

Data from the laser scanner and camera pass throughvarious stages and produce relative position measurements todetected fiducial harnesses, which are fused in a local EKFto produce a relative pose measurement for each observedrobot. AprilTag software1 is used to identify and measurerelative poses to AprilTag fiducials fixed to the fiducialharnesses. Laser scanner point clouds are segmented, anditerative closest point (ICP) scan matching is used to attemptto fit segments to the known horizontal geometry of thefiducial harness. When sufficient matches are found, ICPalso provides a relative pose measurement of the observedrobot. Laser and camera pose measurements are fused withrobot odometry in a local EKF, providing single relative poseestimate for each observed robot. The estimates are sent to acentral server along with odometry information where theyare combined with relative pose estimates and odometry fromother robots to form a multi-robot pose graph as shown inFigure 8. Globally consistent pose estimates for all robots aregenerated using the g2o graph optimization package [10].

1http://people.csail.mit.edu/kaess/apriltags/

1568

Fig. 7. Block diagram illustrating pipeline for localization system. Cameraand laser data are locally fused into relative pose estimates based onobservations between robots, and the results are relayed to a central serverwhere they are assembled into a pose graph, which is optimized to produceglobally-consistent pose estimates of all robots in the system.

Fig. 8. Illustration of how relative pose estimates and odometry mea-surements are fused into a multi-robot pose graph. Red arrows representodometry measurements for individual robots. Blue arrows represent relativepose measurements generated by robots observing one another. These poseestimates and measurements become edges in the pose graph, while theposes of the robots at times when observations were made are the posegraph nodes.

C. Multi-robot Coordination: Coordination at the Micro-Level

Assembly planning is done using a geometric modelof the wing parts and how they are supposed to connecttogether, plus a rough model of the uncertainty in motionand positioning of the robots. For instance, the amount therib carrier initially turns before moving between the sparsis calculated from the length of the rib being carried, theminimum clearance between the spars up until the pointwhere the rib is to be inserted, and the uncertainty in theglobal positioning system (plus some padding). Similarly,the arc that the robot should take to ”close the door” iscalculated based on the robot’s current pose vis-a-vis theright-side L-bracket, the length of the rib and the distancebetween the two spars. This enables the system to emplacespars anywhere along the ladder using the same algorithmand, in fact, enables the system to be adaptable to a rangeof other wing designs.

Successful execution of such behaviors requires the robotsto coordinate in passing information and being at the rightplaces at the right times. For efficiency, the robots moveconcurrently, when feasible, pausing until each has achievedits goal. This coordination is facilitated by a layered ar-

chitecture, named Syndicate [19], that we developed for aprevious project. Syndicate has capabilities to define data-driven behaviors and hierarchically defined tasks that canbe distributed amongst multiple robots. The tasks decom-pose into subtasks, bottoming out in tasks that activate anddeactivate goal-achieving behaviors (such as moving androtating the spar catcher until its limit switches indicate thatthe end effector is aligned with the spar cap). Temporalconstraints between subtasks are represented using the TaskDescription Language [20], an extension of C++ that in-cludes syntax for task decomposition, temporal coordination,execution monitoring, and error recovery. Exchange of datais performed using IPC [21], an efficient publish-subscribemessaging system that is cross-platform and cross-language(while most of the system is written in C++, the visionprocessing is in Python). Together, these tools enable usto specify complex coordination strategies succinctly, andenable them to be carried out efficiently, with only a smalllatency in communications.

In addition, Syndicate enables us to flexibly define be-haviors that can take advantage of robot resources andavailability. In the current system, for instance, the guardedmove behavior consists of one robot that moves and anotherrobot that senses some condition that acts as the guard.The guards that we use are either accelerometers or limitswitches. The type of sensor being used and the robot onwhich that sensor is mounted can be indicated at run time,which provides for a large degree of flexibility. While wecurrently do not change the mappings at run time, we douse the same guarded move behavior in several differentways, mixing and matching robots and sensors to get themost effective behavior for achieving a particular subtask.

D. Dynamic Scheduling: Coordination at the Macro-Level

At the ladder assembly level, we take a dynamic schedul-ing approach to allocating transport, assembly, fixture andassist tasks to robots, and coordinating their execution inspace and time. Our approach combines several basic con-cepts to reduce the overall uncertainty introduced by ourmobile assembly approach and retain manufacturing effi-ciency. First, we assume a structured factory floor, whereparts are transported from storage areas to assembly sitesalong well-defined corridors. This assumption allows us tomanage interactions between robots exclusively at intersec-tions and assembly sites, and promotes efficient overall trafficflow. The corridor network also provides a structure fordecomposing part delivery and insertion tasks into sequencesof localized component behaviors (e.g., corridor segmenttravel with following and queuing behaviors). To retainflexibility in exceptional situations (e.g., a failed robot),we advocate integration with contemporary path planningtechniques [23], [2] to temporarily reconfigure the corridornetwork.

Second, we assume that basic traffic control protocolsare used to govern robot movements through intersections.Using knowledge of this protocol together with a planningmodel that accounts for both temporal and spatial constraints

1569

on part movement, it becomes possible to reliably predictpart delivery durations. This capability in turn, enables ahierarchical planning approach, similar to that proposedin [8], where robot plans are generated according to anabstract model, and expanded closer to execution time whenuncertainty is reduced. In contrast, prior work in task alloca-tion for mobile manufacturing has either assumed simpler,homogenous (e.g., tress) assembly structures [30], [29],where assembly mass computations can effectively driveallocation decisions and prediction of part delivery durationsis not important, or mobile robot manufacturing applications(e.g., cooperative painting of large structures) where robotmovement constraints can be reasonably ignored [4].

Third, part delivery and insertion plans are generateddynamically as a function of the current execution state.Exploiting an abstract assembly planning graph, a forwardlook-ahead projection is repeatedly used to determine 1) theearliest time at which a given assembly station will go idle(i.e., waiting for the next part to arrive) and 2) the lead timerequired to deliver the next part, and this look-ahead infor-mation provides a basis for scheduling “just-in-time” partdeliveries. The forward projection utilizes a state-dependentshortest-path calculation to determine the most efficient pathfor a given part to take at a given time, and a hedgingparameter is utilized to account for remaining durationaluncertainty. Timing the start of part delivery tasks for arrivaljust when they are needed is advantageous in two respects:planning decisions are made with the most current (and hencemost certain) information possible about the global executionstate, and traffic congestion on the floor is minimized asa side-effect, which minimizes the uncertainty associatedwith look-ahead prediction. As new tasks are allocated torobot teams and scheduled, constituent subtask sequences arecommunicated to Syndicate, and mapped to correspondingbehaviors for execution. As execution proceeds, results arecommunicated back to the dynamic scheduler to update itsmodel of the current execution state.

IV. EXPERIMENTS

We ran a series of trials to help quantify the performanceof our rib-insertion system (minus the visual-servoing align-ment behavior, which had not been completely integratedat the time of the experiment). Each trial started about twometers from the entrance to the spars at different left-to-rightdistances from the center. The result of each trial was classi-fied qualitatively, in terms of overall success/failure (i.e., didthe robots have to be stopped manually before completing thefinal ”close the door” behavior) and quantitatively, in termsof the final position of the rib with respect to the L-bracket.

25 trials were performed. A distribution of starting posi-tions was chosen for the trials, ranging plus or minus 60 cmfrom the center of the two spars and plus or minus 70 degreesfrom facing towards the spars. 22 trials (88%) percent of thetrials were successful, as defined above. Two of the failureswere caused by the brake on the spar carrier not holdingsufficiently, and the rib pushed the target plate back and was

emplaced on the far side of the L-bracket. In the other failure,the robot stalled during the ”close the door” behavior.

Each successful insertion took on average 88 seconds(maximum of 108, standard deviation of 9.4). For the suc-cessful trials, the average distance from the desired positionof the rib was 0.68cm in the vertical direction (maximumof 1.0cm, standard deviation of 0.32cm) and 0.77cm in thehorizontal direction (maximum of 1.0cm, standard deviationof 0.30cm). The average gap between the top hole of therib and the right L-bracket was 0.86cm (maximum of 2cm,deviation of 0.79cm) and that of the top rib hole and theleft L-bracket was 0.53cm (maximum 1.2cm, deviation of0.40cm). In all successful cases, the bottom of the rib wasfirmly pressed against both brackets. Note that all thesefigures are significantly less than the precision of globallocalization system. This demonstrates the utility of usingtemporary fixtures and compliance in achieving precisionassembly. We can still further improve these results. Thecurrent visual servoing implementation requires an initialerror of .8cm, which the “close the door” behavior providesin more than half of our trials. In the near term, we willeither improve the visual serving to tolerate a 1cm initialerror, improve the “close the door” behavior to result in asub-centimeter accuracy, or both.

V. A PROPOSED GENERAL ASSEMBLY FRAMEWORK

The overall assembly sequence and each of the componenttechniques described above provide an effective solution tothe example assembly task, however the sequencing and theimplementation of the components relied on the experienceand intuition of designers. It would be desirable to providea more general framework that can be used to inform thecreation of similar assembly sequences. In this section, wepropose such a framework based on the idea of “funneling”.Typically, a manipulation task requires a sequence of actionsthrough which an object is oriented and re-oriented untilit arrives at a desired configuration. This strategy has beeninvestigated in the context of funnels: given a part and a set ofpose constraints, one can prescribe a controller to orient andposition a simple part through a sequence of actions [12],[3]. An individual funnel can be viewed as a policy thathas a domain of initial configurations, a goal set of finalconfigurations, and a controller that directs motion from thedomain to the goal set. This idea was generalized to a conceptcalled sequential composition [1] where feedback controllersare defined over local domains such that their action bringsthe system to a goal set within the domain. One policy,denoted by Φ, is said to prepare another if the goal set ofthe first policy is contained (either wholly or partially) withinthe domain of the second policy. A graph, called a preparesgraph, is formed where a node corresponds to a controller’sdomain and a directed edge from policy one to policy twoindicates the controller of policy one converges on a goal setwhich is a subset of policy two. Once the prepares graph isformed, a planner can search the graph for a path between anode that contains the initial state and a node that containsthe overall goal state.

1570

Drawing inspiration from funneling and sequential com-position, we believe that precision in our mobile frameworkmay be recovered by a sequence of operations, as opposed tojust one. Assume without loss of generality that the controland sensor error of the mobile manipulator can be lumpedinto one error term. Now, lets consider a series of examplesof rib insertion where the robot’s error increases. Startingwith a very small mobile robot error, we can prescribe acontroller for the robot to precisely line up with holes of therib and the bracket. The problem is, such a low-error mobilemanipulator is either impossible to build or too expensiveto purchase. Now, lets increase the error where the mobilerobot is able to drive the rib to the bracket, but lacks theprecision to line up the holes. Instead, the rib holder’s four-bar mechanism lines up the holes using visual feedback, i.e.,the holes are lined up using visual servoing. Once more,lets increase the error of the mobile robot, so that the ribinitially makes contact with the bracket, moves the rib alongthe bracket until the holes are within some proximity of eachother, and then once again relies of visual servoing to finishthe task.

Often, the features of the assembly or those in the envi-ronment will not be of sufficient size to form a funnel. Letsreturn to our example where we increase the lumped errorof the mobile manipulator. Now, the error is larger than thecharacteristic length of the bracket, which means the robotcannot reliably drive until it hits the bracket and then drivealong the bracket until the holes come into proximity of eachother. This is where the spar catchers come in. Since thefeature on the spar is too small for the robot to track, weintroduce temporary features. As can be seen in Figures 3and 6, the temporarily deployed feature engages with thespar, and enlarges the bracket feature size so that the errorof the mobile robot is smaller than the characteristic size ofthe feature. In a sense, we formed mechanical funnels withpassively compliant features mounted on a mobile robot. Wealso form “virtual” funnels using vision and range sensors,also mounted on mobile robots. So, multiple robots willcooperate to form a single funnel. The use of multiplerobots to create multiple funnels requires coordination amongrobots, both at the micro-level to achieve the individuals tasksand at the macro level to optimize the allocation of the tasksto the robots and the robots’ movements.

VI. CONCLUSION AND FUTURE WORK

This paper describes the development of robotic tech-nologies that enable a new vision of collaborative mobilemanufacturing by teams of robots. Of particular interest is theassembly of large structures (e.g., aircraft, ships, constructionequipment) where in current practice, customized facilitiesincorporating extensive infrastructure in the form of fixedtooling, permanent fixtures, and special systems are built toenable efficient, high-precision manufacturing of a specificproduct. To increase capacity or to introduce new productsin these industries, tremendously expensive facilities must bebuilt, which can take years.

Our approach is to replace conventional robot systems andlarge infrastructure with mobile manipulators to carry outthe assembly operation. Conventional robotic manipulatorshave high precision because they are rigidly attached to theground and are made of heavy fixed objects that tend to beinvariant to dynamic parameters such as payload and speed.Mobile manipulators do not enjoy the benefits of high massand rigid fixturing to ensure precision, and hence we needto create mechanisms and control strategies for the mobilemanipulators so they can precisely mate parts for assembly.

Thus far, this project has been driven by experimentation,engineering intuition, and the integration of other roboticsfields such as localization and software architectures forrobots. Through this experience, we have identified tech-niques that, when combined, mitigate the uncertainty inprecision and efficiency that is introduced by replacing thepermanent fixtures with mobile robots. While the efficacyof these techniques has been born out only in the wing-ladder domain, ultimately, our goal is to create a designframework where we can specify an assembly and thenproduce a planner that directs the motions of all of the robotsand their manipulators to carry out the assembly process.This design framework will be mediated by funnels, whichwe seek to express in a sequential composition framework.Sequential composition allows us to prescribe a complexmanipulation task, which requires precision, with a sequenceof intermediate motions, each reducing uncertainty in com-ponent directions.

In the longer term, we seek to extend this frameworkto allow people and machine to cooperate and play totheir strengths in a manufacturing setting. When doing this,one issue to consider is safety. Today, to maintain safetyfor human operators, the overall environment is incrediblywell-structured. This includes human only and robot onlypathways to avoid unnecessary interferences and to simplifymotion planning algorithms. However, when humans mustshare the workspace there needs to be clear warning lightsto indicate robot intentions but these necessary interactionsalso need to have pinch points and other hazards designedout.

REFERENCES

[1] Robert R Burridge, Alfred A Rizzi, and Daniel E Koditschek. Se-quential composition of dynamically dexterous robot behaviors. TheInternational Journal of Robotics Research, 18(6):534–555, 1999.

[2] Marcello Cirillo, Federico Pecora, Henrik Andreasson, Tansel Uras,and Sven Koenig. Integrated motion planning and coordination forindustrial vehicles. In Proceedings 24th International Conference onAutomated Planning and Scheduling, 2014.

[3] Michael A Erdmann and Matthew T Mason. An exploration ofsensorless manipulation. Robotics and Automation, IEEE Journal of,4(4):369–379, 1988.

[4] Matthew Gombolay, Ronald Wilcox, and Julie A Shah. Fast schedulingof multi-robot teams with temporospatial constraints. In Robotics:Science and Systems, 2013.

[5] Satyandra K Gupta, Christiaan JJ Paredis, Rajarishi Sinha, and Peter FBrown. Intelligent assembly modeling and simulation. AssemblyAutomation, 21(3):215–235, 2001.

[6] Hirohisa Hirukawa, Yves Papegay, and Toshihiro Matsui. A motionplanning algorithm for convex polyhedra in contact under translationand rotation. In Robotics and Automation, 1994. Proceedings., 1994IEEE International Conference on, pages 3020–3027. IEEE, 1994.

1571

[7] Xuerong Ji and Jing Xiao. Planning motions compliant to complexcontact states. The International Journal of Robotics Research,20(6):446–465, 2001.

[8] Leslie Pack Kaelbling and Tomas Lozano-Perez. Hierarchical task andmotion planning in the now. In Robotics and Automation (ICRA), 2011IEEE International Conference on, pages 1470–1477. IEEE, 2011.

[9] O Khatib, Kazu Yokoi, K Chang, Diego Ruspini, Robert Holm-berg, and Arancha Casal. Coordination and decentralized coopera-tion of multiple mobile manipulators. Journal of Robotic Systems,13(11):755–764, 1996.

[10] R. Kummerle, G. Grisetti, H. Strasdat, K. Konolige, and W. Burgard.g2o: A general framework for graph optimization. In Robotics andAutomation (ICRA), 2011 IEEE International Conference on, May2011.

[11] Christian Laugier. Planning fine motion strategies by reasoning in thecontact space. In Robotics and Automation, 1989. Proceedings., 1989IEEE International Conference on, pages 653–659. IEEE, 1989.

[12] Tomas Lozano-Perez, Matthew T Mason, and Russell H Taylor. Auto-matic synthesis of fine-motion strategies for robots. The InternationalJournal of Robotics Research, 3(1):3–24, 1984.

[13] Matthew T Mason. Compliance and force control for computercontrolled manipulators. Systems, Man and Cybernetics, IEEE Trans-actions on, 11(6):418–432, 1981.

[14] Brenan J McCarragher and Haruhiko Asada. A discrete event approachto the control of robotic assembly tasks. In Robotics and Automation,1993. Proceedings., 1993 IEEE International Conference on, pages331–336. IEEE, 1993.

[15] W. Meeussen, E. Staffetti, H. Bruyninckx, J. Xiao, and J. De Schutter.Integration of planning and execution in force controlled compliantmotion. J. of Robotics and Autonomous Sys, 56(5):437–450, 2008.

[16] E. Olson. Apriltag: A robust and flexible visual fiducial system. InRobotics and Automation (ICRA), 2011 IEEE International Conferenceon, pages 3400–3407, May 2011.

[17] Anna Petrovskaya and Oussama Khatib. Global localization of objectsvia touch. Robotics, IEEE Transactions on, 27(3):569–585, 2011.

[18] G. Pratt and M. Williamson. Series elastic actuators. In Proceedings1995 IEEE/RSJ International Conference on Intelligent Robots andSystems, pages 399–406. IEEE, 1995.

[19] Brennan Sellner, Frederik W Heger, Laura M Hiatt, Reid Simmons,and Sanjiv Singh. Coordinated multiagent teams and sliding autonomyfor large-scale assembly. Proceedings of the IEEE, 94(7):1425–1444,2006.

[20] Reid Simmons and David Apfelbaum. A task description language forrobot control. In Intelligent Robots and Systems, 1998. Proceedings.,1998 IEEE/RSJ International Conference on, volume 3, pages 1931–1937. IEEE, 1998.

[21] Reid Simmons and Gregory Whelan. Visualization tools for validatingsoftware of autonomous spacecraft. In Proceedings of International.Symposium on Artificial Intelligence, Robotics and Automation inSpace. Citeseer, 1997.

[22] Siddhartha S Srinivasa, Dmitry Berenson, Maya Cakmak, AlvaroCollet, Mehmet Remzi Dogar, Anca D Dragan, Ross A Knepper,Tim Niemueller, Kyle Strabala, Mike Vande Weghe, et al. Herb 2.0:Lessons learned from developing a mobile manipulator for the home.Proceedings of the IEEE, 100(8):2410–2428, 2012.

[23] Glenn Wagner and Howie Choset. Subdimensional Expansion forMultirobot Path Planning. Artificial Intelligence, to appear:1–46, 2014.

[24] D. E. Whitney. Quasi-static assembly of compliantly supported rigidparts. ASME J. of Dynamic Sys., Measure., and Control, 104:65–77,1985.

[25] Randall H Wilson. Geometric reasoning about assembly tools. Artifi-cial Intelligence, 98(1):237–279, 1998.

[26] Randall H Wilson and Jean-Claude Latombe. Geometric reasoningabout mechanical assembly. Artificial Intelligence, 71(2):371–396,1994.

[27] Tony C Woo and Debasish Dutta. Automatic disassembly and totalordering in three dimensions. Journal of Manufacturing Science andEngineering, 113(2):207–213, 1991.

[28] Jing Xiao and Xuerong Ji. Automatic generation of high-levelcontact state space. The International Journal of Robotics Research,20(7):584–606, 2001.

[29] Seung-kook Yun and Daniela Rus. Adaptation to robot failuresand shape change in decentralized construction. In Robotics andAutomation (ICRA), 2010 IEEE International Conference on, pages2451–2458. IEEE, 2010.

[30] Seung-kook Yun, Mac Schwager, and Daniela Rus. Coordinating con-struction of truss structures using distributed equal-mass partitioning.In Robotics Research, pages 607–623. Springer, 2011.

1572