Estimation of Surface Geometries in Point Cloudsfor the Manipulation of Novel Household Objects

Siddarth Jain and Brenna ArgallNorthwestern University, Evanston, IL, USA 60208Shirley Ryan AbilityLab, Chicago, IL, USA 60611

Email: sidd@u.northwestern.edu, brenna.argall@northwestern.edu

Abstract—Real world environments typically include objectswith different perceptual appearances. The classification of objectpoint cloud clusters to surface and primitive geometries can bevery useful in manipulation planning. In this work, we presentand evaluate the utility of 3D surface estimation methods toefficiently classify the local and global surface geometry in thepoint cloud representations of household objects. We also presenta Global Principal Curvature Shape Descriptor (GPCSD) to cate-gorize object point clusters that are geometrically equivalent intosimilar primitive shape categories and compare its performanceto Global Radius-Based Surface Descriptor (GRSD) on a largescale RGB-D object dataset and real-time data from the Kinect.Furthermore, we present a manipulation framework for graspplanning and selection on unknown objects based on geometricsurface estimation. The method uses 3D sensor data directly toproduce a set of grasps for the objects in a scene based onboth the overall shape of the object and its local features. Theframework provides kinematically feasible grasp selection andranking for manipulation planning from the candidate grasps,which is suitable for both autonomous grasp execution andshared-control assistive teleoperation.

I. INTRODUCTION

Environment perception is crucial for the accomplishmentof manipulation tasks by robots. In some sense the robotperception system can be trained on object models and thuscan specialize to perceive and manipulate a set of objects.However, the great diversity in the types and classes of objectspresented by unstructured environments makes it difficultto provide and use accurate and/or comprehensive modelsof all relevant objects that can be encountered in everydaymanipulation tasks.

The set of everyday objects that a robot could encounterin its tasks is vast and diverse as household objects varyin their size, color, texture and shape. However, there existcertain regularities that can be exploited to inform and helprobot perception in order to act on novel objects. In particular,the 3D surface geometries of objects contain characteristicinformation that generalizes over a large variety of objects andthus can provide useful information for recognition as well aseveryday manipulation.

Geometrical analysis of a point’s neighborhood can providediscriminative information about the local surface type. Thelocal surface geometries can be characterized by basic surfacetypes in the 3D point clouds of objects, such as corner,cylindrical, edge, planar and spherical. The identification ofsuch local surface types in object point clouds can be utilized

to reason about the shape and global surface characteristics ofthe object—without the need to recognize the specific objectinstance type or category.

Such local and global surface geometry information canbe used in manipulation planning for the robots. Thus, theaddition of this semantic information to the 3D representationof object point clouds can enable the robot perception systemto operate in a more general way—enabling the robots toperceive and manipulate unseen objects with more flexibilityand reliability.

In this paper, we investigate methods for the estimation oflocal and global surface geometries in the 3D point cloudsof household objects. We furthermore present a frameworkfor perception and manipulation based on such geometriccharacteristics, without the need to recognize a specific objectcategory or instance.

In particular, the utility of Principal Curvatures (PC) [17]and the Radius-based Surface Descriptor (RSD) [14] are ex-amined for their effectiveness in local surface estimation frompoint clouds of household objects. A global principal curvaturebased shape descriptor (GPCSD) is generated based on thelocal PC values. The computation is similar to GRSD [14],but we use voxel-based labeling based on Principal Curvatures(instead of the RSD). The performance of global descriptorsGPCSD and GRSD, in concert with artificial neural networks,in the classification of geometric characteristics of objects areevaluated on a large object dataset and also on real-time sensordata. Furthermore, the utility of the classified shape charac-teristics are presented in a framework for the manipulationof household objects. The framework provides kinematicallyfeasible grasp selection and ranking for manipulation planning,which is suitable for both autonomous grasp execution andshared-control assistive teleoperation.

The remainder of the paper is organized as follows. Re-lated work is described in Section II. Section III explainsthe methods for surface estimation. Section IV presents theclassification and evaluation of global descriptors. A manip-ulation framework utilizing shape characteristics is presentedin Section V. In Section VI, we present the conclusions.

II. RELATED WORK

In recent years, different descriptors for point clouds havebeen developed with the goal of addressing problems related

to object recognition and 3D registration. Point Feature His-tograms (PFH) [17] were proposed as an informative pose-invariant feature for geometrical surface description. FastPoint Feature Histograms (FPFH) [17] were later deployedto reduce the computational complexity of PFH for real-timeapplications. The Radius-Based Surface Descriptor (RSD)was proposed to describe the geometry of points in a localneighborhood using surface normals by estimating radial re-lationships. The utility of principal curvatures estimation thatrelies on surface normals is shown [17] to represent geometric3D edges and region growing segmentation in point clouds.These features perform local surface categorization, whereeach point is associated with a descriptor able to describethe local geometry of that point. By contrast, global featurehistograms (for example, GRSD [14]) are high-dimensionalrepresentations that capture the notion of objects as wholefor the purpose of object recognition. For a more detailedinformation on features we refer the readers to [1].

The utility of surface classification using local features ismostly demonstrated on data obtained with laser scanners [17],which are less prone to noise. A comparative evaluation [3]of FPFH, RSD and PC local descriptors on 8 indoor scenescaptured with a less accurate 3D sensor (ASUS Xtion PROLIVE) finds FPFH to be better for complex shapes but tohave trouble dealing with noisy data. RSD and PC performsimilarly, but PC is found to be more robust and smooth forlocal scene classification.

In this work we consider the local surface labels for geomet-ric classification of points, but more importantly we evaluatethe global feature representations from the local surface labelsto determine the overall shape characteristics. Furthermore,the majority of the existing evaluations tackle the less generalproblem of specific object recognition using descriptors [2].We intend to perform a more general classification of objectsinto global primitive shapes, which can be used to informperception for grasp generation, selection and execution onnovel objects.

Grasp planning has been widely studied in the literature.For a more detailed review of the field, we refer the readersto survey articles [5, 18]. Similar to this work, there exist ap-proaches that explore shape primitives and depth data in graspplanning. Miller et al. [15] present the use of shape primitivesfor object grasping and their approach relies on having knownobject models based on which the grasps can be analyzed insimulation. Jain and Kemp [9] plan overhead grasps using a setof simple heuristics. Ten Pas and Platt [20] propose a geometrybased method for generating grasp candidates by samplingfrom a large set of grasp hypotheses. Hsiao et al. [8] searchfor orthogonal grasps by fitting a bounding box primitive onobject point clouds and produce top and side grasps based ona small set of simple feature weights. Jain and Argall [10] fitprimitive shapes to cylindrical and spherical objects in additionto the box shapes for planning top, side and pinch grasps, usingsurface variation [17] to determine the best fit primitive shape.

In this work, we evaluate and explore more robust surfacegeometry based approaches for determination of object shape

characteristics and present a manipulation planning frameworkfor grasp selection, ranking and execution on novel objects.The determination of shape characteristics is a three stepprocess: (1) computation of local features, (2) aggregatingthe local features across the entire object (called a globaldescriptor) and (3) classifying the object, based on the globaldescriptor, into a primitive shape type. Next, based on theidentified primitive shape and estimated pose, grasps aregenerated on objects, which are further ranked and selectedin manipulation planning.

III. SURFACE GEOMETRY ESTIMATION

The following section gives an overview of surface geome-try estimation methods for point clouds. First, we discuss localsurface geometry estimation approaches (such as RSD and PClocal features), and then a global descriptor based approach todetermine the overall shape characteristics of object clusters.These shape characteristics then will be used (in Section IV)to classify the object into a primitive shape class.

Methods based on the Radius-based Surface Descriptor(RSD) and Principal Curvatures (PC) have shown good perfor-mance on classification of local geometry in point cloud scenes(Sec. II). Thus, we select these local features and compare theirperformance on point clouds of household objects.

RSD [14] describes the geometry of points in a localneighborhood by estimating their radial relationships. Theradius estimation is performed by assuming each point pairto lie on a sphere. By exploiting the relation of the distanced between the points and the angle α between the two pointnormals, the radius r is estimated,

√2r√1− cos(α) = d,

r ≈ d/α.

The radius approaches infinity for a planar surface and takes onincreasingly lower values for surfaces with a higher curvature.The RSD feature for a point, based on its nearest neighbors,consists of a minimum radius and maximum curvature radius[rmin, rmax] taken from all the point-neighbor spheres.

PC [17] describes a point’s local surface geometry fromthe eigenvectors and eigenvalues of the principal surfacecurvatures on that point. For a query point pq ∈ R3, allnormals nj ∈ R3, j = 1...N , of the set of its N neighborhoodpoints are projected onto the tangent plane of the surfacedefined by the normal nq at the query point. The centroidof the projected points p ∈ R3 and the covariance matrix Cfrom all projections are computed as,

p =

∑Nj=1 pj

N,

C =1

N

N∑j=1

(pj − p) · (pj − p)T

wherepj = (I − nq · nT

q ) · nj ,

and I is a 3×3 identity matrix.

Fig. 1. Surface geometry estimation for an example object cluster (cerealbox). Local feature class labels shown as colors on the point cloud (Left: RSD,Right: PC). Global descriptor histogram bins shown as plots (Left: GRSD,Right: GPCSD).

Principal Components Analysis (PCA) [12] is then per-formed on the point normals of the surface patch in the tangentplane of the given point normal, to find the eigenvalues.The eigenvalues λk ∈ R and eigenvectors vk ∈ R3 aredefined by the following relation and form an orthogonal framethat corresponds to the principal components of the set ofneighborhood points:

C · vk = λk · vk, k ∈ {0, 1, 2}

where λ3 corresponds to the maximum curvature and λ2 to theminimum curvature denoted as pcmax and pcmin, respectively.The PC feature consists of these minimum and maximumcurvatures [pcmax, pcmin] as well as the principal curvaturedirection v3, the normalized eigenvector of pcmax and pcmin.

Using the RSD and PC values, a geometric class label canbe assigned to each point in the object cluster. To determinethe class label, thresholds on the feature values of the RSDdescriptor are used to categorize local surfaces into classesplanar, cylindrical, spherical, edge, corner or noisy. In par-ticular, for each voxel with a width of 2.5cm, we computethe minimum and maximum curvature radii [rmin, rmax] andlabel the voxel surface by successive checks of the RSD radiiin order to categorize it into one of the geometric classes. Thisis a simple and fast way to categorize local surfaces, and itdivides the feature space formed by the radii well enough forthe computation of a global feature such as GRSD.

We apply a rule-set of classification thresholds (empiricallydetermined) in order to classify local geometries in the pointclouds of household objects. Since RSD and PC are based ona similar geometrical approach (highest and lowest curvature),this thresholds technique is applicable to classify the PC values[pcmax, pcmin].

A global representation of object type then is made from thelocal features (geometries). Specifically, for the RSD feature,once all voxels are annotated locally with a geometric class,the Global Radius-based Surface Descriptor (GRSD) [14] iscomputed, which produces a unique signature for a givenobject cluster based on the RSD local feature values. GRSDdescribes the transitions between different surface types (andfree space) for an object. Note that in this work we countthe transitions between surface types between the occupied

voxels instead of along lines between occupied voxels. Eachbin counts a transition between a pair of specific surface types,or between a specific surface type and free space. By usingfive local surface types (planar, cylindrical, spherical, edge andnoisy/corner) and considering free space, the number of GRSDhistogram bins is 21.

Following a similar approach, for the PC feature, we intro-duce a global histogram that counts transitions between thevoxelized PC labels within an object cluster and also includebins for the number of voxels labeled as planar, cylindricaland spherical (empirically we found this to slightly increasethe shape classification performance), making the bin size 24.We call this new assembled descriptor the Global PrincipalCurvature Shape Descriptor (GPCSD). Figure 1 shows thesurface estimation on a cereal box object cluster.

IV. GLOBAL SHAPE CLASSIFICATION AND EVALUATION

As noted above, we are interested in the global featurerepresentations for objects and only compute local surfacelabels in order to aid in this global classification. We considerthree primitive geometry classes to represent many objectsthat a robot could perceive and manipulate in householdenvironments: box-shaped, cylindrical and spherical.

We select 12 object categories from the large scale RGBD-Washington dataset [13] to evaluate the mapping from globaldescriptors of object type to the primitive shape classes. Foreach category, there are four separate object instances in thedataset and each instance has more than 600 views of theobject from different angles. Figure 2 shows an instance ofeach object for all categories along with the ground truthclass labels. Note that RGB values from the dataset are onlyrepresented for illustration are not utilized in the descriptorcomputations. GRSD and GPCSD descriptors are computedfor all instances of each object in the 12 categories.

Fig. 2. One instance of each object category in the dataset used for evaluation.

Fig. 4. Framework for surface estimation and manipulation of novel objects.

Fig. 3. Normalized confusion matrices for GRSD and GPCSD classificationusing ANN models on the test set objects. Class labels are Box-shaped (B),Cylindrical (C) and Spherical (S).

An artificial neural network (ANN) is trained on threeinstances of each object category in the dataset, with one in-stance being randomly picked to leave out from each categoryin order to form the test set. The ANN structure has 21 inputnodes for the GRSD descriptor (one node for each histogrambin) and 24 for the GPCSD descriptor, one hidden layer with10 nodes and three output nodes for the class labels. (Note thatmultiple network structures were tested to prevent overfittingbefore arriving at this structure.)

Both models perform well on the unseen test set objectsand the classification accuracy of the learned GRSD modelis 94.93% and that of the GPCSD model is 91.02%. Thenormalized confusion matrices are shown in Figure 3. Themain performance difference between the two approachesappears to be the occasional mislabelling of spherical ascylindrical (1% for GRSD and 8% for GPCSD). These global

results are in contrast to findings [3] where PC performs betterthan RSD for local surface classification on point cloud scenes.

We further test the performance of the learned models onlive data from a Kinect sensor. Each object instance from thetest set is presented one by one in front of the Kinect atdistances of 100 cm, 125 cm and 150 cm from the sensor.We use the same trained models obtained from the RGB-Ddataset to query the segmented object clusters. Both modelsperform well on the live test data, but we find the GRSD modelpredictions comparatively more robust to distance variations.

V. MANIPULATION PLANNING

We now present a framework that uses shape estimation(Section IV) from real-time sensor data for grasp planning onnovel objects. Our framework is overviewed in Figure 4. Thefirst objective is to find a combination of gripper orientationsand positions relative to a given object using the sensory dataavailable to the robot. The sensory input to our framework is3D point cloud data of the scene from a Kinect RGB-D sensor.

We first identify the parts of the input point cloud that arelikely to belong to a single object (clusters). Two simplifyingassumptions are made: the objects are sitting on a flat surfacesuitable for manipulation (such as a table), and the minimumdistance between two objects in a scene is at least 3cm.We compute a planar fit using Random Sample Consensus(RANSAC)1 to generate a model hypotheses for the flatsurfaces and extract the dominant planar surface that providessupport for the objects. We then find the individual objectclusters O in the scene by performing Euclidean clustering.

1Our implementation uses the RANSAC implementation provided withPoint Cloud Library (PCL).

Fig. 5. Sample grasps on a scene of objects. The predicted shape primitives are shown (center) along with the grasps produced on the objects (right). Thered arrow points along the approach direction, and the green box is positioned along the axis in which the gripper closes.

Fig. 6. Pose estimation and grasp generation on a variety of novel objects. The segmented point clouds (middle row) are fit to primitive shapes (bottomrow) and candidate grasps are generated. Grasp approach and orientation indicated by the red arrows and green boxes, as in Figure 5.

Next, we compute the global surface descriptor (GRSD)2

for each segmented object cluster and query the learned ANNmodel to classify the cluster to a primitive shape category. Amodel hypothesis of the geometric model parameters for theidentified shape category is generated using RANSAC. Theobtained model then is refined using a linear least-squares fitfor the plane and nonlinear Levenberg-Marquardt optimizationfor the cylindrical and spherical primitives.

The framework generates, for every observed object pointcloud cluster O, a set of possible grasps G. A study involvinggrasping by human users [4] and pilot experiments of assistiverobotic arm teleoperation [10] show that humans tend to graspwith wrist orientations that are orthogonal to the object’sprincipal axis and its perpendiculars—the hypothesis beingthat such grasps often are more stable. We generate Top, Sideand/or Pinch grasps for each object cluster around the modeledshape centroid, by using an established grasp generationheuristics [10]. Note that here we represent each grasp g ∈ Gas a Cartesian position and an approach direction quaternion,in the robot base frame. The grasps can readily be used as amotion planning goal for a path planner, in order to execute

2Here our implementation uses GRSD only as the global surface descriptor,because of its slightly superior performance in Section IV.

the grasp on the desired object.Figure 5 shows examples of the grasps generated by our

method on point clouds of household objects. The primitiveshape is estimated for segmented object clusters using theGRSD feature neural network model, and the grasps aregenerated according to the heuristics described above. Figure6 shows the performance of our approach on novel objects.The framework is able to identify shape primitives, estimatetheir pose and generate grasps for a variety of different objecttypes and shapes. Note that these object types do not exist inthe set used to train the ANN model—they are not just newinstances of a known object type, but rather are entirely newobject types.

For manipulation planning, it is desirable to first evaluategrasps that are more likely to be physically reachable by therobotic arm. Since there is likely more than one solution in G,it is preferable to choose grasps such that the planning solutionis quickly found. Furthermore, grasp approach directions thatminimize the distance between the end-effector and the objectcentroid are preferred.

We compute metrics with the aim to prefer grasps thatresult in shorter planning times by incorporating the robot armkinematics. For this consideration, we compute the condition

Fig. 8. Intent inference on grasps for shared-control assistance. The grasps preferred by the human operator g∗ (shown in orange) are predicted based onthe configuration of the robot and the possible grasps (shown in green) generated on the object.

Fig. 7. An example of quality metric κ computed for side and top graspconfigurations. Under our approach, the side grasp would be preferred sincea larger κ value indicates that the arm configuration for the grasp is moreisotropic and farther from a singularity.

number κ as the ratio of the smallest and the largest singularvalue of the Jacobian at configuration q,

κ =σmin,q

σmax,q.

If the value of the condition number κ is close to one,the manipulability ellipsoid is isotropic—indicating that themanipulator has the same motion capability in all task spacedirections. On the other hand, small values of κ indicatecloseness to a singular configuration. Therefore, by using κas a quality measure, grasp configurations far away from asingular one can be preferred (Figure 7). For a grasp forwhich multiple candidate arm configurations q might exist,the average manipulability for a set of collision-free armconfigurations qi that achieve g instead is computed,

κ =1

N

N∑i=1

σmin,qi

σmax,qi

where σmin,qiis the smallest singular value and σmax,qi

isthe largest singular value of the Jacobian at configuration qi.Note that additionally only collision-free grasps that reach theobject are considered by performing environment check.3

If the robot is operating with full autonomy, the grasp g∗

that maximizes κ,

g∗ = argmaxg∈G κ(qg)

is selected, where qg is the arm configuration at grasp g(obtained via inverse kinematics). Sampling-based planners

3Our implementation uses the Flexible Collision Library (FCL) in MoveIt!for collision checking.

from the OMPL [19] library then are utilized to plan optimizedmotions for grasping and manipulating objects.

Another target application domain for this work is to provideassistance in a shared-control framework. In shared controlteleoperation [6, 7, 11, 16], both the human user and therobotics autonomy control the robot, in order to achieve theuser’s desired goal. The grasps generated by our frameworkare suitable candidates for the potential user goals and canthus be used readily in the shared control framework.

One key challenge in shared teleoperation is to infer theuser’s desired goal in order to inform the robotics autonomy.In our approach the user-preferred grasp g∗ out of possiblechoices G is inferred based on: (1) the Euclidean distancebetween the robot end-effector position and the grasp position,and (2) the end-effector orientation alignment with the grasppose. Once the end-effector is in the vicinity of an object, thenearest and best aligned grasp pose then is selected. Figure 8shows examples of grasp intent inference in shared-autonomy.

VI. CONCLUSION

In this paper, we have explored methods for local and globalsurface estimation in point clouds of household objects. A newglobal descriptor, GPCSD, was introduced and its performancewas compared to the GRSD descriptor for the estimation ofglobal shape from object clusters. Evaluation was performedon a large scale RGB-D dataset and also on real-time data fromthe Kinect using a neural network to classify simple shapeprimitives. Both descriptors performed well, but GRSD wasfound to be more robust to distance variations. Furthermore,we presented a manipulation framework for grasp selectionand execution on novel objects. The grasp selection methodused the 3D sensor data directly to determine a ranked setof grasps for objects in a scene based on the overall shapeof the object and its local features. Our results showed thatthe presented framework is capable of correctly estimatingthe shape of unseen household objects and producing multiplecandidate grasps which can be ranked and selected basedon their kinematic feasibility and also on inferred intent forshared-control teleoperation. Our next steps will be to evaluatethe manipulation framework in a shared-control system thatincludes an end-user study.

ACKNOWLEDGMENT

Research reported in this publication was supported by theNational Institute of Biomedical Imaging and Bioengineering(NIBIB) and National Institute of Child and Human Devel-opment (NICHD) under award number R01EB019335. Thecontent is solely the responsibility of the authors and doesnot necessarily represent the official views of the NationalInstitutes of Health.

REFERENCES

[1] Aitor Aldoma, Zoltan-Csaba Marton, Federico Tombari,Walter Wohlkinger, Christian Potthast, Bernhard Zeisl,Radu Bogdan Rusu, Suat Gedikli, and Markus Vincze.Tutorial: Point cloud library: Three-dimensional objectrecognition and 6 dof pose estimation. IEEE Robotics &Automation Magazine, 19(3):80–91, 2012.

[2] Luıs A Alexandre. 3D descriptors for object and categoryrecognition: a comparative evaluation. In IROS 2012Workshop on Color-Depth Camera Fusion in Robotics,2012.

[3] Georg Arbeiter, Steffen Fuchs, Richard Bormann, JanFischer, and Alexander Verl. Evaluation of 3D feature de-scriptors for classification of surface geometries in pointclouds. In Proceedings of the IEEE/RSJ InternationalConference on Intelligent Robots and Systems (IROS),2012.

[4] Ravi Balasubramanian, Ling Xu, Peter D Brook,Joshua R Smith, and Yoky Matsuoka. Human-guidedgrasp measures improve grasp robustness on physicalrobot. In Proceedings of the IEEE International Con-ference on Robotics and Automation (ICRA), 2010.

[5] Jeannette Bohg, Antonio Morales, Tamim Asfour, andDanica Kragic. Data-driven grasp synthesisa survey.IEEE Transactions on Robotics, 30(2):289–309, 2014.

[6] Anca D Dragan and Siddhartha S Srinivasa. A policy-blending formalism for shared control. The InternationalJournal of Robotics Research, 32(7):790–805, 2013.

[7] Deepak Gopinath, Siddarth Jain, and Brenna D Argall.Human-in-the-loop optimization of shared autonomy inassistive robotics. IEEE Robotics and Automation Let-ters, 2(1):247–254, 2017.

[8] Kaijen Hsiao, Sachin Chitta, Matei Ciocarlie, and E GilJones. Contact-reactive grasping of objects with partialshape information. In Proceedings of the IEEE/RSJ In-ternational Conference on Intelligent Robots and Systems(IROS), 2010.

[9] Advait Jain and Charles C Kemp. EL-E: an assistivemobile manipulator that autonomously fetches objectsfrom flat surfaces. Autonomous Robots, 28(1):45–64,2010.

[10] Siddarth Jain and Brenna Argall. Grasp detection for as-sistive robotic manipulation. In Proceedings of the IEEEInternational Conference on Robotics and Automation(ICRA), 2016.

[11] Siddarth Jain, Ali Farshchiansadegh, Alexander Broad,Farnaz Abdollahi, Ferdinando Mussa-Ivaldi, and BrennaArgall. Assistive robotic manipulation through sharedautonomy and a body-machine interface. In Proceedingsof the IEEE International Conference on RehabilitationRobotics (ICORR), 2015.

[12] Ian Jolliffe. Principal component analysis. Wiley OnlineLibrary, 2002.

[13] Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. Alarge-scale hierarchical multi-view rgb-d object dataset.In Proceedings of the IEEE International Conference onRobotics and Automation (ICRA), 2011.

[14] Zoltan-Csaba Marton, Dejan Pangercic, Nico Blodow,and Michael Beetz. Combined 2D–3D categorizationand classification for multimodal perception systems.The International Journal of Robotics Research, 30(11):1378–1402, 2011.

[15] Andrew T Miller and Peter K Allen. Graspit! a ver-satile simulator for robotic grasping. IEEE Robotics &Automation Magazine, 11(4):110–122, 2004.

[16] Katharina Mulling, Arun Venkatraman, Jean-SebastienValois, John Downey, Jeffrey Weiss, Shervin Javdani,Martial Hebert, Andrew Schwartz, Jennifer Collinger,and Andrew Bagnell. Autonomy infused teleoperationwith application to bci manipulation. In Proceedings ofRobotics: Science and Systems, 2015.

[17] Radu Bogdan Rusu. Semantic 3D object maps foreveryday manipulation in human living environments.KI-Kunstliche Intelligenz, 24(4):345–348, 2010.

[18] Anis Sahbani, Sahar El-Khoury, and Philippe Bidaud.An overview of 3d object grasp synthesis algorithms.Robotics and Autonomous Systems, 60(3):326–336, 2012.

[19] Ioan A Sucan, Mark Moll, and Lydia E Kavraki. Theopen motion planning library. IEEE Robotics & Automa-tion Magazine, 19(4):72–82, 2012.

[20] Andreas ten Pas and Robert Platt. Using geometry todetect grasp poses in 3D point clouds. In Proceedingsof the International Symposium on Robotics Research(ISRR), 2015.