Slide 1

PixelLaser: Range scans from image segmentationNicole Lesperance 11Michael Leece 11Steve Matsumoto 12Max Korbel 13Kenny Lei 15Zach Dodds 62

REU1Inspiration

Horswill (polly) 94Saxena (rccar) 052Scans ?

C. Plagemann et al., ICRA 2008platform

"omnicam" imageserrors...3

Cheap!Expensive!Less information: 180 rays of distancesMore information: 640 columns of pixels, each of which we can get a range fromBut its harder (for a computer) to extract data from an image than from a laser range scan. Which is where we come in!Motivation44ClassifySegmentScansMaps!

(in pictorial form!)

obstacleground(and localization)Overall Strategy55Training and Classifying

hand-segmented imagefeatures, in kd-trees

194 191 211 3.2 25.6 4.1 ... 138 87 53 -1.14 8.6 1.4 ...filters6Nearest-neighbor classification

RGB alone doesnt work well

7Nearest-neighborsRGB + texture produces better segmentations

8Classifier VariationsMany attempts were made, but the overall accuracy for reasonable versions hovered around 95%3-tree Classifier: used above, below, and line categoriesGenetic Algorithm: attempted to find the optimal weight values, but optimized for the set of images used to evolve the classifier. The weight-set did not generalize to other environmentsDifferent Environments: the classifier was tested in environments varying in difficulty 9Overall Accuracy ResultsRegardless of the environment, the Classifier performs consistently

10Hallway: Overall Accuracy 95.40 % Library Downstairs: Overall Accuracy 94.01 %

11FLANN (Fast Library for Approximate Nearest Neighbors)C++ knn implementation with variable precision levelsDecreased kd-tree lookup time by 2 orders of magnitudeAllows for more complex real-time segmentation algorithms Allows the robot to autonomously wander in real time using only the segmenter

Marius Muja: Our hero12Now Arriving atSegmentation!Uses the information from the classifier to draw the line between the ground and obstacles

Sends the information to the scanner to make laser range scans

Classify Segment Scan Map13Initial approaches

bottom-up

patch pairs

multiresolution14Transition SegmentationUses transition strength rather than classificationAssigns transition strength based on the certainty that the below patch is traversable and the top patch is notTakes the difference of the above strength of the top patch and the below strength of the bottom oneIssuesMisclassifications can result in incorrectly strong transitions

15Transitions using a Line TreeThe strongest transitions are found and the area closest matching the line category is chosen

16Seam CarvingStart at one end of the image and find the strongest transitionAssign a cost to deviating from the line height in the previous column

17Smoothing Certainty LocallyCalculate certainties for all patches in the imageApply averaging filter on the certainties to suppress rogue misclassifications Look for the strongest transition in each column and draw line

18Edge DetectionGlobal image edge detection is not useful, but snapping to the nearest edge in the strongest transition area reduces segmentation error

19ResultsSprague LabSprague First FloorLibra HallwaysMulti-resolution9.6716.2419.75Smooth7.445.1022.11Transition Line4.124.5214.27Sprague LabSprague First FloorLibra HallwaysMulti-resolution12.2816.0564.52Smooth11.089.8130.15Transition Line11.018.7222.78

Average Pixel Error per ColumnAverage Distance Error per Column (inches)

20Results - MovingAverage Pixel Error per ColumnSprague LabSprague First FloorLibra HallwaysMulti-resolution8.8611.1314.10Smooth3.627.9910.34Transition Line2.796.0814.01Sprague LabSprague First FloorLibra HallwaysMulti-resolution27.3435.1055.34Smooth17.6433.0159.18Transition Line19.6836.4575.28Average Distance Error per Column (inches)

21

From segmentation to scan

pixel row numberrange

row-to-range mapscansegmentation22Scan MatchingMatch two consecutive laser scans and find the corresponding transformationCan be used to correct poor odometry

23SLAM Map BuildingCombine all laser scans to buildmaps using CoreSLAM (simultaneous localization and mapping)

24Monte Carlo LocalizationCreate random list of possible locations of robotMonte Carlo Localization Cycle:Move: Move all points based on change in odometrySense: Compare the laser scan from segmented image with laser scans at possible locationsResample: Redistribute points based on probability at each location25Monte Carlo Localization

26Point and Click NavigationGiven a known map, we want to be able to click on a location in the map and have the robot navigate thereThe current implementation works only if there is a straight, uninterrupted path between the current location and the destination (havent implemented a path planning algorithm)Things to think about include navigation around obstacles and odometry correction27Autonomous Wandering28Future WorkSegmenting with a projected laser level line possibly to be used for automated training as wellStabilizing the camera keeping the horizon constant while moving is almost impossibleSpeeding up the wandering speed a more stable camera would help with thisAutonomous mapping29Questions?

30Segmentationclassification + confidencesegmentations

31

../TrainingImages/Playspacepswo13Patches/00029/randomBelow/0009.png 194.2575 191.4525 211.4775 195.0 192.0 212.0 8.67070895314 8.76885076564 7.29105573631 211.4775 191.4525 194.2575 212.0 192.0 195.0 7.29105573631 8.76885076564 8.67070895314 3113.18473022 2918.61969259 194.707808243 -8.2951355554e-09 -2.99999999989 3.48871383089 -0.1821058002 1.05861940504 0.798119246226 0.142200402323 1.37643522494 1.51710154734 -3.21108530262 25.6053670274 4.08979546594 0.20963020652 3.42407524192 7.98287776033 0.655198704147 1.40172476457 1.2862398764 -0.068793854887 7.97682628048 3.32687477389 -0.83219461218 2.58844742173 1.83432937157 -0.239667915222 1.22096756236 3.56803311807 0.0253655120168 0.415881369811 7.11907142389 -0.829352050768 1.14166676605 4.1432245279 -0.33452013644 -0.0121915477333 6.30267114206 -0.714448599523 0.711384510758 3.38445323448 -0.566103623094 -0.00278409428233 5.92992221434 -0.932328715083 1.17018711051../TrainingImages/Playspacepswo13Patches/00026/randomabove/0007.png 138.1075 87.9525 53.4725 138.0 88.0 53.0 3.65184114523 3.44532200962 5.05512054752 53.4725 87.9525 138.1075 53.0 88.0 138.0 5.05512054752 3.44532200962 3.65184114523 1584.29001984 1485.27385943 99.0473083038 2.67691915514e-09 -2.99999999998 0.839568062542 -0.0913685033632 5.68745353063 0.334375851147 -0.0494906892805 5.40416976808 0.617058294109 -1.14642893693 8.69695694778 1.38147297404 -0.87936321568 4.10330225187 1.59491339286 -0.69926257399 0.679234808869 0.526708837532 -0.253691320988 7.92226252642 0.619482921372 -0.155302887182 1.51598656224 0.455410613623 0.896134364593 4.90251778762 1.44298148203 -1.32247289798 1.54652876227 1.82381743704 1.10446810418 0.850926709651 1.23981013329 -0.346508022218 -0.252958061091 1.53636914298 1.26906228604 1.09496630534 1.45689240135 1.4011501228 1.85701734954 1.68840166501 1.0053643485 0.471348780351

194 191 211 3.2 25.6 4.1 138 87 53 -1.14 8.6 1.432

33