Question: “Can an autonomous Robotic Astrobiologist someday be taught to understand a geological scene like the one pictured below?”

“The Cyborg Astrobiologist: Teaching Computers to Find Uncommon or Novel

Areas of Geological Scenery in Real-time”Network Conference of the Alexander von Humboldt Foundation, Muenster, Germany, November 2008.Authors: Patrick C. McGuire1,5,6,*, Enrique Díaz Martínez1,3, Jens Ormö1, Javier Gómez Elvíra1, Virginia Souza-Egipsy1, Helge Ritter2, Markus Oesker2, Robert Haschke2, Jörg Ontrup2, Florian Schmidt2, Alexandra Bartolo4, Richard Bose5, Lorenz Wendt6

Institutions:

1Centro de Astrobiología (CAB, INTA/CSIC), Instituto Nacional Técnica Aeroespacial, Torrejón de Ardoz, Madrid, Spain: Robotics Laboratory, Planetary Geology Laboratory, Transdisciplinary Laboratory

2Universität Bielefeld, Germany, Neuroinformatics Group, Computer Science Department, Technische Fakultät3(currently at) Dirreción de Geología y Geofísica, Instituto Geológico y Minero de España, Tres Cantos, Madrid, Spain4Engineering Faculty, University of Malta, Malta 5McDonnell Center for the Space Sciences, Washington University, St. Louis, Missouri, USA 6Institute for Geosciences, Freie Universität Berlin, Department of Planetary Science and Remote Sensing, Berlin, Germany

*(corresponding author) Email: mcguire@epsci.wustl.edu

Cyborg Astrobiologist -- Basic Idea:

Real-time Image-segmentation

Real-time Selection of Interest PointsWearable

Computer

Human Mobility

Geologists’ intuition & high-level planning

Video Camera, Firewire communication

The Cyborg Astrobiologist:Geological Field Missions to

Rivas Vaciamadrid, Guadalajara and Malta

McGuire, Ormö et al.,Int’l Journal of Astrobiology(2004)

McGuire, Diaz-Martinez et al.,Int’l Journal of Astrobiology(2005)

Algorithm: Simple Uncommon Map

Simple Image Uncommon MapBrighter = “more uncommon”(For the image on the left)

Mission Riba1: Tripod Position “#2”,Inside the Cyborg Astrobiologist’s ‘Brain’

Cooccurence

2D Histogram

for Saturation

Sat(pix[i][j])

Sat

(pix

[i][

j+1]

)

McGuire, Diaz-Martinez et al.,Int’l Journal of Astrobiology(2005)

Not Yet

Not Yet

Novelty Detection with the FamE (FamiliarityEnergy) Hopfield Neural Net (after R. Bogacz, U. Bristol)

Compute:

averages: <H>,< S>, <I> for each segment

x = (<H>,< S>, <I> )T

If (Energy_Hopfield( x ) < -N/4 then

Novel = FALSE;

Else {

Novel = TRUE;

Store Pattern x in Hopfield Net; }

Repeat for all segments;

Repeat for all incoming images;

Novelty Detection:Cyborg Astrobiologist gets a memory (of Familiar patterns)

The Cyborg Astrobiologist:Ported from a wearable computer to the Astrobiology Phone-cam

Before:

Now:

Bartolo, McGuire et al.,International Journal of Astrobiology(2007)

HA

RD

WA

RE

IM

PR

OV

EM

EN

TS

(Cyborg Astrobiologist)++tested at Rivas Vaciamadrid (Sept 6, 2005)

#1(Cam)

#19(Micro)

#23(Micro)

Lichen=WhiteSporing = RedRock=Brown

Lichen = Black, OrangeGypsum Crystal = SmoothGray

Novel SegmentsOrig (Cam or Micro)

Full-colorImage Segmentation

UncommonMap

Uncommonness

Lichen=Algae + Fungus(in Symbiosis)

Microscope in Field!

Head-mounted Display failed in field

Pictured here: tablet display

Tripod!!

Cyborg Astrobiologist, with microscope, studying lichens

#18(Micro)Lichen=WhiteSporing = RedRock=Brown

FIE

LD

TE

ST

S

Summary of Concept: Developing computer-vision technologies to automate some of a Human Geologist’s low-level thought processes. We are testing these computer vision technologies in the field in real-time with low-cost platforms, with the intention of later putting these algorithms on robotic platforms for exploring the Moon and Mars.

Currently working with collaborators in Madrid, Malta, St. Louis, Bielefeld and Berlin, in order to:

a) Do more comprehensive field tests at sites of geological and astrobiological interest using both the uncommon-map and novelty-detection systems.

b) Make the phone-cam system work with a field laptop enabled with Bluetooth, for automatic transfer of the images from the phone to the laptop and (marked-

up) back to the phone. This should speed up the testing.

Future: 0) Improved image-segmentation (to include texture) 1) Improved Novelty Detection (better statistical measures) 2) Improved cameras (hyperspectral cameras) 3) Improve technological readiness for planetary robotics missions.

CO

NC

EP

T

VIS

ION

AL

GO

RIT

HM

S