Navigating with an animal brain: a neural network for landmarkidentification and navigation.

Philippe Gaussier * , Stéphane Zrehen *** ENSEA ETIS, 6 Av du Ponceau, 95014 Cergy Pontoise Cedex, France

** Laboratoire de Microinformatique, EPFL-DI, CH-1015 LausanneTel : ++ ++ 1 30 73 66 13

EMmail : gaussier@ensea.fr, zrehen@lamisun.epfl.ch

1 Introduction

Navigating in an unknown environment is a taskcommonly accomplished by most animals.Nevertheless, it is not justified to infer that thiscapacity needs complex reasoning involving abstractgeometrical computations. Indeed, it is hard toimagine that ants or wasps have such knowledge,while they are clearly able to locate a source of food,and then go back and forth between that source andtheir home. In this paper, our aim is to show thatsuch behavior, including switching between goals,can be simulated by simple artificial NeuralNetworks (NN) where no complex computation isperformed. We will present a real development andsimulations about a Khepera™ robot (fig. 1) and asimulated system named Prometheus.

Figure 1: The Khepera™ robot developed at theLAMI [Mon93].

We use a novel neural architecture named PerAc(Perception-Action) which is a systematic way todecompose the control of an autonomous robot in

perception and action flows [Gau94b]. We show thataction simplifies the interpretation of perception:each action is a choice and conditions entirely thefuture of the robot. That way, acting in the world isnecessary to the categorization and hence theinterpretation of perceived signal, i.e., to theemergence of an elementary "cognition". Thegreatest advantage of this type of approach is that itmakes cognition sequential, thereby avoiding thepossible large duplications and relaxationmechanisms needed by massively parallel systems.We also focus on the interest to perform anautonomous on line learning of the relevant placesto the robot in its environment. Furthermore, wecompel ourselves not to touch modify the internalstructures of the artificial robot "brain" by hand,while it operates. Thus we have to:

- design a self modifiable connection diagram.- pay attention to the self adaptability of each simple

block to data variations.- allow the robot to use the signals correlations

which are really relevant to it.- introduce a limbic system to control the robot's

learning, motivation and behaviors.

We emphasize the interest of a constructivistapproach [Mat87], [Ste91] as implemented forinstance by the subsumption architecture [Bro86]. Aspecial stress is put on introducing goal resolution inour biologically plausible model of vision andnavigation system.We propose a design for an extremely simple limbicsystem (the one mainly involved with emotions inour own brains), in order to deduce the overallstructure of the neuronal basic element. This leadsus to use the concept of simulated cortical columnfirst proposed by [Bur89].In a first part, we briefly sum up the characteristic ofthe PerAc architecture and we show how it can be

used to extract localization information from avisual scene. Next we show how a robot can learn toreturn from any starting point to a previouslydiscovered and learned position without any a priorisymbolic representation. At last, we simulate acomplete behavior consisting in avoiding dangerouszones to go to "eat" and next to return at home whenthe robot is "tired."

2 The PerAc building architecture

We have already realized robot 'brains" with simpleconditioning that allows them to learn sensorysituations and at the same time to learn whatmovement must be performed. For instance, whenthe robot collides in a wall a pain signal is emittedand a reinforcement learning rule is used to increasethe synaptic weights in order to avoid obstacles onfollowing similar situations [Gau94a]. The samemechanism is involved to recognize objects. Therobot's eye is able to move its eye from one point tothe other in a perceived scene [Gau92]. It learns anobject as a sequence of local recognitions associatedto particular ocular saccades. All this system issimulated with unsupervised neural networks. Theoutput of the visual system, that is, the localrecognitions and the angular movement to go fromone focus point to the other are used as inputs toanother part involved in targets retrieval. The localrecognition is associated to the identification oflandmarks in the visual environment and the ocularmovements provides information about the anglebetween two landmarks. A simple neuron called"place cell" can then learn a particular location[Zip85] and react according to the proximity of therobot to this stored location.In our system, all the neural groups involved aremodelled by self organized topological maps[Gau94a]. This implies that our robot is able to storenew information near similar ones previouslylearned. Then, a lateral diffusion mechanism allowsto react to the new learned shape in the same way asfor the previous ones. Whenever the reaction iswrong, the robot can learn to refine is classificationaccording to the action that must be performed. Prometheus' “ brain ” architecture is summarizedon fig. 2. The same neural architecture is used torecognize an object or a landmark and to control therobot movements. The PerAc blocks of which it ismade appear to be a kind of basic building block anda systematic tool to combine motor and perceptiveinformation. Perac architecture relies on thepostulate that the recognition of any cue can be

simplified if the system can act on it. This justifiesto cut any perceived cue into two parts: a) a motorpart which is the result of a hardwired conventionalprocessing and b) a cognitive one which proposes tolearn/record important situations and to allow aquicker adaptation of the system's response.

VisualPerception

ocular saccadesor/and

head movements

Perception Integration

Action

PanoramicPerception

RobotMovements

Recognition

Action

Recognition

Action

PerAc BlockPerAc Block

EnvironmentPerception

Figure 2: The complete neural network for targetretrieval modelled by PerAc blocks

Our model also offers an alternative to the classicalscheme of hierachical classification because weintegrate not only static perceptive recognitioninformation but also motor information provided bythe input cue or/and the local recognition.Navigation problems are a good example forillustrating the problems to manage goalachievement and switching in a completelyautonomous system controled by a single neuralnetwork without any programmed trick to allow thegood choice at the right time. The solution wepropose can be understood on fig. 3.

Competition

Panoramic perception(hippocampus region)

Place recognition

Hunger

Pain

Movement

Competition

tiredness

Figure 3: Intuitive structure of a N.N. that allows theswitch between different motivations.

The neural network is composed of a recognitionpart with a Winner Takes All (WTA) competitionmechanism that allow to recognize a situation and toinforce the activation of a particular movementaccording to what has been learnt. The differentmotivation nuclei allow to favour a particular subgroup of recognition neurons. In the next part, wewill describe how a such basic network can explain

navigation behaviors. Next, we will return to theproblem of behavior switching, and therefore wewill study how to link properly the differentmotivation nuclei with the simple PerAc N.N.structure.

3 Target retrieval using landmarks

At the beginning, we suppose Prometheus movesrandomly. When it finds "food", it moves around itand learns that from several particular locations itcan go to the target by performing a movement in agiven direction.

� � �� � � � � �

� � �

� �� �

� �� �

Landmark

Landmark

Voronoï frontier

Landmark Landmark

Landmark

Landmark

G OAL

Robot init ial posit ion

N1

N2

N3

N4

N5

� �� �� �

� � � �� � � �� � � �� � � �� � � �� � � �

Figure 4: Local exploration around a target. The target is represented by a circle at the intersection ofthe dotted line representing the associated recognitiondomain of Ni neurons. The robot records at certain points(represented by small circles Ni) their relative position tothe landmarks and the direction to the target.

Later, when Prometheus wants to find "food", itconsiders the information of the place cellsassociated to the food and goes in the directionassociated to the most activated place fields(competitive mechanism). Thus at each time, thedistance to the target is reduced (fig. 4) andPrometheus is bounded to return to the learnedposition of the food. The complete "brain" isdepicted on fig.5. The local visual recognition (LR)and the information about the eye movements (EM)can be joined to provide information about “ what ”the landmarks are and “ where ” they are from eachother. Simple product or logical AND neurons canbe used to merge those different information type ina map of neurons that reacts only if a particularlandmark is recognized at a particular place: GVIgroup (Global Visual Input) in fig. 5. Matchingbetween a proposed visual scene and a learned sceneis performed with a topology preserving map.

GVI SR

RMP RM RM' Robot Movement

(-1) Mental rotationCommand

FP

Goal achievement

LR

EM

PerAc Block

Motor Flow

Sensorial Flow

Pleasure

Figure 5: The navigation neural network.

When a movement direction is selected (RM': RobotMovement), the robot makes one step of givenlength in that direction. The input to this networkare the north direction, and the food and landmarkspositions in the robot's visual space. We assume thata compass is available. It could be replaced by avestibular system or a gyroscopic mechanism thatwould produce low precision information about thebody orientation (a local landmark could also beused but it would reduce the generalizationcapabilities of the robot to very distant situations).This system allows target retrieval when the placecells have been learned. We have proposed aneurally-coded reflex mechanism [Gau94b] thatallows to visit several evenly disposed places aroundthe target, which includes a pleasure-linkedregulation for learning control.

Figure 6: Different trajectories. The Place-cells (PC) are indexed by their order duringexploration. The Voronoi tessellation is represented by thethick lines, the landmarks by the rectangles and the targetby the inner circle. The large circle represents the limitbeyond which the target is not perceived. Thin linesrepresent trajectories from various starting-points.

We have succesfully implemented and tested theneural network described above on a Khepera robot.Due to the tremendous computing time required, wesimulate the visual part that has been testedelsewhere [Gau92]. The robot succeeds in learningthe food position, and later, it always takes the rightdirection, whatever its starting point (fig. 6). More

realistic trajectories can be obtained if the movementis performed according to a probabilistic vote ratherthan a determinist WTA mechanism.

4 Avoidance of forbidden areas

The previous mechanism allows Prometheus to goback to a learned position in a somewhat straightline but it does not take into account any zone therobot must not go through. Moreover, forbiddenareas not necessarily have an intrinsic reality for therobot. Hence, we need to introduce ad hocmechanisms that allow to generate for instance apain signal when the robot enters a forbidden area.Such a zone is then perceived as dangerous and therobot uses the same mechanism as for obstacleavoidance to learn the direction to avoid pain. As aresult, learning a forbidden zone is equivalent tolearning an interesting place. The robot learnsdifferent meaningful places where the pain signal ishigh and also the association to the propermovement in order to avoid pain (see fig. 7).

Goal

Danger

L1

L2

L3

L4

L5

Figure 7: An example of situation with a dangerousarea that must be avoided to reach the goal.

With such a system, a problem arises when the robotis closer to the dangerous area than to the food area.Indeed, the neurons associated to the recognition ofthe dangerous area have a higher activity than thoseassociated to the goal. Then, the most activated ofthem must win and the robot will perform amovement to avoid the forbidden zone, thus movingin a direction opposite to its goal, which is thusnever reached. That problem can be solved if theinfluence of the neurons associated to "non-goal"actions can be inhibited or switched off when therobot is far enough from the dangerous area. This

means that when the recognition of a dangerousposition is not high enough, recognition ofdangerous areas has no effect on the robot behavior.But how to switch of the avoidance behavior?Moreover, according to the simple retrieval systemwhen the robot has found its target once, it willremain forever in its proximity. So the generalquestion is how the robot itself can be able to switchoff its unexpectated behavior.A first solution to inhibit a particular recognitioncould have consisted in tuning the neurons'selectivity but it is impossible because this parameteris already used to automatically control theunsupervised learning of the neurons [Gau94a].Moreover, it seems difficult to change only theselectivity of one particular neuron that interests usand not of the others.Actually, the problem is that Prometheus shouldrecognize a situation that is not the best fit but thatagrees its goal. The solution, we propose is inspiredby Grossberg's studies about contour closing inpreattentive vision [Gro87] and by Burnod's modelof the cortical column and cortical map [Bur89].Grossberg proposes a structure composed of twolayers of neurons with feed-forward and feed-backlinks. The former is a competitive network withneurons associated to the recognition of a particularsituation (i.e: a piece of straigth line) whereas thesecond level tries to propose positions for contourcompletion by reacting on the first neuron layer. Ifthe total sum of the input activities is high enough,the neuron on the first layer is activated and a pieceof edge is "recognized" at that position even if theinitial recognition was not high enough. This N.N.can so be seen as a way to achieve a preattentivegoal of contour completion. On a other level,Burnod's formalization of the cortical columndistinguishes a thalamic level associated to actionchoices and a cortical level in which rewarded goalsand subgoals can propagate freely according to mostfrequent transititions between different situations, asin a relaxation or a Markov process.

Goal Level

Competition

Recognition

Choice

Max

Σ

And

Figure 8: representation of a cortical column

To sum up, our neural building block must havethree layers (fig. 8). A first level must be associatedto the recognition of the current situation accordingto their synaptic weights. A second level consists ina competition mechanism that must choose therecognized situation according to the first levelinformation and to those coming from goal. And thethird one can be used to propagate the goals andsubgoals. Fig. 9 represents such an architecture,which allows to avoid one forbidden area. Inaddition to the cortical column structure, the limbicsystem is represented through different neural andchemical nuclei. Those nuclei are linked to thecortical maps in the same way as action units areassociated to recognition units in the simpler N.N.When pain is active, the links betwen the neuronsrepresenting dangerous situations and the nucleusassociated to pain are increased, thus allowing therobot to feel pain earlier when it sees once more agiven dangerous situation. We also suppose thatwhen the pain nucleus is inactive, it induces anegative activity to its associated neurons on thegoal level of the recognition cortical map. The samething goes for the hunger nucleus.

Recognition

Situation Input

Hunger

Food in sight

Animat hungry

Pain

Danger

signal not enough highto trigger on the painsituation recogition

No pain signal

Choice according to the goal and to the situation

Goal Level

links with otherareas

Figure 9: How Prometheus inhibits an avoidancebehavior when it is hungry enough.

Next, we have to suppose that the different nucleitake their inputs from the analog value of therecognition and not from the competition. Indeed,they need to have an analog value that reallyrepresents the proximity to the learned situation andthat can allow to trigger the avoidance behavior ifthe situation is too similar to the painful situationlearned. For instance, on fig. 7, we suppose the robotis closer to the forbidden zone than to the food.Thus, the most activated neuron on the recognitionlevel is one associated to the forbidden area. But itsactivity may not be high enough to switch on thepain nucleus, whereas the hunger nucleus isactivated because the robot "needs" food. Then, the

goal level associated to the recognition of the foodplaces can allow them to win and to induce themovement in the good direction, ie: to go to thefood. If both nuclei are inactive then the associatedrecognition neurons cannot be activated and noinformation is provided to the motor group. Hence,the robot moves randomly.

Figure 10: Simulation of a forbidden area avoidanceand goal reaching.

Fig. 10 represents a simulation in which the robot ishungry. It goes in the direction to the food since it istoo near to the forbidden area. Then, the avoidanceof pain is activated and the robot goes out to theinfluence domain of pain nucleus and it tries againto reach the food. The probabilist WTA mechanismprevents it from being blocked and allows it to movearound the forbidden area.

5 Choosing between different goals

We have shown that a limbic system is needed tocontrol the switching of behaviors between a goaland a prescriptive constraint such as the avoidanceof a dangerous area. Now, we are interested in howto switch from one behavior to another. Forinstance, after having found food, the robot maywant to go home. According to the previousdiagrams, the robot should stay forever near the foodbecause the recognition of the place associated to thefood is very high. Having already eaten shouldreduce the intensity of the will to move towardsfood. The will to see home should favour therecognition of the associated places to allow thegood movements.

The mechanism presented in the previous part canbe extended to choose between different simple goalsthat must be associated to different internalmotivations. According to the activity of eachmotivation nucleus, the competition mechanism onthe recognition cortical map should allow the robotto choose the most interesting behavior that can beperformed according to the situation and to itsmotivations. The neuron activation rule used is thedifference betwen the maximum of the excitatoryand inhibitory input links. This activity (Si) is put to0 when it is negative. Hence, a constant input allowsthe unreached neurons to win in front of inhibitedneurons.S I Max positive activation Min negative activation

if x

i = + −

= =

+

+ +

_ _b g b g x > 0 x else x 0

The Max operator is a way to make the neuronalactivity independant of the number of input linksand to give rise to stability of the neuronal activitythrough the cumulative effect of feedback positivelinks. If the weight is lower than 1 there is nodivergence problem.The goal level can also be used to add capabilities ofplan generation. When a goal is proposed, if itcannot be satisfied possible subgoals are proposeduntil a subgoal is really satisfied. Then the actionscan be performed according to their goal pathway.The mechanism can be extended to topological maps[Gau94a] if the inhibition is applied to the wholeactivity bubble in order to inhibit all the recognitionneurons associated to the same movement.Motor groups use the same law but thesimplification comes from the absence of neededrecognition. In fact it can be added if actions canonly be performed in particular situations or if thechoice of the action depends on the position of themotor system; for instance, a mechanical arm withseveral freedom degrees and forbidden angularzones...

6 Conclusion

We have shown that a simple neural system can beused to control robot navigation and goalmanagement.Research on these mechanisms should lead to definean explicit parallel langage to “ program ” animal-like robots with adaptation and autonomycapabilities.The neural vision sytem associated to the robot hasnot been used in the experiments presented above forreasons of computation time. We are now working

on a multi processor architecture that may allow todispatch simply the explicit parallel program thatthe N.N. represents. Other work focuses on learningwith delays between the conditional stimulus and theunconditional one or the reinforcement signal. Atanother level algorithms able to learn a succession ofperception action according to a latter goal havebeen successfully tested. Future work will consist inassembling all these part to (really) realize a(animal) robot able to navigate with a real autonomyin an outdoor environment.

7 References

[Bro86] Brooks R.A., A robust layered control systemfor a mobile robot. IEEE Journal of Robotics andAutomation RA-2, p 14-23, 1986.[Bur89] Burnod Y. , An adaptive neural network :The cerebral cortex, Masson, Paris, 1989.[Gau92] Gaussier P. , Cocquerez, J.-P., NeuralNetworks For Complex Scene Recognition:Simulation Of A Visual System With SeveralCortical Areas, in Proceedings of IJCNN, Baltimore,Vol. 3, p 233-259, 1992.[Gau94a] Gaussier P. & Zrehen S., A TopologicalNeural Map for On-line Learning: Emergence ofObstacle Avoidance in a Mobile Robot, SAB 94,Brighton, not yet known, 1994[Gau94b] Gaussier P. Zrehen S., A ConstructivistApproach for Autonomous Agents, Wiley & Sons, EdN. Thalman, 1994[Gro87] Grossberg S., Mingolla E., Neural Dynamicsof Surface Perception: Boundary Webs,Illuminants,and Shape-from-Shading, CVGIP No 37, p 116-165,1987.[Mat87] Maturana & Varela, The tree of knowledge,Shambhala ed., 1987.[Mon93] Mondada F., Franzi, Ed., Ienne P., MobileRobot Miniaturisation: A Tool For Investigation InControl Algorithms, Proceedings of the ThirdInternational Symposium on Experimental Robotics,Kyoto, Oct. 28-30, 1993.[Ste91] Stewart, J., Life = Cognition: Theepistemological and Ontological significance ofArtificial Life, proccedings of SAB 91 Paris,Bourgine P. & Varela F. eds, MIT Press, pp 475-483,1991.[Zip85] Zipser D., A Computational Model ofHippocampal Place Fields, Behavioral Neuroscience99, 5, p 1006-1018, 1985.