1

Pursuit-Evasion (Robotic) GamesThe Marco Polo Case

Army Research Lab

Rafael Fierro

MARHES LaboratorySchool of Electrical and Computer

EngineeringOklahoma State University

Stillwater OK 74078

Systems Workshop on Adaptive & Networks

SWAN 06

December 8, 2006

2

Outline• Motivation and interests

• Vision-based formation control

• Dynamic boundary tracking

• Robotic Games The Marco Polo Case

• Hierarchical Optimization Shape changes in robot formations

• Conclusions

3

Boeing/Insitu ScanEagles® in a cooperative target tracking mission

Surveillance

Reconnaissance Coordination

Goals:

• Use multiple low-cost vehicles to cooperatively track evading targets in urban environments.

• Deployment of Ad Hoc wireless sensor networks WSN with nodes supporting limited mobility.

• Extending the mission life of Ad Hoc wireless sensor nets.

• To develop a mathematically sound framework for the analysis and design of feedback control of UV teams in the presence of intermittent, asynchronous communication and estimation.

High-confidence Distributed Vehicles Networks for Urban Navigation

4

Target Acquisition, Deployment, and Surveillance

• Developing algorithms to deal with the heterogeneity in the system (information and vehicles) is a key challenge in multi-vehicle coordination.

• To the best of our knowledge, this is a scientific gap that still requires closing.

5

Vision-based Control of FormationsO.A.A. Orqueda and R. Fierro, “Robust vision-based nonlinear formation control,” Proceedings of the American Control Conference, Minneapolis, MN, June 2006, pp. 1422-1427.

XI

YI

Robot j

ij

ij

ij

i

p

j

Robot i

cZ

d

cX

m

ij

• Algorithm is based on a high-gain observer (HGO) and second order sliding mode,

• Robust to communication failures,

• Closed-loop system is Lyapunov stable.

6

Marker and ID Detection

Image capture Grayscale conversion

ThresholdingContour

extraction

Contour selection

ID recognition

7

Noise Effect - Fog

Distances from 0.6 m to 3.065 m

8

Information flow/sensing levels

• Partial state feedback controller (FSFB)• Robust partial state feedback controller (RSFB)• Output feedback controller (OFB)

9

Control for Sensing: Dynamic Boundary Tracking

Environment

Dynamic BoundaryAbstraction and

Estimation

Remote SensingSatellite or UAVs

0

Dynamic Sensor Net

4

1

2

N

35

( , )i i i ix f x u

{, }( )ii e if

( )

( )

x t

y t

http://www.bairdsoftware.com/

Y. Cao and R. Fierro, CDC 2006.

10

Pursuit-Evasion (Robotic) Games

• Multi-objective optimization for detecting and intercepting evading targets.

• References

S. Ferrari, C. Cai, R. Fierro, and B. Perteet, “A multi-objective optimization approach to detecting and intercepting targets in pursuit-evasion games,” American Control Conference, September 2006. (Submitted)

B. Perteet, J. McClintock, and R. Fierro, “A Multi-Vehicle Framework for the Development of Robotic Games: The Marco Polo Case,” IEEE Int. Conf. on Robotics and Automation, September, 2006. (Submitted)

S. Ferrari, “Track coverage in sensor networks,” in Proceedings of the American Control Conference, Minneapolis, MN, June 14-16 2006, pp. 2053–2059.

V. Isler, S. Kannan, and S. Khanna, “Randomized pursuit-evasion in a polygonal environment,” IEEE Trans. on Robotics, vol. 21, no. 5, pp. 875–884, 2005.

11

The Marco Polo Case

Game Rules:

One player is labeled Marco. The others are called Polo.

The player labeled Marco mustkeep their eyes shut while chasing the other players.

If Marco announces “Marco”, allother players immediately announce “Polo”. This audio cueallows Marco to track the otherplayers.

When a player is tagged by Marco they become the new Marco.

Marco Polo was a famous Croatian (Italian) trader and explorer who lived during the 13th century.

12

Why is this problem interesting?

• It combines vSLAM Motion coordination (i.e., control algorithms and

communication protocols) within uncertain and dynamic environments,

• Intermittent communication and estimation,• Occasional cooperation

Targets must (asynchronously) broadcast their positions, but limited sensing,

• Human-robot interaction• Learning

• Robust Intelligence -- new NSF cluster!!

13

Mobile sensor agents and targets

• Mobile sensor agents (autonomous)

• Target model (may be tele-operated by a child)

cos

sin

i i ip p p

i i ip p p

i ip p

x v

y v

where v is uniformly

distributed in [0, V] and is uniformly distributed in [0, 2).

cos

sin

j j j

j j j

j j

x v

y v

14

More formally…

• Problem

Given a set of N pursuers and a set of M target agents within a specified game area , find a set of policies for all pursuers in which maximizes the of probability of intercepting partially-observed and unobserved tracks and minimizes the time tc required to capture fully-observed targets in

.

,i i tp iu c p Z U

tZ denotes the set of detections of target positions

i.

15

Example of workspace

Partially-observed track, R i

A

D

Y

r

L2

L1

r

CO j

CR i

X

16

Track Coverage

• S. Ferrari, “Track coverage in sensor networks,” in Proceedings of the American Control Conference, Minneapolis, MN, June 14-16 2006, pp. 2053–2059.

x

2-Coverage cones

y

ξ2

y’

x’

L1 L2 RectangularAera of Interest (S)

ξ1

17

Optimal Sensor Placement

Before optimization After optimization (k = 3)

Goal: Place sensors such that we maximize the probability of detecting new targets entering the environment with at least k different sensors

18

Control Strategies

Hierarchical Hybrid System (Pursuer)

19

Pursuit Strategy

20

Track Coverage and Detection

• The map of the environment is known a priori and is cell decomposed into regions free of and occupied by obstacles.

• Two classes of sensors: static and mobile.• Sensors are optimally placed in the environment

offline such that the probability of detecting targets with k = 3 different sensors is maximized (using track coverage algorithm).

• Mobile sensors are deployed after at least two detections.

• Mobile sensors deploy to gain additional information about the target in order to capture it.

• A* algorithm used to find optimal path with a reward function.

21

Track Coverage and Detection

• Sensor paths are planned by choosing a sequence of adjacent cells, or channel, in the environment such that obstacles are avoided while moving toward hypothesized target tracks maximizing the reward:

where

The reward function is

is the Euclidean distance,

is the change in sensor coverage,

is the probability of detecting the target at the given cell (assumed to be 1 or 0), and

are weighting parameters.

22

Preliminary Simulation

Initial location of mobile sensor

Optimal path

Detection points

Static sensors

Target

Mobile sensor

Hypothesized trackObstacles

23

Preliminary Simulation

24

Marco Polo: Experimental Results

25

Hierarchical Optimization for Deployment of MAS

• We integrate model predictive control (MPC) and mixed integer linear programming (MILP) into a hierarchical framework.

J(X)X,C

JH(XH)XH,CH

J1L(X1

L)X1

L,C1L

J2L(X2

L)X2

L,C2L

JNL(XN

L)XN

L,CNL

|X| = O((Nr2+NrNo)T)

|XH| = O(Nr2)

LP problem

Easy to solve!!!!

|XiL| = O((Nr+No)T) each problem can be

solved in parallel from each agent

26

Case Study 1: Surveillance

27

Case Study 2: Shape Changes in Robot Formations

• The Problem: Consider a formation of robots currently in shape , with an objective of transitioning to shape . Minimizing the combined total distance that the

robots must travel. Minimizing the maximum distance that any robot

must travel.

Spletzer & Fierro, ICRA 2005.

28

Shape changes in robot formations (cont…)

• Definition: The shape of a formation is the geometrical information that remains when location, scale, and rotational effects are removed. I. L. Dryden and K. V. Mardia. Statistical Shape Analysis. John Wiley and Sons, 1998.

• Given a shape icon defined by a km matrix

where k is the number of robots, and m the dimension of the Euclidean Space, the shape [] is defined by

• Equivalent shapes examples:

TkssS ],,[ 1

[ ] { 1 : , ( ), ]T mkS SR d R SO m d

29

Shape changes in robot formations

Objective Shape

Initial Shape

Spletzer & Fierro, ICRA 2005Second-order cone programming

Branca & Fierro, ACC 2006Hierarchical optimization with collision avoidance

30

Conclusions

• The difficulty in reliably operating robotic networks lies in the need to carefully integrate communication, sensing, processing, power management, and motion control.

• Goals of this research effort: The development of formal models and analytical tools to characterize

the properties of coordination algorithms for teams of mobile robots and networked embedded systems,

the development of software tools to assess the complexity and support the synthesis of high-confidence and efficient cooperative strategies of coordination algorithms,

Experiments with more robots in more realistic outdoor scenarios, Evaluation of the performance of the team based on different

communication schemes.

• Stability of networked hybrid systems.

31

Questions

Thanks for your time!!http://marhes.okstate.edu

?