october 26, optimization
DESCRIPTION
Multi-Robot SystemsTRANSCRIPT
Multi-Robot Systems
CSCI 7000-006Monday, October 26, 2009
Nikolaus Correll
So far
• Probabilistic models for multi-robot systems– Extract probabilistic behavior of sub-systems– Small state space: rate equations– Large state space: DES simulation
Today
• System optimization using probabilistic models– Find optimal control parameters– Explore new capabilities using models– Find optimal control and system parameters
Comparison of Coordination Schemes
• Metrics for comparison– Speed– System cost– System Size– System Reliability– Benefits to the User
System-design is a constraint optimization problemSolution: Appropriate Models
Speed
Cost
Size
Reliability
Benefits
Mapping
No Mapping
Too large
Too slow
Too expensive
Model-based design
Real System Model Controller Design
Size, Cost, … Speed, Reliability, … Control parameters
Model-based optimization
• Physical simulator– Simulate controllers and
robot designs• DES simulator– Simulate controllers and
available information• Optimize using– Systematic experiments– Learning/optimization
Communication
Navigation accuracy
Optimization using analytical models
• Probabilistic state machine is derived from the robot controller
• One difference equation per state
111
)(1
)()1(
)(1
0
kNkNNkN
kNT
kNpkNkN
TkNpkNpkNkN
rcs
cc
sccc
rsrsrrr
Search Collision1/Tc
pc
Ns Nc
Restpr
Tr
Optimization using analytical models
• Probabilistic state machine is derived from the robot controller
• One difference equation per state
111
)(1
)()1(
)(1
0
kNkNNkN
kNT
kNpkNkN
TkNpkNpkNkN
rcs
cc
sccc
rsrsrrr
Search Collision1/Tc
pc
Ns Nc
Restpr
Tr
System parameters
Optimization using analytical models
• Probabilistic state machine is derived from the robot controller
• One difference equation per state
111
)(1
)()1(
)(1
0
kNkNNkN
kNT
kNpkNkN
TkNpkNpkNkN
rcs
cc
sccc
rsrsrrr
Search Collision1/Tc
pc
Ns Nc
Restpr
Tr
Control parameters
Optimal Control: Brief Intro
• Find optimal control inputs for a dynamical system to optimize a metric of interest
• Example: Tank reactor, maximize quantity B by tuning inflow and outflow
• Known: system dynamics
Ainflow
outflow
A->B->C
A->C
Static Optimization
• Find optimal control inputs (constant)
• Example: inflow 50l/min, outflow 10l/min
• Constraint: Volume of the tank at final time
inflow
A
outflow
flow volume
time
Dynamic Optimization• Find optimal control
input profiles (time-varying)
• Example: max inflow for 10s, outflow off, after 50s and outflow max
• Constraint: Volume of the tank during the whole process
flow volume
time
inflow
A
outflow
Optimal Control
• Capture terminal and stage cost as well as constraints using a single cost function
• The optimization problem is then solved by minimizing this cost function
Example: Coverage
• Collaboration policy:– Robots wait at tip for Ts – Waiting robots inform other
robots to abandon coverage • Trade-Off between additional
exploration versus decreased redundancy
• Communication introduces coupling among the robots (non-linear dynamics)
N. Correll and A. Martinoli. Modeling and Analysis of Beaconless and Beacon-Based Policies for a Swarm-Intelligent Inspection System. In IEEE International Conference on Robotics and Automation (ICRA), pages 2488-2493, Barcelona, Spain, 2005.
Optimal Control Problem
• A static beacon policy does not reduce completion time but only energy consumption
• Is there a dynamic policy which improves coverage performance?
• Find the optimal profile for the parameter Ts minimizing time to completion
Model
N. Correll and A. Martinoli. Towards Optimal Control of Self-Organized Robotic Inspection Systems. In 8th International IFAC Symposium on Robot Control (SYROCO), Bologna, Italy, 2006.
Optimization Problem
• Terminal cost: time to completion• Stage cost: energy consumption• Constraints: number of virgin blades zero
u=
Possible optimization method: Extremum Seeking Control
• Necessary condition of optimality:
• Optimization as a feedback control problem:
• Gradient Estimate by sinusoidal perturbation:
Optimal Marker PolicyStationary Marker
Optimal policy“Turn marker onafter around 180s,mark for 5s and goOn.”
Methodfmincon using the macroscopic model and optimal parameters based on base-line experiment.
Results/Discussion
• Optimal results when beacon behavior is turned on toward the end of the experiment
• Intuition: Exploration more important in the beginning
• An optimal beacon policy only exists if there are more robots than blades
Randomized Coverage with Mobile Marker-based Collaboration
Search Inspect MobileMarker
Avoid Obstacle
Wall | Robot Obstacle clear
Blade pt
1-pt | Marker
Tt expired
Translate Inspect Inspect
g=0 no collaborationg=1 full collaboration
No Collaboration vs. Mobile Markers
20 Real Robots Agent-based simulation
No Markers
Mobile Markers
Model-based design: Pitfalls
• Model-based controller design depends on – accurate parameters– Ideal model
• Optimization problem(s)– Find optimal control parameters– Find optimal model parameters
e
m
l
e
m
l
Estimate both model and control parameters simultaneously
M
M
Model
Model
“Optimal control under uncertainty of measurements”
Simultaneous optimization of model and control parameters
• How to select pi when Ti are unknown?• Optimization algorithm– Initial guess for model and control parameters– Run the system and collect data– Find optimal fit for model parameters– Find optimal control parameters– Repeat until error between experiment and model
vanishesN. Correll. Parameter Estimation and Optimal Control of Swarm-Robotic Systems: A Case Study in Distributed Task Allocation. In IEEE International Conference on Robotics and Automation, pages 3302-3307, Pasadena, USA, 2008.
Optimal Control of System and Control Parameters
Control Parameters
System Parameters
All experimentsNext experiment
Case Study: Task Allocation
• Finite number of tasks• Robots select task i with
probability1. pi =const.
(Independent robots)2. pi (k) function of Nj (k)
• Task i takes time Ti in average
K. Lerman, C. Jones, A. Galstyan, and M. Matarić, “Analysis ofdynamic task allocation in multi-robot systems,” Int. J. of RoboticsResearch, 2006.
1. Independent Robots
• Model(Number of robots in state i)
• Parameters– probability to do a task– System parameters
• Analytical optimization–
n ,,1
2. Threshold-based Task Allocation
• Probability to do a task– Stimulus– Threshold
• Stimulus: Number of robots doing the task already
• Model (non-linear)
• Optimization: numerical
Experiment
• Step 1: Estimate model parameters– Ti are unknown– Take random control parameters– Measure steady state– Find Ti given known control parameters
• Step 2: Find optimal control parameters
System dynamicsIn
depe
nden
t Rob
ots
(Lin
ear M
odel
)Th
resh
old-
base
d (N
on-L
inea
r Mod
el)
Difference Equations DES Simulation
100 robots 1000 robots
Results
Linear Model Non-Linear Model
25 robots 25 robots 100 robots
Summary
• System models can be used for finding optimal control policies and parameters
• Models can be physical simulation, DES, or analytical
• More abstract model allow for more efficient search, even analytical
• System parameters can be optimized simultaneously with system in the loop
Upcoming
• Multi-Robot Navigation (M. Otte)• Learning and adaptation in swarm systems• 3 weeks lectures, 1 week fall break• November 29: reports due• 2 weeks project presentations, random order