value-of-information aware active task assignment aware active task assignment ... Ì value of evs:...
TRANSCRIPT
Value-of-Information Aware Active Task Assignment
Beipeng Mu, Girish Chowdhary and Jonathan P. How
Laboratory for Information and Decision SystemsDepartment of Aeronautics and Astronautics
Massachusetts Institute of Technology
September 9th, 2013
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 1 / 14
Introduction
Motivation
▶Problem: Robustify mission planning to uncertainty caused byclassification errors in situational awareness∎ Uncertainty in target classification
(not just location)∎ Uncertainty impacts mission∎ Heterogeneous UV – explore & exploit∎ Velocity, fuel, time UV constraints
▶Contributions:∎ Evaluate information gathering by
improvement in mission score (VoI)∎ Develop algorithms that couple
exploration activities into taskingactivities
∎ CBBA to incorporate UV constraints∎ Test algorithm in hardware
Target Uncertainty
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 2 / 14
Introduction
Motivation
▶Problem: Robustify mission planning to uncertainty caused byclassification errors in situational awareness∎ Uncertainty in target classification
(not just location)∎ Uncertainty impacts mission∎ Heterogeneous UV – explore & exploit∎ Velocity, fuel, time UV constraints
▶Contributions:∎ Evaluate information gathering by
improvement in mission score (VoI)∎ Develop algorithms that couple
exploration activities into taskingactivities
∎ CBBA to incorporate UV constraints∎ Test algorithm in hardware
Target Uncertainty
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 2 / 14
Introduction
Motivation
▶Problem: Robustify mission planning to uncertainty caused byclassification errors in situational awareness∎ Uncertainty in target classification
(not just location)∎ Uncertainty impacts mission∎ Heterogeneous UV – explore & exploit∎ Velocity, fuel, time UV constraints
▶Contributions:∎ Evaluate information gathering by
improvement in mission score (VoI)∎ Develop algorithms that couple
exploration activities into taskingactivities
∎ CBBA to incorporate UV constraints∎ Test algorithm in hardware
Target Uncertainty
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 2 / 14
Introduction
Motivation
▶Problem: Robustify mission planning to uncertainty caused byclassification errors in situational awareness∎ Uncertainty in target classification
(not just location)∎ Uncertainty impacts mission∎ Heterogeneous UV – explore & exploit∎ Velocity, fuel, time UV constraints
▶Contributions:∎ Evaluate information gathering by
improvement in mission score (VoI)∎ Develop algorithms that couple
exploration activities into taskingactivities
∎ CBBA to incorporate UV constraints∎ Test algorithm in hardware
Target Uncertainty
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 2 / 14
Introduction
Motivation
▶Problem: Robustify mission planning to uncertainty caused byclassification errors in situational awareness∎ Uncertainty in target classification
(not just location)∎ Uncertainty impacts mission∎ Heterogeneous UV – explore & exploit∎ Velocity, fuel, time UV constraints
▶Contributions:∎ Evaluate information gathering by
improvement in mission score (VoI)∎ Develop algorithms that couple
exploration activities into taskingactivities
∎ CBBA to incorporate UV constraints∎ Test algorithm in hardware
Target Uncertainty
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 2 / 14
Introduction
Robust UV Planning
▶Assign tasking vehicles (TV): mathematical programming [1, 2]
maxN
∑i=1
f(Ci)xi s.t.N
∑i=1
xi ≤ n
∎ Ci: random variable indicating target i’s possible return at time k∎ xi ∈ {0,1}: whether a UV is assigned to do tasks at target i∎ f(C): score function provides quality measure of score distribution C
▶ If score deterministic & known, can use integer programming
▶With score uncertainty, can use various metrics:
∎ Expected score [3] f(C) = E(C)∎ Worst case score [4] f(C) =minC∎ Expected with offset [2] f(C) = µ(C) − σ(C)∎ Chance constraint [5–7]f(C) = c s.t. Prob(C < c) ≤ ε
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 3 / 14
Introduction
Robust UV Planning
▶Assign tasking vehicles (TV): mathematical programming [1, 2]
maxN
∑i=1
f(Ci)xi s.t.N
∑i=1
xi ≤ n
∎ Ci: random variable indicating target i’s possible return at time k∎ xi ∈ {0,1}: whether a UV is assigned to do tasks at target i∎ f(C): score function provides quality measure of score distribution C
▶ If score deterministic & known, can use integer programming
▶With score uncertainty, can use various metrics:
∎ Expected score [3] f(C) = E(C)∎ Worst case score [4] f(C) =minC∎ Expected with offset [2] f(C) = µ(C) − σ(C)∎ Chance constraint [5–7]f(C) = c s.t. Prob(C < c) ≤ ε
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 3 / 14
Introduction
Robust UV Planning
▶Assign tasking vehicles (TV): mathematical programming [1, 2]
maxN
∑i=1
f(Ci)xi s.t.N
∑i=1
xi ≤ n
∎ Ci: random variable indicating target i’s possible return at time k∎ xi ∈ {0,1}: whether a UV is assigned to do tasks at target i∎ f(C): score function provides quality measure of score distribution C
▶ If score deterministic & known, can use integer programming
▶With score uncertainty, can use various metrics:
∎ Expected score [3] f(C) = E(C)∎ Worst case score [4] f(C) =minC∎ Expected with offset [2] f(C) = µ(C) − σ(C)∎ Chance constraint [5–7]f(C) = c s.t. Prob(C < c) ≤ ε
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 3 / 14
Introduction
Uncertainty Reduction by Exploration
▶Given the importance of score uncertainty to the mission, can use UVto take observations ⇒ uncertainty reduced, performance improved∎ Standard sensor management procedure [8–12]∎ Exploration/exploitation here focused on classification rather than just
localization∎ Extends prior work to non-Gaussian uncertainty distributions
▶ Exploration Vehicles (EV) take observations
▶ EV assignment: maximize uncertaintyreduction based on expected posterior∎ Uncertainty measure: g(Ci)⇒ g(Ci∣zi)∎ zi unknown Ezi[g(Ci∣zi)] = ∫ g(Ci∣zi)dp(zi)∎ Could assign EV based on uncertainty reduction
maxN
∑i=1
(g(Ci) − g(Ci∣zi)) yi s.t.N
∑i=1
yi ≤m
∎ yi ∈ {0,1} denotes whether EV assigned to target i
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 4 / 14
Introduction
Uncertainty Reduction by Exploration
▶Given the importance of score uncertainty to the mission, can use UVto take observations ⇒ uncertainty reduced, performance improved∎ Standard sensor management procedure [8–12]∎ Exploration/exploitation here focused on classification rather than just
localization∎ Extends prior work to non-Gaussian uncertainty distributions
▶ Exploration Vehicles (EV) take observations
▶ EV assignment: maximize uncertaintyreduction based on expected posterior∎ Uncertainty measure: g(Ci)⇒ g(Ci∣zi)∎ zi unknown Ezi[g(Ci∣zi)] = ∫ g(Ci∣zi)dp(zi)∎ Could assign EV based on uncertainty reduction
maxN
∑i=1
(g(Ci) − g(Ci∣zi)) yi s.t.N
∑i=1
yi ≤m
∎ yi ∈ {0,1} denotes whether EV assigned to target i
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 4 / 14
Value of Information based Assignment
Assignment of Heterogeneous Vehicles
▶ Straightforward approach:∎ Assign Tasking Vehicle (TV) to maximize score∎ Assign Exploration Vehicle (EV) to maximize uncertainty reduction
Assignments of Tasking/Exploration Vehicles in a decoupled way
TVs and EVs areassigned to differenttargets
Exploration by EVdoes not necessarilyhelp TV obtain higherscore
▶Decoupled Assignment – Information gathered possibly has novalue in terms of mission return [2]
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 5 / 14
Value of Information based Assignment
Value of Information based Exploration
▶Real question: What is the value of information gathered?∎ Information gathered will change score distribution of a task
Ezi [f(Ci∣zi)] = ∫ f(Ci∣zi)dp(zi)
▶How to tie information gathering to missions (TV activities)?∎ Exploration adds value only when a TV is also assigned to the target
▶Value of Information based Task Assignment
maxN
∑i=1
Ezif(Ci∣zi)xiyi + f(Ci)xi(1 − yi)
s.t. xi, yi ∈ {0,1}, ∑xi ≤ n, ∑ yi ≤m∎ Recover value of information gathering/exploration activities∎ Note complexity with products of xiyi
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 6 / 14
Value of Information based Assignment
Value of Information based Exploration
▶Real question: What is the value of information gathered?∎ Information gathered will change score distribution of a task
Ezi [f(Ci∣zi)] = ∫ f(Ci∣zi)dp(zi)
▶How to tie information gathering to missions (TV activities)?∎ Exploration adds value only when a TV is also assigned to the target
▶Value of Information based Task Assignment
maxN
∑i=1
Ezif(Ci∣zi)xiyi + f(Ci)xi(1 − yi)
s.t. xi, yi ∈ {0,1}, ∑xi ≤ n, ∑ yi ≤m∎ Recover value of information gathering/exploration activities∎ Note complexity with products of xiyi
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 6 / 14
Value of Information based Assignment
VoI based Active Task Assignment
Assignments of Tasking/Exploration Vehicles in a coupled way
▶Assignments of EVs are coupled into assignments of TVs∎ EVs and TVs are paired up∎Value of EVs: help TVs to reduce uncertainty and thus get higher score
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 7 / 14
Value of Information based Assignment
Consensus-Based Bundle Algorithm (CBBA)
▶ System dynamics and constraints∎ Various features
e.g. Location, speed∎ Cooperation put extra constraints
EV finish exploration before TV starts task∎ Integer programming ⇒ combinational number of candidate solutions
▶ CBBA∎ Given agents and tasks features, give agent-task assignment pairs
▶ Feature∎ Incorporates system dynamics∎ Decentralized∎ Polynomial in number of targets and vehicles
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 8 / 14
Value of Information based Assignment
VoI based Coupled Planning using CBBA
Algorithm 1 VoI Based Coupled Distributed Planning
1: initialize candidate solution queue Q2: while UTV requirement not satisfied do3: P = nextBestCandidate(Q)4: Q = {Q, sub-candidates(P) }5: call CBBA to assign UTVs6: AT = CBBA(P, agents, tasks)7: call CBBA to assign UEVs8: AE = CBBA(AT , agents, tasks)9: check whether UTV requirement
10: checkMatch(AT ,AE)11: end while12: return AT , AE
▶ Iterates between assignments of EV and TV until the best feasiblesolution is found.
▶More details in posterJ.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 9 / 14
Numerical Result
Simulation Result
▶NEV = 3 and NTV = 5
▶Targets have uncertainties with categorical distribution
▶ Constraint failure chance: ∀i, f(Ci) = ci s.t. p(Ci < ci) ≤ ε
100 200 300 400 500 600 700 8000
5
10
15
20
25
30
35
40
45
Score
Fre
qu
ency
UTV onlyDecoupledCoupled
Histogram of score
▶ Expected score, decoupled > TV only
▶ Expected score, coupled > decoupled
40 50 60 70 80 90 100 1100
5
10
15
20
25
30
Running Time
Fre
qu
ency
UTV onlyDecoupledCoupled
Histogram of expected execution time
▶ No significant impact on execution time
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 10 / 14
Numerical Result
Simulation Result
▶During execution, when exploration gives new measurements, replanto obtain better assignments
▶Decoupled between coupledand TV only
▶ Less targets ⇒ higher chanceEVs pair up with TVs ⇒ score ofdecoupled ∼ coupled
▶More targets ⇒ less chance EVspair up with TVs ⇒ score ofdecoupled ∼ TV only
0 10 20 30 40 50 60 70 80 90 1000
100
200
300
400
500
600
700
800
900
1000
Number of Tasks
Sco
re
UTV onlyDecoupledCoupled
Compares statistics vs. # of targets
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 11 / 14
Numerical Result
Experiments – Detail in Poster
Robot system in the MIT Aerospace Control Laboratory [13]J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 12 / 14
Numerical Result
Conclusion and Future Work
▶Conclusion∎ Developed an approach to compute the value of information for
multiagent systems by explicitly coupling the exploration to improvementsin the mission performance
Coupled exploration into mission assignmentsImplemented VoI based coupled planning algorithm in a distributedframework using CBBATested proposed approaches in hardware
▶ Future Work∎ Sub-team size optimization∎ Dynamics in targets
e.g. moving targets where uncertainty of targets may change with time∎ Extend to more general sensing problems
e.g. information gathering in nonparametric models.
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 13 / 14
Numerical Result References
References I
[1] J. Bellingham, M. Tillerson, A. Richards, and J. How. Multi-task allocation and path planning for cooperating UAVs.In Proceedings of Conference of Cooperative Control and Optimization, Nov. 2001.
[2] Luca F. Bertuccelli. Robust Planning for Heterogeneous UAVs in Uncertain Environments. Master’s thesis,Massachusetts Institute of Technology, Department of Aeronautics and Astronautics, Cambridge MA, June 2004.
[3] J.R. Birge and F. Louveaux. Introduction to Stochastic Programming. Springer Series in Operations Research andFinancial Enginee. Springer, 2011.
[4] Pavlo Krokhmal, Robert Murphey, Panos Pardalos, Stanislav Uryasev, and Grigory Zrazhevski. Robust decisionmaking: Addressing uncertainties. In in Distributions”, In: S. Butenko et al. (Eds.) ”Cooperative Control: Models,Applications and Algorithms, pages 165–185. Kluwer Academic Publishers, 2003.
[5] A. Nemirovski and A. Shapiro. Convex approximations of chance constrained programs. SIAM Journal onOptimization, 17(4):969–996, 2007.
[6] E. Delage and S. Mannor. Percentile optimization for markov decision processes with parameter uncertainty.Operations research, 58(1):203–213, 2010.
[7] L. Blackmore and M. Ono. Convex chance constrained predictive control without sampling. AIAA Proceedings.[np].10-13 Aug, 2009.
[8] Loredana Arienzo. An information-theoretic approach for energy-efficient collaborative tracking in wireless sensornetworks. EURASIP J. Wirel. Commun. Netw., 2010:13:1–13:14, January 2010.
[9] Hanbiao Wang, Kung Yao, and Deborah Estrin. Information-theoretic approaches for sensor selection and placementin sensor networks for target localization and tracking. Communications and Networks, Journal of, 7(4):438 –449, dec.2005.
[10] Feng Zhao, Jaewon Shin, and J. Reich. Information-driven dynamic sensor collaboration. Signal Processing Magazine,IEEE, 19(2):61 –72, mar 2002.
[11] C. Kreucher, M. Morelande, K. Kastella, and A.O. Hero. Particle filtering for multitarget detection and tracking. InIEEE Aerospace Conference, pages 2101 –2116, march 2005.
[12] A. Logothetis, A. Isaksson, and R.J. Evans. An information theoretic approach to observer path design forbearings-only tracking. In Decision and Control, 1997., Proceedings of the 36th IEEE Conference on, volume 4, pages3132 –3137 vol.4, dec 1997.
[13] J. P. How, B. Bethke, A. Frank, D. Dale, and J. Vian. Real-time indoor autonomous vehicle test environment. IEEEControl Systems Magazine, 28(2):51–64, April 2008.
J.P. How (LIDS, MIT) VoI Task Assignment 09/09/13 14 / 14