long-run multi-robot planning under uncertain task durations · long-run multi-robot planning under...
TRANSCRIPT
Long-Run Multi-Robot Planning Under Uncertain TaskDurations
Doctoral Consortium
Carlos AzevedoInstitute For Systems and Robotics, Instituto Superior Técnico, University of Lisbon, Portugal
ABSTRACTThis paper presents part of the work developed so far within thescope of my PhD and suggests possible future research directions.My thesis tackles the problem of multi-robot coordination underuncertainty over the long-term. We present a preliminary approachthat tackles multi-robot monitoring problems under uncertain taskdurations. We propose a methodology that takes advantage of amodeling formalism for robot teams: generalized stochastic Petrinets with rewards (GSPNR). A GSPNR allows for unified modelingof action selection and uncertainty on duration of action execution.At the same time, it allows for goal specification through the useof transition rewards and rewards per time unit. The proposedapproach exploits the well-defined semantics provided by Markovreward automata in order to synthesize policies.
KEYWORDSMulti-robot systems, Planning under uncertainty, Long-run averageoptimization
ACM Reference Format:Carlos Azevedo. 2020. Long-Run Multi-Robot Planning Under UncertainTask Durations. In Proc. of the 19th International Conference on AutonomousAgents and Multiagent Systems (AAMAS 2020), Auckland, New Zealand, May9–13, 2020, IFAAMAS, 3 pages.
1 INTRODUCTIONLong-term multi-robot coordination is important in many real-world robotics applications, such as disaster prevention, mining,traffic-monitoring, and ware-house automation. We focus on theproblem of synthesizing policies to coordinate teams of robotsin these applications. In such scenarios, it is crucial to deploy ro-bust and efficient behaviors. There are many factors that influencethe team’s performance, and consequently, approaches that ac-count for as many factors as possible usually perform better. Animportant factor is uncertainty that emerges from the executionin non-controlled environments. Common sources of uncertaintyare present in the outcome of the actions, in the duration of eachtask, or in the battery autonomy of each robot. Moreover, sincethese applications are inherently long term missions, approachesthat take into account infinite horizons usually perform more effi-ciently. Many of the currently deployed solutions for multi-robotplanning [7] make use of hand-crafted behaviors, not reasoning
Proc. of the 19th International Conference on Autonomous Agents and Multiagent Systems(AAMAS 2020), B. An, N. Yorke-Smith, A. El Fallah Seghrouchni, G. Sukthankar (eds.), May9–13, 2020, Auckland, New Zealand. © 2020 International Foundation for AutonomousAgents and Multiagent Systems (www.ifaamas.org). All rights reserved.
explicitly about uncertainty. These ad-hoc approaches can be de-pendable, but every new scenario requires a significant engineeringeffort. Furthermore, as the scale of the deployments grows, rulebased approaches tend to be harder to define and become moresuboptimal.
In recent works, including one awaiting publication at AAMAS,we proposed a methodology to solve multi-robot persistent moni-toring problems. We proposed a model-based methodology basedon an generalized stochastic Petri net with rewards (GSPNR), anextension of generalized stochastic Petri nets (GSPNs) [1] to in-clude rewards. We take advantage of the model checking algorithmproposed by [2], in order to synthesize policies that optimize thelong-run average reward criterion.
2 GSPNRS FOR MULTI-ROBOT TEAMSExample 2.1. Consider the problemwhere an homogeneous team
of robots can move between a set of locations. The goal is to per-sistently monitor some of these locations. In order to achieve that,each robot can perform some tasks such as navigate or gather data.However, each robot has a limited battery autonomy and, can onlyexecute tasks for a certain amount of time. The time that it takesto execute each task as well as the time that it takes to dischargeor recharge is not deterministic since it is influenced by manyuncontrollable factors.
We extend the GSPN-based modeling approach presented in [5]to capture in one single model multi-robot persistent monitor-ing problems including the objective to be optimized. The keypoint to achieve this is representing each robot as a token, whichis possible under the assumption that our team of robots is ho-mogeneous. A GSPNR for multi-robot teams is defined as GR =⟨P ,T ,W +,W −, F ,m0, rP , rT ⟩. P is a finite set of places that repre-sent tasks that each robot can execute, or external processes thateach robot must wait to be accomplished, such as battery charg-ing. T = TI ∪TE is a finite set of transitions partitioned into twosubsets, where TI contains all immediate transitions and TE con-tains all exponential transitions. The exponential transitions, modelthe time uncertainty associated with these uncontrollable events,such as the time that a robot takes to move from one location toanother. The immediate transitions represent the controllable ac-tions that the team of robots has available.W − : P ×T → N andW + : T × P → N are input and output arc weight functions, re-spectively. Input arcs assign to tasks uncontrollable events or thechoice of deciding among multiple controllable actions. Output arcsassign to each event the outcome states to where the system islead after selecting a particular action or after the conclusion ofan uncontrollable event. The goal is specified through the use of
Doctoral Consortium AAMAS 2020, May 9–13, Auckland, New Zealand
2168
rewards. There are two types of rewards that can be assigned tothe model: place rewards and transition rewards. The place reward,rP : P → R≥0, is awarded, per time unit, while at least one tokenis present in the assigned place. Therefore, the reward obtained bythe multi-robot system is proportional to the amount of time inthe assigned places, encouraging the team of robots to remain oravoid those places, depending on whether it is posed as a maximiza-tion or minimization problem. The transition rewards are givenby rT : TI → R≥0, with reward rT (tn) being awarded every timetn ∈ TI is fired, i.e. whenever a robot takes the action correspondingto ti . These rewards are useful when specifying a goal where someparticular actions should be promoted or discouraged.
Figure 1 shows the GSPNRmodel that captures an instance of themonitoring problem stated in Example 2.1. This problem consistsin a team of 3 robots that must monitor 2 locations. The robots, de-noted as tokens, are able to monitor, navigate and recharge. Thesetasks are captured by the places labeled asMonitorinд, Naviдatinдand Charдinд, respectively. The uncertainty associated with theduration of each of these tasks is captured by the exponential tran-sitions labeled as Finished ,Arrived andCharдed , respectively. Theexponential transitions labeled as Discharдed capture the stochas-ticity of the multi-robot team battery autonomy. Moreover, theplaces labeled as Ready represent states where the robots can de-cide between monitoring again the same location or moving toanother location, by choosing between the immediate transitionsRepeat and GoFromTo, respectively.
The interpretation of the marking process of the GSPNR modelas a Markov reward automaton (MRA) [4] allows the use of themethod proposed by [2] to compute the optimal long-run averagereward and extract the policy that maximizes this criterion. Figure 2exemplifies how to represent a GSPNR as an MRA. Each reachablemarking is interpreted as a state of the MRA, and the initial mark-ing corresponds to the initial state. Immediate and exponentialtransitions have the same meaning in GSPNR and MRA. The setof actions is formed by the set of immediate transitions, plus anaction to characterize all exponential transitions, denoted as ⊥. Thestate rewards correspond to the sum of all place rewards with atleast one token while the transition rewards have a one-to-onecorrespondence.
However, this method only allows for a straightforward ex-traction of the corresponding policy when the produced MRA isunichain [8]. To guarantee that we only produce unichain MRAs,we restrict our GSPNR models to be reversible [6], i.e. such thatevery marking is reachable from all the other markings.
3 FUTUREWORKWe split the future work into three research directions: more expres-sive models, scalable methods and approaches that allow capturingthe trade-off between multiple objectives.
Currently the presented model only captures the multi-robotteam behavior and its interaction with the environment. We intendto extend the model to also capture resources such as the timeelapsed since the last time a location was visited, or the batterylevel of each robot. Since the MRA also accounts for uncertainty inthe action outcomes we plan to take advantage of this and incor-porate in our GSPNR model the possibility to obtain policies that
FinishedL1
<latexit sha1_base64="9DNcEWUyWTCtCPLnypFByzOTkz0=">AAACBHicbVDLSsNAFJ34rPEVddlNsBFclaQouiwI4sJFBfuANpTJ5KYdOpmEmYlQQhdu/BU3LhRx60e482+ctllo64ELh3Punbn3BCmjUrnut7Gyura+sVnaMrd3dvf2rYPDlkwyQaBJEpaIToAlMMqhqahi0EkF4Dhg0A5GV1O//QBC0oTfq3EKfowHnEaUYKWlvlXuEeAKBOUD07mmnMohhI5pOree07cqbtWdwV4mXkEqqECjb331woRksX6RMCxl13NT5edYKEoYTMxeJiHFZIQH0NWU4xikn8+OmNgnWgntKBG6uLJn6u+JHMdSjuNAd8ZYDeWiNxX/87qZii79nPI0U8DJ/KMoY7ZK7GkidkgFEMXGmmAiqN7VJkMsMNGxSFOH4C2evExatap3Vj2/q1XqnSKOEiqjY3SKPHSB6ugGNVATEfSIntErejOejBfj3fiYt64YxcwR+gPj8we8bpZF</latexit>
MonitoringL1
<latexit sha1_base64="jkKESR30aCHY3kHct3428jdiSo0=">AAACBnicbZBNS8MwGMdTX2d9q3oUIbgKnkY7FD0OvHhQmOBeYCsjzdItLE1Kkgqj7OTFr+LFgyJe/Qze/DamWw+6+YfAj//zPEmef5gwqrTnfVtLyyura+ulDXtza3tn19nbbyqRSkwaWDAh2yFShFFOGppqRtqJJCgOGWmFo6u83nogUlHB7/U4IUGMBpxGFCNtrJ5z1MWEayIpH9jureBUi5xd23ZvfLfnlL2KNxVcBL+AMihU7zlf3b7AaWzuxAwp1fG9RAcZkppiRiZ2N1UkQXiEBqRjkKOYqCCbrjGBJ8bpw0hIc7iGU/f3RIZipcZxaDpjpIdqvpab/9U6qY4ug4zyJNWE49lDUcqgFjDPBPapJFizsQGEJTV/hXiIJMImGGWbEPz5lRehWa34Z5Xzu2q51i7iKIFDcAxOgQ8uQA1cgzpoAAwewTN4BW/Wk/VivVsfs9Ylq5g5AH9kff4Am+WXVQ==</latexit>
ReadyL1
<latexit sha1_base64="rmCkCWBWcH3SWCUCQRIZbiLynhQ=">AAACAXicbVDLSsNAFJ34rPEVdSO4CTaCq5IURZcFNy5cVLEPaEOZTG7boZNJmJkIIdSNv+LGhSJu/Qt3/o3TNgttPXDhcM69M/eeIGFUKtf9NpaWV1bX1ksb5ubW9s6utbfflHEqCDRIzGLRDrAERjk0FFUM2okAHAUMWsHoauK3HkBIGvN7lSXgR3jAaZ8SrLTUsw67BLgCQfnAdO4Ah5ljms6N5/Sssltxp7AXiVeQMipQ71lf3TAmaaSfIwxL2fHcRPk5FooSBmOzm0pIMBnhAXQ05TgC6efTC8b2iVZCux8LXVzZU/X3RI4jKbMo0J0RVkM5703E/7xOqvqXfk55kirgZPZRP2W2iu1JHHZIBRDFMk0wEVTvapMhFpjoTKSpQ/DmT14kzWrFO6uc31bLtXYRRwkdoWN0ijx0gWroGtVRAxH0iJ7RK3oznowX4934mLUuGcXMAfoD4/MHW/qU8g==</latexit>
GoFromL1ToL2
<latexit sha1_base64="m+CiYnqKJmz0No9ge63/3b4MHnU=">AAACBnicbVDLSgMxFM3UVx1foy5FCLaCqzJTFF0WBHXRRYW+oB1KJs20oZlkSDJCGbpy46+4caGIW7/BnX9j2s5CWw9cOJxzb3LvCWJGlXbdbyu3srq2vpHftLe2d3b3nP2DphKJxKSBBROyHSBFGOWkoalmpB1LgqKAkVYwup76rQciFRW8rscx8SM04DSkGGkj9ZzjLiZcE0n5wC7eihspoqpXtO1iXVTLxZ5TcEvuDHCZeBkpgAy1nvPV7QucROZNzJBSHc+NtZ8iqSlmZGJ3E0VihEdoQDqGchQR5aezMybw1Ch9GAppims4U39PpChSahwFpjNCeqgWvan4n9dJdHjlp5THiSYczz8KEwa1gNNMYJ9KgjUbG4KwpGZXiIdIImyCUbYJwVs8eZk0yyXvvHRxXy5U2lkceXAETsAZ8MAlqIA7UAMNgMEjeAav4M16sl6sd+tj3pqzsplD8AfW5w+oHpa6</latexit>
NavigatingL1ToL2
<latexit sha1_base64="EiDuk2MFF0sfNUK2MhI63lep1Qs=">AAACCnicbVDLSsNAFJ3UV42vqEs3o43gqiRF0WXBjYsiFfqCNpTJdJIOnUzCzKRQQtdu/BU3LhRx6xe482+ctllo64ELh3Punbn3+AmjUjnOt1FYW9/Y3Cpumzu7e/sH1uFRS8apwKSJYxaLjo8kYZSTpqKKkU4iCIp8Rtr+6Hbmt8dESBrzhpokxItQyGlAMVJa6lunPUy4IoLy0LTv0ZiG2uChbZp2zW3EtYrdt0pO2ZkDrhI3JyWQo963vnqDGKeRfhczJGXXdRLlZUgoihmZmr1UkgThEQpJV1OOIiK9bH7KFJ5rZQCDWOjiCs7V3xMZiqScRL7ujJAaymVvJv7ndVMV3HgZ5UmqCMeLj4KUQRXDWS5wQAXBik00QVhQvSvEQyQQ1uFIU4fgLp+8SlqVsntZvnqolKqdPI4iOAFn4AK44BpUwR2ogybA4BE8g1fwZjwZL8a78bFoLRj5zDH4A+PzBxCxmKA=</latexit>
RepeatL1
<latexit sha1_base64="hf0UHIdDUBbxK2LS3RSkh/H724c=">AAACAnicbVDLSsNAFJ34rPEVdSVuBhvBVUmKosuCGxcuqtgHtKFMprft0MkkzEyEEoobf8WNC0Xc+hXu/BunbRbaeuDC4Zx7Z+49YcKZ0p73bS0tr6yurRc27M2t7Z1dZ2+/ruJUUqjRmMeyGRIFnAmoaaY5NBMJJAo5NMLh1cRvPIBULBb3epRAEJG+YD1GiTZSxzlsUxAaJBN9272DBIh2bdu98d2OU/RK3hR4kfg5KaIc1Y7z1e7GNI3Me5QTpVq+l+ggI1IzymFst1MFCaFD0oeWoYJEoIJsesIYnxili3uxNCU0nqq/JzISKTWKQtMZET1Q895E/M9rpbp3GWRMJKkGQWcf9VKOdYwneeAuk0A1HxlCqGRmV0wHRBJqQlG2CcGfP3mR1Msl/6x0flsuVpp5HAV0hI7RKfLRBaqga1RFNUTRI3pGr+jNerJerHfrY9a6ZOUzB+gPrM8fLjOVaA==</latexit>
ArrivedL2
<latexit sha1_base64="Tzs/UKS7QJOcTEk/sng2XRl4yZI=">AAAB/3icbVDLSsNAFJ3UV42vqODGTbAVXJWkKLqsuHHhooJ9QBvKZHLTDp1MwsykUGIX/oobF4q49Tfc+TdO2yy09cCFwzn3ztx7/IRRqRzn2yisrK6tbxQ3za3tnd09a/+gKeNUEGiQmMWi7WMJjHJoKKoYtBMBOPIZtPzhzdRvjUBIGvMHNU7Ai3Cf05ASrLTUs466BLgCQXnfLF8LQUcQ3FXLPavkVJwZ7GXi5qSEctR71lc3iEka6ccIw1J2XCdRXoaFooTBxOymEhJMhrgPHU05jkB62Wz/iX2qlcAOY6GLK3um/p7IcCTlOPJ1Z4TVQC56U/E/r5Oq8MrLKE9SBZzMPwpTZqvYnoZhB1QAUWysCSaC6l1tMsACE52INHUI7uLJy6RZrbjnlYv7aqnWzuMoomN0gs6Qiy5RDd2iOmoggh7RM3pFb8aT8WK8Gx/z1oKRzxyiPzA+fwDEDpVb</latexit>
DischargedL1
<latexit sha1_base64="fkh9+q6mZEgtfxP2pQBzyVB4xGI=">AAACAnicbVDLSsNAFJ34rPEVdSVugq3gqiRF0WVBFy5cVLAPaEOZTG7SoZNJmJkIJRQ3/oobF4q49Svc+TdO2i609cCFwzn3ztx7/JRRqRzn21haXlldWy9tmJtb2zu71t5+SyaZINAkCUtEx8cSGOXQVFQx6KQCcOwzaPvDq8JvP4CQNOH3apSCF+OI05ASrLTUtw57BLgCQXlkVq6pJAMsIghu3UrfKjtVZwJ7kbgzUkYzNPrWVy9ISBbr9wjDUnZdJ1VejoWihMHY7GUSUkyGOIKuphzHIL18csLYPtFKYIeJ0MWVPVF/T+Q4lnIU+7ozxmog571C/M/rZiq89HLK00wBJ9OPwozZKrGLPOyACiCKjTTBRFC9q11kgIkORZo6BHf+5EXSqlXds+r5Xa1c78ziKKEjdIxOkYsuUB3doAZqIoIe0TN6RW/Gk/FivBsf09YlYzZzgP7A+PwBAamWmQ==</latexit>
DischargedL2
<latexit sha1_base64="Vclyifb6AMoFZha+nosQNKvY6K8=">AAACAnicbVDLSsNAFJ34rPEVdSVugq3gqiRF0WVBFy5cVLAPaEOZTG7SoZNJmJkIJRQ3/oobF4q49Svc+TdO2i609cCFwzn3ztx7/JRRqRzn21haXlldWy9tmJtb2zu71t5+SyaZINAkCUtEx8cSGOXQVFQx6KQCcOwzaPvDq8JvP4CQNOH3apSCF+OI05ASrLTUtw57BLgCQXlkVq6pJAMsIghua5W+VXaqzgT2InFnpIxmaPStr16QkCzW7xGGpey6Tqq8HAtFCYOx2cskpJgMcQRdTTmOQXr55ISxfaKVwA4ToYsre6L+nshxLOUo9nVnjNVAznuF+J/XzVR46eWUp5kCTqYfhRmzVWIXedgBFUAUG2mCiaB6V7vIABMdijR1CO78yYukVau6Z9Xzu1q53pnFUUJH6BidIhddoDq6QQ3URAQ9omf0it6MJ+PFeDc+pq1LxmzmAP2B8fkDAy6Wmg==</latexit>
MonitoringL2
<latexit sha1_base64="tvl8qv4aae7IuIJS1RDk1HxOjHg=">AAACBnicbZBNS8MwGMdTX2d9q3oUIbgKnkY7FD0OvHhQmOBeYCsjzdItLE1Kkgqj7OTFr+LFgyJe/Qze/DamWw+6+YfAj//zPEmef5gwqrTnfVtLyyura+ulDXtza3tn19nbbyqRSkwaWDAh2yFShFFOGppqRtqJJCgOGWmFo6u83nogUlHB7/U4IUGMBpxGFCNtrJ5z1MWEayIpH9jureBUi5xd23Zvqm7PKXsVbyq4CH4BZVCo3nO+un2B09jciRlSquN7iQ4yJDXFjEzsbqpIgvAIDUjHIEcxUUE2XWMCT4zTh5GQ5nANp+7viQzFSo3j0HTGSA/VfC03/6t1Uh1dBhnlSaoJx7OHopRBLWCeCexTSbBmYwMIS2r+CvEQSYRNMMo2IfjzKy9Cs1rxzyrnd9VyrV3EUQKH4BicAh9cgBq4BnXQABg8gmfwCt6sJ+vFerc+Zq1LVjFzAP7I+vwBnWqXVg==</latexit>
FinishedL2
<latexit sha1_base64="s/vlXhg/tkYD+//qLlcDTlaXluE=">AAACBHicbVDLSsNAFJ34rPEVddlNsBFclaQouiwI4sJFBfuANpTJ5KYdOpmEmYlQQhdu/BU3LhRx60e482+ctllo64ELh3Punbn3BCmjUrnut7Gyura+sVnaMrd3dvf2rYPDlkwyQaBJEpaIToAlMMqhqahi0EkF4Dhg0A5GV1O//QBC0oTfq3EKfowHnEaUYKWlvlXuEeAKBOUD07mmnMohhI5pOrc1p29V3Ko7g71MvIJUUIFG3/rqhQnJYv0iYVjKruemys+xUJQwmJi9TEKKyQgPoKspxzFIP58dMbFPtBLaUSJ0cWXP1N8TOY6lHMeB7oyxGspFbyr+53UzFV36OeVppoCT+UdRxmyV2NNE7JAKIIqNNcFEUL2rTYZYYKJjkaYOwVs8eZm0alXvrHp+V6vUO0UcJVRGx+gUeegC1dENaqAmIugRPaNX9GY8GS/Gu/Exb10xipkj9AfG5w+985ZG</latexit>
ReadyL2
<latexit sha1_base64="6dErJAyF5tXvgZgZei0FJqLvHi4=">AAACAXicbVDLSsNAFJ34rPEVdSO4CTaCq5IURZcFNy5cVLEPaEOZTG7boZNJmJkIIdSNv+LGhSJu/Qt3/o3TNgttPXDhcM69M/eeIGFUKtf9NpaWV1bX1ksb5ubW9s6utbfflHEqCDRIzGLRDrAERjk0FFUM2okAHAUMWsHoauK3HkBIGvN7lSXgR3jAaZ8SrLTUsw67BLgCQfnAdO4Ah5ljms5N1elZZbfiTmEvEq8gZVSg3rO+umFM0kg/RxiWsuO5ifJzLBQlDMZmN5WQYDLCA+hoynEE0s+nF4ztE62Edj8Wuriyp+rviRxHUmZRoDsjrIZy3puI/3mdVPUv/ZzyJFXAyeyjfspsFduTOOyQCiCKZZpgIqje1SZDLDDRmUhTh+DNn7xImtWKd1Y5v62Wa+0ijhI6QsfoFHnoAtXQNaqjBiLoET2jV/RmPBkvxrvxMWtdMoqZA/QHxucPXX+U8w==</latexit>
GoFromL2ToL1
<latexit sha1_base64="Gt4IeTQKq86X3Y5vSul0MupPlqQ=">AAACBnicbVDLSgMxFM3UVx1foy5FCLaCqzJTFF0WBHXRRYW+oB1KJs20oZlkSDJCGbpy46+4caGIW7/BnX9j2s5CWw9cOJxzb3LvCWJGlXbdbyu3srq2vpHftLe2d3b3nP2DphKJxKSBBROyHSBFGOWkoalmpB1LgqKAkVYwup76rQciFRW8rscx8SM04DSkGGkj9ZzjLiZcE0n5wC7eihspomq5aNvFuqh6xZ5TcEvuDHCZeBkpgAy1nvPV7QucROZNzJBSHc+NtZ8iqSlmZGJ3E0VihEdoQDqGchQR5aezMybw1Ch9GAppims4U39PpChSahwFpjNCeqgWvan4n9dJdHjlp5THiSYczz8KEwa1gNNMYJ9KgjUbG4KwpGZXiIdIImyCUbYJwVs8eZk0yyXvvHRxXy5U2lkceXAETsAZ8MAlqIA7UAMNgMEjeAav4M16sl6sd+tj3pqzsplD8AfW5w+oJpa6</latexit>
NavigatingL2ToL1
<latexit sha1_base64="6Yag16x1trUB8BWhAy8KyLrlTVM=">AAACCnicbVDLSsNAFJ3UV42vqEs3o43gqiRF0WXBjYsiFfqCNpTJdJIOnUzCzKRQQtdu/BU3LhRx6xe482+ctllo64ELh3Punbn3+AmjUjnOt1FYW9/Y3Cpumzu7e/sH1uFRS8apwKSJYxaLjo8kYZSTpqKKkU4iCIp8Rtr+6Hbmt8dESBrzhpokxItQyGlAMVJa6lunPUy4IoLy0LTv0ZiG2uChbZp2rdKIa67dt0pO2ZkDrhI3JyWQo963vnqDGKeRfhczJGXXdRLlZUgoihmZmr1UkgThEQpJV1OOIiK9bH7KFJ5rZQCDWOjiCs7V3xMZiqScRL7ujJAaymVvJv7ndVMV3HgZ5UmqCMeLj4KUQRXDWS5wQAXBik00QVhQvSvEQyQQ1uFIU4fgLp+8SlqVsntZvnqolKqdPI4iOAFn4AK44BpUwR2ogybA4BE8g1fwZjwZL8a78bFoLRj5zDH4A+PzBxC1mKA=</latexit>
Charging
<latexit sha1_base64="QS5Yv+NzXKDmmToOvaJAby/1dOI=">AAAB/nicbZDLSsNAFIZPvNZ4i4orN4Ot4KokRdFloRuXFewF2lAm00k6dDIJMxOhhIKv4saFIm59Dne+jdM2C209MPDx/+fMnPmDlDOlXffbWlvf2NzaLu3Yu3v7B4fO0XFbJZkktEUSnshugBXlTNCWZprTbiopjgNOO8G4MfM7j1QqlogHPUmpH+NIsJARrI00cE77hApNJRORXWmMsIwMVQZO2a2680Kr4BVQhqKaA+erP0xIFpu7CMdK9Tw31X6OpWaE06ndzxRNMRnjiPYMChxT5efz9afowihDFCbSHKHRXP09keNYqUkcmM4Y65Fa9mbif14v0+GtnzORZpoKsngozDjSCZplgYZMUqL5xAAmkpldETERYGICUbYJwVv+8iq0a1Xvqnp9XyvXu0UcJTiDc7gED26gDnfQhBYQyOEZXuHNerJerHfrY9G6ZhUzJ/CnrM8fWj+VKQ==</latexit>
Charged
<latexit sha1_base64="Io3/YWQdDw13/FueXNV+XZgjXzc=">AAAB/XicbVDLSsNAFJ3UV42v+Ni5CbaCq5IURZeFblxWsA9oQ5lMbtKhk0mYmQg1FH/FjQtF3Pof7vwbp20W2nrgwuGce2fuPX7KqFSO822U1tY3NrfK2+bO7t7+gXV41JFJJgi0ScIS0fOxBEY5tBVVDHqpABz7DLr+uDnzuw8gJE34vZqk4MU44jSkBCstDa2TAQGuQFAemdXmCIsIgurQqjg1Zw57lbgFqaACraH1NQgSksX6KcKwlH3XSZWXY6EoYTA1B5mEFJMxjqCvKccxSC+fbz+1z7US2GEidHFlz9XfEzmOpZzEvu6MsRrJZW8m/uf1MxXeeDnlaaaAk8VHYcZsldizKOyACiCKTTTBRFC9q010ApjoPKSpQ3CXT14lnXrNvaxd3dUrjV4RRxmdojN0gVx0jRroFrVQGxH0iJ7RK3oznowX4934WLSWjGLmGP2B8fkDfDWUqg==</latexit>
ArrivedL1
<latexit sha1_base64="eUDjEMMtd9JJJ1u69DY5sMngM9A=">AAAB/3icbVDLSsNAFJ3UV42vqODGTbAVXJWkKLqsuHHhooJ9QBvKZHLTDp1MwsykUGIX/oobF4q49Tfc+TdO2yy09cCFwzn3ztx7/IRRqRzn2yisrK6tbxQ3za3tnd09a/+gKeNUEGiQmMWi7WMJjHJoKKoYtBMBOPIZtPzhzdRvjUBIGvMHNU7Ai3Cf05ASrLTUs466BLgCQXnfLF8LQUcQ3LnlnlVyKs4M9jJxc1JCOeo966sbxCSN9GOEYSk7rpMoL8NCUcJgYnZTCQkmQ9yHjqYcRyC9bLb/xD7VSmCHsdDFlT1Tf09kOJJyHPm6M8JqIBe9qfif10lVeOVllCepAk7mH4Ups1VsT8OwAyqAKDbWBBNB9a42GWCBiU5EmjoEd/HkZdKsVtzzysV9tVRr53EU0TE6QWfIRZeohm5RHTUQQY/oGb2iN+PJeDHejY95a8HIZw7RHxifP8KJlVo=</latexit>
RepeatL2
<latexit sha1_base64="lJ8LGHbGLdFgrvmTVQgsI7LIV0I=">AAACAnicbVDLSsNAFJ34rPEVdSVuBhvBVUmKosuCGxcuqtgHtKFMprft0MkkzEyEEoobf8WNC0Xc+hXu/BunbRbaeuDC4Zx7Z+49YcKZ0p73bS0tr6yurRc27M2t7Z1dZ2+/ruJUUqjRmMeyGRIFnAmoaaY5NBMJJAo5NMLh1cRvPIBULBb3epRAEJG+YD1GiTZSxzlsUxAaJBN9272DBIh2bdu9Kbsdp+iVvCnwIvFzUkQ5qh3nq92NaRqZ9ygnSrV8L9FBRqRmlMPYbqcKEkKHpA8tQwWJQAXZ9IQxPjFKF/diaUpoPFV/T2QkUmoUhaYzInqg5r2J+J/XSnXvMsiYSFINgs4+6qUc6xhP8sBdJoFqPjKEUMnMrpgOiCTUhKJsE4I/f/IiqZdL/lnp/LZcrDTzOAroCB2jU+SjC1RB16iKaoiiR/SMXtGb9WS9WO/Wx6x1ycpnDtAfWJ8/L7iVaQ==</latexit>
Figure 1: GSPNR model of a simple monitoring examplewith 3 robots and 2 locations.
�
<latexit sha1_base64="LqGeAOjJVRvwWX8iuvzNiaBqF7g=">AAAB7nicbVDLSsNAFL2pr1pfVZduBovgqiRS0WXBjcsK9gFtKDeTSTt0MgkzE6GEfoQbF4q49Xvc+TdO2yy09cDA4ZxzmXtPkAqujet+O6WNza3tnfJuZW//4PCoenzS0UmmKGvTRCSqF6BmgkvWNtwI1ksVwzgQrBtM7uZ+94kpzRP5aKYp82McSR5xisZK3YGw0RCH1Zpbdxcg68QrSA0KtIbVr0GY0Cxm0lCBWvc9NzV+jspwKtisMsg0S5FOcMT6lkqMmfbzxbozcmGVkESJsk8aslB/T+QYaz2NA5uM0Yz1qjcX//P6mYlu/ZzLNDNM0uVHUSaIScj8dhJyxagRU0uQKm53JXSMCqmxDVVsCd7qyeukc1X3GvXrh0at2SvqKMMZnMMleHADTbiHFrSBwgSe4RXenNR5cd6dj2W05BQzp/AHzucPRfuPmw==</latexit>
Ready
<latexit sha1_base64="pXWSya6J9UcICwPdMjcCP5hZVcc=">AAAB7HicbVBNS8NAEJ3Ur1q/qh69BIvgqSSi6LHgxWMV0xbaUDabSbt0swm7GyGE/gYvHhTx6g/y5r9x2+agrQ8GHu/NMDMvSDlT2nG+rcra+sbmVnW7trO7t39QPzzqqCSTFD2a8ET2AqKQM4GeZppjL5VI4oBjN5jczvzuE0rFEvGo8xT9mIwEixgl2kjeA5IwH9YbTtOZw14lbkkaUKI9rH8NwoRmMQpNOVGq7zqp9gsiNaMcp7VBpjAldEJG2DdUkBiVX8yPndpnRgntKJGmhLbn6u+JgsRK5XFgOmOix2rZm4n/ef1MRzd+wUSaaRR0sSjKuK0Te/a5HTKJVPPcEEIlM7fadEwkodrkUzMhuMsvr5LORdO9bF7dXzZavTKOKpzAKZyDC9fQgjtogwcUGDzDK7xZwnqx3q2PRWvFKmeO4Q+szx/ISY6/</latexit>
ExecuteTask
<latexit sha1_base64="cROY7T0taW4NmfYph65QhhoyALA=">AAACAXicbVDLSgMxFM3UVx1fo24EN8EiuCozRdFlQQSXFfqCdiiZ9E4bmskMSUYsQ934K25cKOLWv3Dn35i2s9DWA4HDOfcmOSdIOFPadb+twsrq2vpGcdPe2t7Z3XP2D5oqTiWFBo15LNsBUcCZgIZmmkM7kUCigEMrGF1P/dY9SMViUdfjBPyIDAQLGSXaSD3nqEtBaJBMDOybB6CpBtuuEzXqOSW37M6Al4mXkxLKUes5X91+TNPIXEc5UarjuYn2MyI1oxwmdjdVkBA6IgPoGCpIBMrPZgkm+NQofRzG0hyh8Uz9vZGRSKlxFJjJiOihWvSm4n9eJ9XhlZ8xkZhggs4fClOOdYyndeA+k0A1HxtCqGTmr5gOiSTUdKJsU4K3GHmZNCtl77x8cVcpVdt5HUV0jE7QGfLQJaqiW1RDDUTRI3pGr+jNerJerHfrYz5asPKdQ/QH1ucPfc+WVg==</latexit>
ExecutingTask
<latexit sha1_base64="y5EDjK7m3ml/slHXW7j4UT/2tQ8=">AAACA3icbZDLSgMxFIYz9VbH26g73QSL4KrMFEWXBRFcVugN2qFk0tM2NJMZkoxYhoIbX8WNC0Xc+hLufBsz7Sy09YfAx3/OSXL+IOZMadf9tgorq2vrG8VNe2t7Z3fP2T9oqiiRFBo04pFsB0QBZwIammkO7VgCCQMOrWB8ndVb9yAVi0RdT2LwQzIUbMAo0cbqOUddCkKDZGJo3zwATXRGdp2occ8puWV3JrwMXg4llKvWc766/YgmobmQcqJUx3Nj7adEakY5TO1uoiAmdEyG0DEoSAjKT2c7TPGpcfp4EElzhMYz9/dESkKlJmFgOkOiR2qxlpn/1TqJHlz5KRNxokHQ+UODhGMd4SwQ3GcSqOYTA4RKZv6K6YhIQk0qyjYheIsrL0OzUvbOyxd3lVK1ncdRRMfoBJ0hD12iKrpFNdRAFD2iZ/SK3qwn68V6tz7mrQUrnzlEf2R9/gAksZdD</latexit>
ExecutionFinished
<latexit sha1_base64="SDkNa9zfNrI4pqWKHQMBpZ2tm+0=">AAACB3icbVDLSsNAFJ3UV42vqEtBgkVwVZKi6LIgissK9gFtKJPJTTt0MgkzE7GE7tz4K25cKOLWX3Dn3zhps9DWAxcO59w7c+/xE0alcpxvo7S0vLK6Vl43Nza3tnes3b2WjFNBoEliFouOjyUwyqGpqGLQSQTgyGfQ9keXud++ByFpzO/UOAEvwgNOQ0qw0lLfOuwR4AoE5QPz6gFImsumeU05lUMI+lbFqTpT2IvELUgFFWj0ra9eEJM00o8ShqXsuk6ivAwLRQmDidlLJSSYjPAAuppyHIH0sukdE/tYK4EdxkIXV/ZU/T2R4UjKceTrzgiroZz3cvE/r5uq8MLLKE9SBZzMPgpTZqvYzkOxAyqAKDbWBBNB9a42GWKBiU5GmjoEd/7kRdKqVd3T6tltrVLvFHGU0QE6QifIReeojm5QAzURQY/oGb2iN+PJeDHejY9Za8koZvbRHxifP1cmmQo=</latexit>
s1 = [2, 0]
<latexit sha1_base64="Su0YEeGP3mnCTaUwMLqI/Ccg6rk=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYBA8SNgNEb0IAS8eI5gHbJYwO5lNhszOLjO9QljyGV48KOLVr/Hm3zh5HDSxoKGo6qa7K0ylMOi6387a+sbm1nZhp7i7t39wWDo6bpkk04w3WSIT3Qmp4VIo3kSBkndSzWkcSt4OR3dTv/3EtRGJesRxyoOYDpSIBKNoJd/0PHJL/OqlG/RKZbfizkBWibcgZVig0St9dfsJy2KukElqjO+5KQY51SiY5JNiNzM8pWxEB9y3VNGYmyCfnTwh51bpkyjRthSSmfp7IqexMeM4tJ0xxaFZ9qbif56fYXQT5EKlGXLF5ouiTBJMyPR/0heaM5RjSyjTwt5K2JBqytCmVLQheMsvr5JWteLVKlcPtXK9s4ijAKdwBhfgwTXU4R4a0AQGCTzDK7w56Lw4787HvHXNWcycwB84nz/uCY/M</latexit>
s2 = [1, 1]
<latexit sha1_base64="/GbycD2Y0b2+ToGN7Bfdnaxhtdw=">AAAB8nicbVDLSgNBEOz1GeMr6tHLYBA8SNgNEb0IAS8eI5gHbJYwO5lNhszOLjO9QljyGV48KOLVr/Hm3zh5HDSxoKGo6qa7K0ylMOi6387a+sbm1nZhp7i7t39wWDo6bpkk04w3WSIT3Qmp4VIo3kSBkndSzWkcSt4OR3dTv/3EtRGJesRxyoOYDpSIBKNoJd/0quSW+N6lF/RKZbfizkBWibcgZVig0St9dfsJy2KukElqjO+5KQY51SiY5JNiNzM8pWxEB9y3VNGYmyCfnTwh51bpkyjRthSSmfp7IqexMeM4tJ0xxaFZ9qbif56fYXQT5EKlGXLF5ouiTBJMyPR/0heaM5RjSyjTwt5K2JBqytCmVLQheMsvr5JWteLVKlcPtXK9s4ijAKdwBhfgwTXU4R4a0AQGCTzDK7w56Lw4787HvHXNWcycwB84nz/vk4/N</latexit>
ExecuteTask
<latexit sha1_base64="GKMyYfxiNZw2gJ0WDwz2D26wisc=">AAAB8nicbVDLSgMxFL1TX7W+qi7dBIvgqsyIosuCCC4r9AXToWTSTBuayQzJHbGUfoYbF4q49Wvc+Tem7Sy09UDgcM695J4TplIYdN1vp7C2vrG5Vdwu7ezu7R+UD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8PR7cxvP3JtRKIaOE55ENOBEpFgFK3k3z1xliFvUDPqlStu1Z2DrBIvJxXIUe+Vv7r9hGUxV8gkNcb33BSDCdUomOTTUjczPKVsRAfct1TRmJtgMj95Ss6s0idRou1TSObq740JjY0Zx6GdjCkOzbI3E//z/Ayjm2AiVGpjKbb4KMokwYTM8pO+0JyhHFtCmRb2VsKGVFOGtqWSLcFbjrxKWhdV77J69XBZqXXyOopwAqdwDh5cQw3uoQ5NYJDAM7zCm4POi/PufCxGC06+cwx/4Hz+AGtvkWw=</latexit>
1.0
<latexit sha1_base64="1GwQSoeNKLTmG6tQc4R9EHRtlJo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0WPBi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t39QPTxq6SRTDJssEYnqhFSj4BKbhhuBnVQhjUOB7XB8O/PbT6g0T+SjmaQYxHQoecQZNVZ68F2vX615rjcHWSV+QWpQoNGvfvUGCctilIYJqnXX91IT5FQZzgROK71MY0rZmA6xa6mkMeogn586JWdWGZAoUbakIXP190ROY60ncWg7Y2pGetmbif953cxEN0HOZZoZlGyxKMoEMQmZ/U0GXCEzYmIJZYrbWwkbUUWZselUbAj+8surpHXh+pfu1f1lrd4p4ijDCZzCOfhwDXW4gwY0gcEQnuEV3hzhvDjvzseiteQUM8fwB87nD1y0jUU=</latexit>
s3 = [0, 2]
<latexit sha1_base64="sxNCjDCxMlUtxwnY3Hww0Jc615w=">AAAB9HicbVBNS8NAEJ3Ur1q/qh69LBbBg5SkVvQiFLx4rGA/IA1ls920SzebuLsplNDf4cWDIl79Md78N27THLT1wcDjvRlm5vkxZ0rb9rdVWFvf2Nwqbpd2dvf2D8qHR20VJZLQFol4JLs+VpQzQVuaaU67saQ49Dnt+OO7ud+ZUKlYJB71NKZeiIeCBYxgbSRP9dPLGbpFrn1R8/rlil21M6BV4uSkAjma/fJXbxCRJKRCE46Vch071l6KpWaE01mplygaYzLGQ+oaKnBIlZdmR8/QmVEGKIikKaFRpv6eSHGo1DT0TWeI9Ugte3PxP89NdHDjpUzEiaaCLBYFCUc6QvME0IBJSjSfGoKJZOZWREZYYqJNTiUTgrP88ipp16pOvXr1UK80unkcRTiBUzgHB66hAffQhBYQeIJneIU3a2K9WO/Wx6K1YOUzx/AH1ucPvZqQ2g==</latexit>
ExecuteTask
<latexit sha1_base64="GKMyYfxiNZw2gJ0WDwz2D26wisc=">AAAB8nicbVDLSgMxFL1TX7W+qi7dBIvgqsyIosuCCC4r9AXToWTSTBuayQzJHbGUfoYbF4q49Wvc+Tem7Sy09UDgcM695J4TplIYdN1vp7C2vrG5Vdwu7ezu7R+UD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8PR7cxvP3JtRKIaOE55ENOBEpFgFK3k3z1xliFvUDPqlStu1Z2DrBIvJxXIUe+Vv7r9hGUxV8gkNcb33BSDCdUomOTTUjczPKVsRAfct1TRmJtgMj95Ss6s0idRou1TSObq740JjY0Zx6GdjCkOzbI3E//z/Ayjm2AiVGpjKbb4KMokwYTM8pO+0JyhHFtCmRb2VsKGVFOGtqWSLcFbjrxKWhdV77J69XBZqXXyOopwAqdwDh5cQw3uoQ5NYJDAM7zCm4POi/PufCxGC06+cwx/4Hz+AGtvkWw=</latexit>
1.0
<latexit sha1_base64="1GwQSoeNKLTmG6tQc4R9EHRtlJo=">AAAB6nicbVBNS8NAEJ3Ur1q/qh69LBbBU0hE0WPBi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTAXXxvO+ndLa+sbmVnm7srO7t39QPTxq6SRTDJssEYnqhFSj4BKbhhuBnVQhjUOB7XB8O/PbT6g0T+SjmaQYxHQoecQZNVZ68F2vX615rjcHWSV+QWpQoNGvfvUGCctilIYJqnXX91IT5FQZzgROK71MY0rZmA6xa6mkMeogn586JWdWGZAoUbakIXP190ROY60ncWg7Y2pGetmbif953cxEN0HOZZoZlGyxKMoEMQmZ/U0GXCEzYmIJZYrbWwkbUUWZselUbAj+8surpHXh+pfu1f1lrd4p4ijDCZzCOfhwDXW4gwY0gcEQnuEV3hzhvDjvzseiteQUM8fwB87nD1y0jUU=</latexit>
?
<latexit sha1_base64="+qTQkDlztf55RODZNYRMlilaCEA=">AAAB63icbVDLSgNBEJyNrxhfUY9eBoPgKexKRI8BLx4jmAckS5idzCZD5rHM9AphyS948aCIV3/Im3/jbLIHTSxoKKq66e6KEsEt+P63V9rY3NreKe9W9vYPDo+qxycdq1NDWZtqoU0vIpYJrlgbOAjWSwwjMhKsG03vcr/7xIzlWj3CLGGhJGPFY04J5NIg0jCs1vy6vwBeJ0FBaqhAa1j9Gow0TSVTQAWxth/4CYQZMcCpYPPKILUsIXRKxqzvqCKS2TBb3DrHF04Z4VgbVwrwQv09kRFp7UxGrlMSmNhVLxf/8/opxLdhxlWSAlN0uShOBQaN88fxiBtGQcwcIdRwdyumE2IIBRdPxYUQrL68TjpX9aBRv35o1Jq9Io4yOkPn6BIF6AY10T1qoTaiaIKe0St686T34r17H8vWklfMnKI/8D5/ACK5jmE=</latexit>
�
<latexit sha1_base64="LqGeAOjJVRvwWX8iuvzNiaBqF7g=">AAAB7nicbVDLSsNAFL2pr1pfVZduBovgqiRS0WXBjcsK9gFtKDeTSTt0MgkzE6GEfoQbF4q49Xvc+TdO2yy09cDA4ZxzmXtPkAqujet+O6WNza3tnfJuZW//4PCoenzS0UmmKGvTRCSqF6BmgkvWNtwI1ksVwzgQrBtM7uZ+94kpzRP5aKYp82McSR5xisZK3YGw0RCH1Zpbdxcg68QrSA0KtIbVr0GY0Cxm0lCBWvc9NzV+jspwKtisMsg0S5FOcMT6lkqMmfbzxbozcmGVkESJsk8aslB/T+QYaz2NA5uM0Yz1qjcX//P6mYlu/ZzLNDNM0uVHUSaIScj8dhJyxagRU0uQKm53JXSMCqmxDVVsCd7qyeukc1X3GvXrh0at2SvqKMMZnMMleHADTbiHFrSBwgSe4RXenNR5cd6dj2W05BQzp/AHzucPRfuPmw==</latexit>
(a) (b)
Figure 2: (a) Example GSPNR, where circles, solid rectanglesand white rectangles represent places, immediate transi-tions and exponential transitions, respectively. (b) TheMRAobtained from the GSPNR. The dashed lines correspond toexponential transitions and the solid lines to immediatetransitions.
take that into consideration. Furthermore, restricting the approachonly to exponential distributions is suitable for a large number ofscenarios, but still a significant restriction. Consequently, we wouldlike to explore other approaches in order to model uncertaintywith other distributions, by for example incorporating phase-typedistributions.
Despite synthesizing policies that perform very well on the long-run, this method lacks scalability due to the slow convergence ofthe relative value iteration algorithm. Therefore, we are currentlyinvestigating methods that exploit sub-optimal policies in order toscale for larger teams. One direction that we are currently exploringis the adaptation of heuristic search algorithms for MDPs, such aslabeled real time dynamic programming. Moreover, we plan toextend this method to arbitrary GSPNR models, since currently themethod is limited to GSPNRs that produce unichain MRAs.
Additionally, in many scenarios, it is very hard to characterizethe solution of a problem in terms of a unique utility function.These cases require the definition of trade-offs between multipleobjectives. For this reason we will also investigate the extension ofthe current method to handle multi-objective problems, possiblyextending ideas from [3] to the context of continuous-time models.
ACKNOWLEDGMENTSThis work was supported by the Portuguese Fundação para a Ciên-cia e Tecnologia (FCT) under grant SFRH/BD/135014/2017.
Doctoral Consortium AAMAS 2020, May 9–13, Auckland, New Zealand
2169
REFERENCES[1] Gianfranco Balbo. 2007. Introduction to generalized stochastic Petri nets. In
International School on Formal Methods for the Design of Computer, Communicationand Software Systems. Springer, 83–131.
[2] Yuliya Butkova, Ralf Wimmer, and Holger Hermanns. 2017. Long-run rewardsfor Markov automata. In Proceedings of the 19th International Conference on Toolsand Algorithms for the Construction and Analysis of Systems. Springer, 188–203.
[3] Krishnendu Chatterjee. 2007. Markov decision processes with multiple long-run average objectives. In International Conference on Foundations of SoftwareTechnology and Theoretical Computer Science. Springer, 473–484.
[4] Christian Eisentraut, Holger Hermanns, Joost-Pieter Katoen, and Lijun Zhang.2013. A semantics for every GSPN. In Proceedings of the 34th InternationalConference on Applications and Theory of Petri Nets and Concurrency (Petri Nets).
Springer, 90–109.[5] Masoumeh Mansouri, Bruno Lacerda, Nick Hawes, and Federico Pecora. 2019.
Multi-robot planning under uncertain travel times and safety constraints. InProceedings of the 28th International Joint Conference on Artificial Intelligence(IJCAI). 478–484.
[6] Tadao Murata. 1989. Petri nets: Properties, analysis and applications. Proc. IEEE77, 4 (1989), 541–580.
[7] Federico Pecora, Henrik Andreasson, Masoumeh Mansouri, and Vilian Petkov.2018. A Loosely-Coupled Approach for Multi-Robot Coordination, Motion Plan-ning and Control. In In Proceedings of the 28th International Conference on Auto-mated Planning and Scheduling (ICAPS).
[8] Martin L Puterman. 2014. Markov Decision Processes.: Discrete Stochastic DynamicProgramming. John Wiley & Sons.
Doctoral Consortium AAMAS 2020, May 9–13, Auckland, New Zealand
2170