Infusion of Advanced Planning and Scheduling Technology in Space

ESA Achievements and Perspectives

Alessandro Donati, Nicola Policella (OPS-HSC)Colin Haddow (OPS-GI)Erhard Rabenau (OPS-OPM)

OPS-Forum, ESA/ESOC19/03/2010

OUTLINE

– Introduction (A. Donati)

– Motivation

– A.I. Planning and Scheduling (N. Policella)

– Technology

– Experience & Evaluation

– Transfer to Infrastructure (C. Haddow)

– MPS Framework

– Conclusions (A. Donati)

– On-going project

– Future work and Lessons Learnt

Introduction

Dwight D. Eisenhower

Plans are nothing; planning is everything.

Observe always that everything is the result of change, and get used to thinking that there is nothing Nature loves so well as to change existing forms and make new ones of them.

- Marcus Aurelius, emperor of Rome (121-180 AD)

What are we talking about today

Generate a planExecute a planRepair a plan

How do it “better” ?

InputInput Generate or repair a

PLAN

Generate or repair a

PLANExecute

+

“better” stands for: -More Robust- Optimal- Automated - Conflict free

Elements of a Planning System

Solver

ProblemTo Solve

Domain

Environment

Domain Description Language

Domain Description Language

Problem Description Language

Problem Description Language

Algorithms

Motivation for Technology Infusion

– Challenging Operations Scenarios

– Planning and Scheduling, a process to consolidate

– Adequately matured techniques ready to be exploited

Operations Pull

Technology Push

AI P&S technology supporting Mission Control, Ground Stations, and On- board P&S processes

Planning & Scheduling : a process to consolidate

– Independent Tools for Mission Specific Long Term/Medium T/Short T P&S

– Science Planning

– Platform Operations Planning

– Labor Intensive Ground Planning Tasks

– Automatic Conflict Detection but Manual Conflict Resolution

– Limited On-board Conditional Execution Plan

Multimission Planning & Scheduling Infrastructure

Intelligent Solver A

Intelligent Solver B

Intelligent Solver C

Planning & Scheduling : a possible future scenario

Catalyst : IWPSS 04Flying Mission Use Case

Mars Expresson-board

memory dump problem

Bridging the gap

Enhanced Enhanced P&SP&S

ConceptsConcepts

EnablingEnablingTechnologyTechnology

PrePre-- OperationalOperationalPrototypingPrototyping

ExtendedExtendedOperationalOperationalValidationValidation

AssessmentAssessment

– Sponsorship, Chicken & Eggs

– Current Missions: Test Beds for the Future

Modeling

Solving

MEXAR 2

RAXEM

A.I. Planning and Scheduling

Definitions

– Planning : to devise or project the realization or achievement of a purpose

– Automated planning and scheduling is a branch of artificial intelligence (AI) that concerns the automated realisation of strategies or action sequences

– Usually there are 3 kinds of input : A domain (e.g., a set of possible actions that the planner can take); an initial state of the world, and the desired goals

– OPS vs A.I. : different meanings

FACD0013|0|5|2|1|A|0|FACD0014|0|5|2|1|A|0|FACD0015|0|5|2|1|A|0|FACD0016|0|5|2|1|A|0|FACD0017|0|5|2|1|B|0|FACD0018|0|5|2|1|B|0|

A.I.

OPS

Planning Scheduling

Time

A.I. P&S Technology

– AI Model-Based Approach– Reusability– Flexibility and Adaptability

– Timeline-Based Modeling– based on modeling core focuses on both temporal evolution

of key components and the ability to capture relevant domain constraints

– Capability of Scheduling Problem Representation and Solution– Relevant to real-world applications

– Space mission operations (HSTS, EUROPA, ASPEN)– Robotics (IxTeT, IDEA, T-REX)– Manufacturing

– Integration of human strategic capabilities and automatic problem solving algorithms

– User Interaction - which supports different levels of interaction with a user

AI Model-Based Approach

AI Model-BasedSolver

Problem

Output

– Problem Solving

– Flexibility

– Scalability

– Adaptability (Unforeseen Events,

Damages,…)

– System Design

– Rapid Prototyping

– Portability

– Reusability

DomainDomain 2Domain 1

Problem 1Problem 2

Output 1Output 2

– Modelling– Focus on key features– Describe their possible

consistent temporal behaviours

– Represent the relevant constraints (domain theory)

– Represent the management policy (control laws)

– Solving– Synthesize timelines

according to current goals satisfying modelled constraints and management policy

Modeling

Solving

Problem Solving = Timeline synthesis

Timeline-Based Modeling

Timeline-Based Modeling Methodology (1)

Slewing(st1,…)Slewing(…

,st12)Unlocked(st1)

tLocked(st1)

34

0

t

t

SwitchOn SwitchOff

1) Choose Components of the Domain

1) Choose Components of the Domain

2) Model how the Components behave

t

OffOn On

[lb,ub]

Durat ion

O

max

ResourceMax Availability

Timeline-Based Modeling Methodology (2)

On

OffBurned

1) Choose Components of the Domain

2) Model how the Components behave

3) Put them together and model the interactions

Panoramic Camera

Mobility System

Communication System

Drive(rk1,rk2)At(rk1) At(rk2)

Place(rk2)Unstow StowStowed TakeImage(rk2)

CoolDownTrackingTakePicture(rk1) Heat TakePict(rk2)

Transmit Off Transmit

Microscopic Imager

Timeline-Based Modeling Methodology (3)

Why Timeline Planning ?

Classical, Activity Planning

– State-transition system

– Pre-conditions

– Post-conditions

– Produces a sequence of actions that lead from an initial state to a state which meets the desired goals

Timeline-based planning

– Temporal reasoning

– Handling concurrency

– Action synchronization

– Explicit resource management

– The approach based on timeline synthesis has root in solid work in the space domain

– RAX-PS/EUROPA [Jonsson et al., 2000], ASPEN [Chien et al., 2000]

Causal Reasoning

Resource and Time Allocation

Current experience and results

Mexar2Raxem

SKeyP

APSI

MrSpock

AIMS

Xmas

2001 2009

From scratch

Framework based

Mexar2

– The problem: generation of spacecraft operations for efficient on-board mass memory dumping for MEX

– The downlink activities were synthesized manually by a team of people continuously dedicated to this task

– Several constraints & requirements: limited on-board memory, limited communication capability, avoid data overwriting

Payloads

Spacecraft

TM (Science + Housekeeping)

TM Router

Science C

Science B

Science A

Housekeepin g

Communication Channel

Limited capacity

Limitedbandwidth

Non visibilitywindows

Earth

Mexar2 Technical features and performance

– Software design

– Object-oriented

– Two modules:

– Problem Solver (PS)

– Man-Machine Interface (MMI)

– Implemented in Java

– Multiplatform: works under UNIX, Windows, Mac OSX

– Interactive problem style allowing what-if analysis

– MEXAR2 is a configurable tool (e.g., adding a new packet store)

– Efficient solving algorithms (e.g., a dump plan over a period of 30 days is computed within 1 minute of computation)

– MEXAR2 has reduced by 50% the time needed to generate dump plans

– Produces plans of higher quality without data loss (robustness)

– Allows to spot in advance resource bottlenecks (increased science return)

SKeyP SOHO Keyhole Planner

– The problem: to generate plans for SOHO Keyhole periods operations

– Keyhole period: The HGA pointing capability, recorder dumping capabilities (possible only with DSN 34/70 m antennas) and recorder capacities are not sufficient to downlink all data,

– selection and prioritization

– Plan :

– What to store in the on- board memory

– Data Downloading Activities

Requirements & Goals

– satisfy the different constraints (e.g., finite recorder capacity, DSN antenna limitations, robustness)

– flexibility in recorder usage, switching commands timings, etc.

– allow exploration of options

– reduce planner’s mechanical and repetitive tasks (and time) needed to produce a baseline solution

– reduce dependence on planner experience

– Integration with the current workflow

SKeyP Achievements

– SKeyP solves the problem and reduces the working time

– It produces a plan in under 10 seconds

– Rapid what-if analysis, parameter set comparisons

– Manual fine-tuning of solutions

– Better understanding of algorithm’s behaviour

– SKeyP permits a fast handover between operational users

– It has been easily integrated with the current workflow

– Different guidelines contributed to the current result

– Users (mission planners) integrated in the development team

– Spiral iterative prototyping & validation cycles

– Solved problems in compatible time constants

APSI Advanced Planning & Scheduling Initiative

– experimental software framework

– operational validation of new AI P&S concepts & algorithms

– open, plug in architecture

– reusable, scalable

– coherent with the EGOS Mission Planning Framework approach

Modeling

Solving

APSI Current Implementation

User Interaction ServicesSoftware Interfaces

Problem SolverSoftware Interfaces

DomainDescriptionLanguage(DDL.3)

DomainLayer

ComponentLayer

Time &Parameters

Layer

DomainManager

Decision Network(current plan)

Component1 Component2

Time & Parameters NetworkTRF

APSI Project Outcome

Spec

ific

Appl

icat

ion

Prob

lem

End

Use

rs fo

r Sp

ecifi

c Ap

plic

atio

n

APSIFRAMEWORK

SpecializedProblem Solving

User Interaction

Know

ledg

e En

gine

erin

gfo

r App

licat

ion

Sup

port MrSPOCK

Mars ExpressLong Term

Planning

APSI-TRFTimeline Representation Framework

XMAS

XMM-NewtonAdv MissionScheduler

AIMS

APSI IntegralMission

Scheduler

MrSpock MEX Science Planning Opportunities Coordination Kit

– Problem: to generate a pre- optimized skeleton plan for Mars Express Long Term Planning

– Integration of:

– Ground station availability

– Uplink activities

– Spacecraft maintenance

– Downlink activities

– Science at pericentres

Aims:

– Minimize the iterations between Science Team and Mission Planning Team, taking into account a very detailed scenario and several co- existing constraints

– Provide the ability to explore the solution space according to different optimization functions

– maximize planned science

– maximize total UpLink/DownLink (UL/DL) time

PI

Science Team Science Team Mission Planning

Team

Mission Planning Team

Payload request

Payload request

Plan refinements

LTPLong Term

Plan

MTP Medium Term

Plan

MTPMedium Term

Plan

STP Short Term

Plan

STPShort Term

Plan

PI

MrSpock Conclusions & Recommendations

– Multi-dimensional constraint / solution space using AI Genetic algorithm

– Successful iterative prototyping development

– Plan to use operationally for 2010

– Expected benefits:

– Improved use of uplink & downlink channels (+ 5% increase of traffic)

– New exploited opportunities: VMC/webcam@Mars

– Faster planning cycle (cost reduction)

– Benefits achieved with the use of the APSI framework:

– The application design time is shortened

– reduced distance between the domain and the application model

– Reduced coding time

AIMS APSI INTEGRAL Mission Scheduler

– The problem: to build and optimize a long-term observation plan (1 year) for INTEGRAL

– Standard constraints:

– obs. activities included in visibility windows

– no overlap for obs. Activities

– Special constraints:

– existence of special observations (periodic, spread, no splitting)

– existence of a maximum filling factor for each revolution

– maximum number of obs. activities per revolution

– Generally, not possible to make all obs. (over-constrained problem)

– Quality of a consistent plan depends on:

– the completion of observations

– the way each observation is realized

– the priority degree of observations

AIMS Achievements

– Scheduling is now automatic: much less physical labour intensive...!

– Provides various solutions: pick the best, save, compare, etc.

– Takes a coffee break to get a decent Long Term Plan

– Easy updates on past schedule info from operational database

– Operational scientists are happy

– Under validation for operational use

– Input, LTP-scheduling, output + monitoring status in 1 tool

– development of eAIMS

Benefits achieved by using APSI

– Tasks delegated to the APSI core framework:

– check all standard constraints

– extract precise start times for observation activities

– Tasks handled directly in AIMS:

– deal with special constraints

– optimization task

Advanced Planning & Scheduling : An added value for operations

Current space operation systems

– Identify, retrieve, and merge necessary information

– Propagation through rules definition

– Identify possible conflicts

Why adding advanced P&S on top?

– Problem solving functionalities

– Managing Resource Conflict

– Timeline model

– Optimization

– Science return

– Platform utilization

– Robustness & Flexibility of the solutions

– Integration of human strategic capabilities and automatic problem solving algorithms

– Decision support system

More science return, Reduced operations cost, Reduced resources utilisation

Transfer to Infrastructure

Mission Planning System Framework

– Objectives

– Provide support for the various types of missions supported by ESOC, e.g.

– Deep Space/Planetary

– Earth Observation

– Observatory

– Provide framework that can be used by mission as a basis for their planning system

– Provide standard format for inputs and outputs (Planning File ICD)

– Provide straightforward mechanisms for allowing for extension (e.g. A.I. algorithms integration)

MPSF will not be a generic planning system

Mission Planning Typical Workflow (Deep Space)

MPSF Architecture – Conceptual View

MPSF – Offline Planning

– Offline Planning

– defining the “building blocks” of a plan, e.g.

– rules and constraints that apply to the mission (e.g. instrument A cannot be active when instrument B is on, resource limits etc.),

– definition of “Plan fragments”, i.e. templates of pre- planned operations that can be used in building a plan.

– large part before launch, but continued evolution throughout mission due to

– instruments degradation

– revision of operational constraints

– mission objectives evolution

MPSF – Online Planning

– Online Planning

– Plan initialization

– External data ingestion

– Rules and constraints propagation

– Plan validation and adjustment

– Plan consolidation

– Scheduling

OCC (OPSLan)

EMSEMSMPS

EMSEMSMCS

Groundstation

TM and TC Data

FIDES

Files

Misc. Files

Monitoring and Control

Data

Radiometric data

Service Instance

Configuration Files (SICF)

G/S Schedules (GRSS)

G/S Schedules (GRSS)+ Service Instance

Configuration Files (SICF)

OCC (RelayLan)

TM, TC Data and files

Predicts and Radiometric data

FDS

OPS EMS

SFC SFS

STC

IFMS

TMTCS

External

EMSEMSExternalEntities

EMSEMSDDSFront end

EMSEMSLTA + DDS

EMSEMSNISEMSEMSSimSat

EMSEMSMATIS

EMSEMSG/S Sub-systems

SMF

TM and TC Data

TM and TC Data

TM and TC Data

Planning and Data Requests from External Entities + Data to External Entities

Planning and Data Requests from External

Entities + Data to External Entities

EMSEMSFARC

Services Provided by SMFFiles Transferred by GFTS

Files from/to File ArchiveOther data transfer mechanisms (e.g. TCP)

MPSF – Status

– Currently

– MPSF Architectural design completed.

– Rules engine not specified in detail

– MPSF development of Online functionality started

– Agile approach adopted – rapid feedback from end users

– Future

– Refinement of offline requirements and architectural design should be carried out

– Now possible to more closely integrate with MOIS than was originally foreseen

– Define requirements for rules engine (probably in the context of a mission development)

– Use 3rd party product, e.g. DROOLS ?

– Build on LMP used in VEX and EMS ?

Conclusions

Ongoing projects

– Autonomous Controller (TRP) (TEC-MMA) (reusing APSI)

– IRONCAP (TRP)

– Consolidation of Domain Description Language

– Formulation of Problem Description Language

Future Work

– Completion of APSI framework and associated DDL and PDL documented and available for ESA member states’ R&D and Industry

– Potential P&S Upcoming Applications (Algorithms) for

– ESTRACK and ESA’ Deep Space Network scheduling

– Science Operations

– GMES

– COL-CC Payload P&S

– ATV rendez-vous

– Contribution as building block for GOAL Based Operation scenario demonstration

Lessons Learnt

– Relationship with Stakeholders

– Mission Operations and Mission Managers

– Key Role of the Use Case Owner:

– Empowerment during the Development

– Ambassadorship during the Operational Assessment

– Ground Segment Infrastructure Managers

– Coordination

– Compatibility with Standards and Interfaces

Lessons Learnt

– Cost Benefit Analysis

– Results have to Justify the Effort Spent

– Use of a Framework Ease Reuse and Improves the ROI

– A consolidated efficient and effective modelling approach will further burst the ROI in introducing AI P&S in the space domain

Lessons Learnt

– Leverage on Scientific Community

– Open Reusable Framework allows easy sharing of use cases, benchmarks, algorithms and innovative technologies

– Timeline based planning and scheduling is becoming the current ESA reference approach for AI P&S

– Other solving approaches might well be considered and validated (e.g. mathematical programming)

Recognition & Acknowledgement

– ESA and ESOC has gained outstanding recognition for the Infusion of A.I. Planning & Scheduling technology in Mission Operations

– Best Application Award @ ICAP 2007

– NASA Recognition of MEXAR @ iSAIRAS 2008

– ESA Keynote Speech at IWPSS 09

– Technology Transfer to D/TEC

– All work so far thanks to full commitment and expertise of

– European Research Institutes (CNR-ISTC, ONERA, PoliMI)

– Industrial Partners

– OPS-O, OPS-G, OPS-HAS, OPS-HSC (initiator & in-house know-how)

– TEC-MMA, SRE-OA, SRE-PAT

Infusion of Advanced Planning and Scheduling Technology in Space

Alessandro Donati Alessandro.Donati@esa.int

Publications available at:http://opstools.esoc.esa.int/wiki/bin/view/Groups/OPS_HSC/PublicationsHSC

Time for Questions