SESAR2020 Technology Evaluation/Performance Assessment

European Technology Evaluation Workshop

José Manuel Cordero on behalf of David Batchelor

Brussels, 17th October 2018

SESAR: The SES ATM research programme Technological pillar of SES

Mechanism coordinating and concentrating all EU research and development (R&D) activities in ATM

Unites around 3000 experts in Europe

SJU set up in 2007 to manage this partnership – single-managing entity

ATM is an essential part of European air transport and aviation, connecting cities and people citizens as well as boosting jobs and growth

SESAR timeline: SESAR 1 (2007-2016), SESAR 2020 Wave 1 (2016-2019), Wave 2(2020-2022)

SESAR: A performance-driven programme

How to achieve these benefits?

SESAR role: defining, developing & deploying new or improved technologies and procedures (SESAR Solutions)

SESAR Vision:

• Builds on trajectory-based operations (TBO)

• Relies on the provision of Air Navigation Services (ANS) supporting this Business/Mission Trajectory execution

The ATM Master Plan

The European ATM Master Plan is the common roadmap for the Development and Deployment of SESAR technologies and procedure, linking them to the Single European Sky Performance Objectives

• Enable a 3-fold increase in capacity which will also reduce delays both on the ground and in the air

• Improve safety by a factor of 10 • Enable a 10% reduction in the effects

flights have on the environment • Provide ATM services to the airspace

users at a cost of at least 50% less

ATM MP Performance Ambitions

6

How can be known that a SESAR solution will bring the expected benefits?

How can be justified that a SESAR Solution is ready for deployment and industrialisation?

How can be known if it is necessary to plan additional assessments or propose new concepts?

PJ19.04 - Performance Assessment Overview WHY do we need Performance Assessment?

7

PJ20

PJ19

SESAR Solution & Enabling Projects

SESAR2020 - main principles

Performance Areas in SESAR 2020

6 transversal areas to be assessed:

(Operational) Performance

Environment

Human Performance

Safety

Security

CBA

8

Reference material,

methodologies, etc…

available for them (output

from SESAR 1)

Support & Guidance through

Community of Practice

(with additional domains)

Performance Workflow in SESAR2020

Progressive Performance Assessment refinement

Validation

Results Estimations

SOL SOL SOL SOL SOL

SOL SOL

SOL SOL

SOL

SOL

Performance

Assessment

Release #1

Performance

Assessment

Release #2

Performance

Assessment

Release #n

VALIDATION TARGETS

Gap Analysis Gap Analysis

Gap Analysis

3 yearly cycles for SESAR 2020 Wave 1: 2017, 2018 and 2019

PERFORMANCE

FRAMEWORK

The Performance Consolidation Factory

Assumptions

Applicability

Traffic

SOL ZZ PAR

SOL XX

PAR

SOL YY

PAR

SESAR 2020 Consolidated Performance Assessment

SESAR Solution XX

SESAR Solution YY

Exercises

Consolidated PAR

v1, v2, v3 exercises

Per Transversal

Area!

Assumptio

ns

Applicability

Traffic

1st level of Consolidation

performed at SOL level per v

phase

2nd level of consolidation,

at Program level,

performed by PJ19.04

SESAR 1 performance results 2005 baseline

11

Not including Release 5 results

SESAR Performance Framework Performance-driven development approach that is applied within the SESAR Programme.

It represents a framework to support the goal of ensuring that the SESAR Programme develops the operational concept and technology

Common reference for the SESAR projects measurements

Ensures that the overall activity of the SESAR projects is coherent and, in addition, that these projects´ results are meaningful, consistent and usable.

Composed of indicators, methodologies and tools

SESAR Performance Framework

SESAR performance framework

(+maintenance)

SES performance

scheme + maintenance

Exploratory Research

results

WAVE 2 Program

definition

Master Plan + update

SESAR experience

SJU Input

Scientific Committee

Input

Indicator evaluation

SESAR Performance Framework KPIs Selected KPA Focus Area KPI KPI Definition

Safety ATM system safety outcome SAF1 Total number of fatal accidents and incidents with ATM

Contribution per year

Environment Fuel Efficiency FEFF1 Actual average fuel burn per flight

Capacity Airspace Capacity

CAP1 TMA throughput, in challenging airspace, per unit time

CAP2 En-Route throughput, in challenging airspace, per unit

time

Airport Capacity CAP3 Peak Runway Throughput (mixed mode)

Predictability and

Punctuality

G2G variability in flight duration

PRD1 Variance of difference in actual and flight plan or RBT

durations

Departure punctuality PUN1 % Flights departing within +/-3 minutes of scheduled

departure time due to ATM and weather related delay causes

Cost Efficiency

G2G ANS Cost Efficiency CEF2 Flights per ATCO-Hour on duty

CEF3 Technology Cost per flight

Security, Flexibility, Human Performance, Civil-Military Coordination, Access and Equity do not have any KPI or Validation Target.

SESAR Performance Framework PIs Selected (overview)

KPA Number of PIs Which mandatory

Safety 7 1

Security 2 2

Environment 6 2

Capacity 10 7

Predictability and Punctuality - -

Cost Efficiency 4 1

Flexibility - -

Civil-Military Cooperation 5 5 (only where relevant)

Human Performance 4 4

Access and Equity - -

17

SESAR Performance Framework Methodologies (Environment)

SESAR Performance Framework Methodologies (Environment)

5 steps approach

SESAR Performance Framework Example of tools (Environment)

The following tools are also recommended for specific purposes:

The Eurocontrol IMPACT tool: implements state-of-the-art methodology for calculating noise footprints, while providing estimations of fuel consumption and emissions released using a common set of aircraft fleet composition and performance assumptions. Generalised use in S2020

Open-ALAQS - for detailed local air quality assessments that might or might not include sources of emissions other than flight movements.

AEM - for fuel and emissions assessments, which would require large amounts of data to be processed, otherwise, it is preferable to use IMPACT.

V-PAT - for vertical flight efficiency analysis and in particular Continuous Climb and Continuous Descent operations.

Conclussions

• SESAR is a performance-driven programme where technology is evaluated mostly from an ATM performance perspective

• A common Performance Framework is provided to all solutions to ensure program-wide coherency and ensure usability of results for further consolidation

• Performance Framework consist on indicators, methodologies and tools

• The current Performance Framework is stable – maintenance to adapt to changing context is performed

• Yearly lifecycle - checkpoints

• Gap Analysis is an essential tool for technology evaluation at program level

• Support to Solutions to ensure applicability is deemed essential

This project has received funding from the SESAR Joint Undertaking under the European Union’s Horizon 2020 research and innovation programme under grant agreement No [number]

The opinions expressed herein reflect the author’s view only. Under no circumstances shall the SESAR Joint Undertaking be responsible for any use that may be made of the information contained herein.

PJ19.4 Coordination with PJ20

Thank you very much for your attention!

What will SESAR deliver in terms of Environment?

SESAR Fuel Saving Ambition (10% per flight)=

SESAR2020 target for solutions (6.8%) + SESAR 1 performance (1.8%) + Deployment Baseline performance (1.4%)

As it is estimated that 50.0 millions of tonnes of fuel will be burned by 2035 ECAC wide by around 10 million flights, the above-mentioned fuel saving targets and performances are equivalent to: SESAR2020 Fuel Saving Ambition (10%) = 500kg/flight or around 1.5 tonnes of CO2/flight SESAR2020 Fuel Saving target for Solutions (6.8%) = 340kg/flight or 1 tonnes of CO2/flight SESAR 1 Fuel Saving performance (1.8%) = 90kg/flight or 283kg of CO2/flight SESAR Deployment Baseline Fuel Saving performance (1.4%) = 70kg/flight or 220kg of CO2/flight Note: while the SESAR 1 baseline was 2005, the SESAR2020 baseline is 2012.

What will SESAR deliver in terms of Environment? 30 out of 85 solutions have expected benefits in Performance.

Top 10:

Solution PJ.07-01 Airspace User Processes for Trajectory Definition PJ.10-01C Collaborative Control PJ.10-02b Advanced Separation Management PJ.09-03 Collaborative Network Management Functions PJ.01-02 Use of Arrival and Departure Management Information for Traffic Optimisation within the TMA PJ2-01 Wake turbulence separation optimization PJ.09-02 Integrated local DCB processes PJ.01-03 Dynamic and Enhanced Routes and Airspace PJ.02-08 Traffic optimisation on single and multiple runway airports PJ.08-01 Management of Dynamic Airspace configurations

SESAR 1 gains for Environment / Fuel efficiency

EUROCONTROL - General Presentation 24

DB (Deploymentbaseline)

DB+STEP1DB+STEP1

+STEP2DB+STEP1+

STEP2+STEP3

Target -0.30% -2.80% -5.65% -8.50%

Cycle 5assessment

-0.30% -2.36%

-9%

-8%

-7%

-6%

-5%

-4%

-3%

-2%

-1%

0%

To

tal E

CA

C F

ue

l sa

vin

gs

Target Cycle 5 assessment

Gap Analysis

SESAR Performance Framework SESAR Performance Management Process

SJU ATM MP Committee

Stakeholders business needs

External forces + EU Performance Scheme

Content Integration Level (PJ19)

Solution Development & Validation Level -

(PJ1 to PJ18)

Master Planning Level (PJ20)

Deployment packaging /

planning

Performance-driven CONOPS (Inc Perf

Expectations) and VALS

2

Consolidated Performance Assessment

5

SESAR Solutions Development and Validation

4

Validation Targets

3

Performance needs

Performance ambitions

1

Consolidated Business Case

6

Framework

Content Integration

• Support decision

making • Reporting

information about proposed solutions

• Analyzing the impact • Identifying gaps and

overloads • Support concept

development and system design

Performance Ambitions

Set on: • Cost efficiency • Operational

Efficiency • Capacity • Environment • Safety • Security

Performance Expectations

• Estimation of

expected achieved performance outcomes

• Concept for evolution of ATM phases

• Initial expert judgment of the magnitude of potential benefits for each KPI

Validation Targets

• Overall

contributions that solutions should target for the programme

• Are aligned with the performance ambitions

• Two types: • Numerical • Qualitative

SESAR Solutions

• Two scenarios for

measuring Solution´s impact

• Reference Scenario (Without the element)

• Solutions Scenario (Element is integrated)

Performance Assessment

• Reported at

Performance Assessment and Gap Analysis Report (PAGAR)

• Covers every Transversal Areas and KPAs

• PAGAR will be use by PJ20

Consolidated Business Case

• Developed by PJ20 • Consolidating

assessments result • Most appropriate

way to support decision making

• Focus on develop-deploy transition

27

A bottom-up aggregation of what is actually being validated iteratively (from V1 to V3 maturity phases)

Consolidated Performance Assessment

Content Integration Level

Solution Validation Results

Content Development and Validation Level

Exercise Validation Result Content Development and

Validation Level

Exercise Validation Result Content Development and

Validation Level

Solution Validation Results

Content Development and Validation Level

Exercise Validation Result Content Development and

Validation Level

Exercise Validation Result Content Development and

Validation Level

Exercise Validation Results Content Development and

Validation Level

Assumptions

Applicabili

ty

Traffic

Assumptions

Applicabili

ty

Traffic

PJ19.04 - Performance Assessment WHAT is a Performance Assessment?

SESAR2020 Performance Framework Relation to Performance Ambitions (1/2)

SESAR2020 Performance Framework Introduction Session

ATM Master Plan SESAR

Performance Ambition KPA

Perf Amb. ID ATM Master Plan SESAR

Performance Ambition KPI

SESAR Performance Framework

KPA

Perf Fram. ID Performance Framework KPI/(PI)/<Design goal>

Cost efficiency

PA1 Gate to Gate direct ANS Cost per Flight

Cost efficiency

CEF1 Direct ANS Gate-to-gate cost per flight

PA1.1 Determined Unit Cost for En-Route ANS

CEF2 Flights per ATCO hour on duty

CEF3 Technology Cost per flight

PA1.2 Determined Unit Cost for Terminal ANS

CEF2 Flights per ATCO hour on duty

CEF3 Technology Cost per flight

Capacity

PA7 Network throughput additional flights (million)

Capacity

CAP1 TMA throughput, in challenging airspace, per unit time

CAP2 En-route throughput, in challenging airspace, per unit time

PA6 Additional flights at congested airports (million)

CAP3 Peak Runway Throughput

<RES1> % Loss of airport capacity avoided

<RES2> % Loss of airspace capacity avoided

PA5 Departure Delay (min/dep) Predictability

and punctuality

PUN1 % departures within +/- 3 minutes vs. Scheduled due to ATM and weather related delay causes

SESAR2020 Performance Framework Relation to Performance Ambitions (2/2)

SESAR2020 Performance Framework Introduction Session

ATM Master Plan SESAR

Performance Ambition KPA

Perf Amb. ID ATM Master Plan SESAR

Performance Ambition KPI

SESAR Performance Framework

KPA

Perf Fram. ID Performance Framework KPI/(PI)/<Design goal>

Operational Efficiency

PA4 Arrival Predictability Predictability

and punctuality

PRD1 Variance of Difference in actual & Flight Plan or RBT durations

PA2 Flight time per flight (min/flight)

Environment

FEFF3 Reduction in average flight duration

PA3 Fuel burn per flight (tonne/flight) FEFF1 Actual average fuel burn per flight

Environment PA8 Co2 emissions (tonne/flights) FEFF2 CO2 Emissions

Safety PA9 Accidents with ATM contribution Safety <SAF1> Total number of fatal accidents and incidents

Security PA10 ATM related security incidents resulting in traffic disruptions

Security SEC1

Personnel (safety) risk after mitigation

SEC2 Capacity risk after mitigation

SESAR2020 Performance Framework Relation to Performance Scheme

SESAR2020 Performance Framework Introduction Session

• SESAR PF and SES Performance Scheme have different scope and applicability

• Both documents use similar KPAs, but selected KPIs are different for

each point of view.

SES Performance Scheme SESAR 2020 Performance Framework

Use of targets (KPIs) and monitoring (PIs) to drive performance improvement in the

deployed system

Used to measure expected improvements from deploying SESAR Solutions

Implicitly includes real world issues (such as indirect ANS costs)

Explicitly excludes real world issues

Measurements are made in the real world Measurements are made in validation

exercises

SESAR2020 Performance Framework Relation to Performance Scheme (1/2)

SESAR2020 Performance Framework Introduction Session

SES Perf Scheme KPA

Perf. Scheme ID

SES Performance Scheme KPI/PI SESAR PF

KPA Perf.

Fram. ID PF KPI/(PI)

Safety

PS1 Effectiveness of Safety Management (both Regulators and ANSPs).

Safety

N/A

PS2 Application of RAT methodology. N/A

PS3 Application of Just Culture (JC) N/A

PS4 Separation infringements

<SAF1> Total number of

fatal accidents and incidents

(SAF1.3)

(SAF2.3) PS5 Runway incursions

(SAF3.4)

PS6 ATM-specific occurrences at ATS units N/A

PS7 Airspace infringements N/A

PS8 Level of occurrence reporting N/A

PS9 Application of automatic data recording for separation minima

infringement monitoring N/A

PS10 Application of automatic data recording for runway incursion

monitoring N/A

Cost-Efficiency

PS22 Determined Unit Cost for En-Route ANS

Cost-Efficiency

CEF2 Flights per ATCO hour on duty (CEF2)

CEF3 Technology Cost per flight (CEF3)

PS23 Determined Unit Cost for terminal ANS CEF2 Flights per ATCO hour on duty (CEF2)

CEF3 Technology Cost per flight (CEF3)

PS24 Costs of EUROCONTROL N/A

SESAR2020 Performance Framework Relation to Performance Scheme (2/2)

SESAR2020 Performance Framework Introduction Session

SES Perf Scheme KPA

Perf. Scheme ID

SES Performance Scheme KPI/PI SESAR PF KPA Perf.

Fram. ID PF KPI/(PI)

Environment

PS11 Horizontal en route flight efficiency of the last filed

flight plan trajectory. [%] Environment

FEFF1.1 Planned Average fuel burn per flight

PS12 Horizontal en route flight efficiency of the actual

trajectory [%] FEFF1 Actual average fuel burn per flight

PS13 Effectiveness of booking procedures for FUA

Civil-Military Cooperation

and Coordination

CMC1.7 Used duration/Allocated duration ARES

PS14 Rate of planning of CDRs CMC2.2 GAT planning efficiency of Available ARES (%GAT

flights planning to use ARES vs. GAT flights for which ARES is available)

PS15 Effective use of CDRs PRD7 No. flights GAT for which ARES airspace is

offered

PS16 Additional time in taxi-out phase [minutes per

departure]

Punctuality and

Predictability

PRD3 Taxi out variability

PS17 Additional time in arrival sequencing and metering area

(ASMA) [minutes per arrival] PRD4 TMA arrival variability

Capacity

PS18 En-Route ATFM delay per flight [min per flight].

PUN1 % Departures < +/- 3 mins. vs. schedule due to

ATM causes and weather conditions

Capacity CAP2

En-route throughput, in challenging airspace, per unit time

PS19 Airport ATFM arrival delay per flight [min/arr]. CAP3 Peak Runway Throughput

PS20 ATFM slot adherence Punctuality

and Predictability

PUN1 % Departures < +/- 3 mins. vs. schedule due to

ATM causes and weather conditions

PS21 ATC pre-departure delay PUN3 % Departures < +/- 3 mins vs. schedule due to

Airport ATM Factors