advanced tools to assess and mitigate the criticality of ... · ref. d1.1 quality plan.docx...

Advanced Tools to assEss and mitigate the criticality of ICT compoNents and their

dependencies over Critical InfrAstructures

General information

Dissemination level PU

State Final

Work package WP1 Project Management

Tasks Task 1.3

Delivery date 01/07/2016

Version 1.0

H2020-DS-2015-1-Project 700581

Tools to assEss and mitigate the criticality of ICT compoNents and their


D1.1 - Quality Plan

Final

WP1 Project Management

Task 1.3

01/07/2016

Project 700581

Tools to assEss and mitigate the criticality of ICT compoNents and their


Ref. D1.1 Quality Plan.docx

Editors

Name

Nazzarena Barbaro, Paolo Pucci

Authors

Name

Serena Mazzoni, Paolo Pucci

Reviewers

Name

Matthieu Aubigny

Type H2020-DS-2015-1-Project 700581 Project Advanced Tools to assEss and mitigate the criticality of ICT compoNents

and their dependencies over Critical InfrAstructuresTitle D1.1 - Quality Plan

Classification PU

, Paolo Pucci

Organisatio

ITRUST

Advanced Tools to assEss and mitigate the criticality of ICT compoNents and their dependencies over Critical InfrAstructures

Page 2 of 22

Organisation

FNM

Organisation

FNM

Organisation Date

30/06/2016


Executive Summary

The current deliverable is provided to European Commission Innovation Action named “Advanced Tools to assEss and mitigate the criticality of ICT compoNents and their dependencies over Critical InfrAstructures” named with acronym ATENA).



Classification PU

is provided to European Commission to present the “Advanced Tools to assEss and mitigate the criticality of ICT compoNents over Critical InfrAstructures” - Grant Agreement Number 700581 (shortly

named with acronym ATENA).


Page 3 of 22

to present the Quality Plan of the “Advanced Tools to assEss and mitigate the criticality of ICT compoNents

Grant Agreement Number 700581 (shortly


Figure 1: Operational Structure

Table 1: Work Package List ................................Table 2: Milestone List ................................Table 3: Annotated schema forTable 4: KPI01 – Delivery to PCTable 5: KPI02 – Delivery to ECTable 6: KPI03 – Risk ManagementTable 7: KPI04, KPI05, KPI06, KPI07 Table 8: KPI08 - Website UnavailabilityTable 9: Audit Timetable ................................Table 10: Classification DocumentTable 11: Quality Records ................................

1 Introduction ................................1.1 Motivation ................................1.2 Objectives ................................1.3 Document Structure ................................1.4 Acronym and symbols ................................

2 Project Description ................................2.1 Work Breakdown Structure

2.1.1 Deliverables ................................2.1.2 Milestones ................................

3 Quality Objectives ................................3.1 Governance metrics ................................3.2 Service Metrics ................................3.3 Measurements ................................

4 Organization and Responsibilities4.1 Project Organization ................................4.2 Responsibilities ................................

5 Activities for the Quality Management System5.1 Quality Plan ................................5.2 Internal Project Reviews 5.3 Verification and Approval of Deliverables5.4 Analysis and Improvement

6 Documentation and Data Control6.1 Document Classification 6.2 Template ................................6.3 Standard and tools ................................6.4 Modification and Update 6.5 Review Procedures ................................

7 Quality Records ................................8 References ................................



Classification PU

Table of Contents

List of figures Operational Structure ................................................................................................

List of table ................................................................................................

................................................................................................Annotated schema for KPI description ................................................................

Delivery to PC ................................................................................................Delivery to EC ................................................................................................Risk Management ................................................................................................

KPI04, KPI05, KPI06, KPI07 - Communication effectiveness ................................Website Unavailability ................................................................

................................................................................................Classification Document ................................................................................................

................................................................................................

................................................................................................................................

................................................................................................................................

................................................................................................................................................................................................................................

................................................................................................

................................................................................................Work Breakdown Structure ................................................................................................

................................................................................................................................................................................................................................................................

................................................................................................................................................................................................

................................................................................................................................................................................................

Organization and Responsibilities................................................................................................................................................................................................

................................................................................................

Activities for the Quality Management System ................................................................................................................................................................................................

................................................................................................Verification and Approval of Deliverables ................................................................Analysis and Improvement ................................................................................................

Documentation and Data Control ................................................................................................ ................................................................................................

................................................................................................................................................................................................................................

................................................................................................................................................................................................

................................................................................................................................................................................................................................


Page 4 of 22

..................................................... 14

............................................................ 8 .................................................................... 9

........................................................... 10 ................................................... 11 ................................................... 11

............................................. 12 ........................................................ 12

....................................................................... 13 ............................................................... 17

................................................. 18 .............................................................. 21

.............................................. 5

.............................................. 5

.............................................. 5............................................................... 5

........................................................... 6

................................................................... 7.................................................... 7

..................................................... 8........................................................ 8

.................................................................. 10............................................................. 10

.................................................................... 12..................................................................... 13

......................................... 14............................................................ 14

.................................................................... 14

................................................... 16......................................... 16

...................................................... 17............................................................ 17

.................................................. 17

......................................... 18...................................................... 18

.............................................. 19............................................................... 19

...................................................... 19.............................................................. 19

...................................................................... 21................................................ 22


1 Introduction

1.1 Motivation

The current deliverable is provided to European Commission (in the following present the Quality Plan (QP) Agreement Number 700581.

This document collects the measures, roles and procedures for assessing the quality of project deliverables and the visibility and the impact of the project results, for the whole lifetimATENA project.

1.2 Objectives

The present QP is compliant to

• EN 9001:2008 [1] This QP is applicable to all the consortium activities relevant to the GA to Consortium Agreement [3].

This is a live document, that will be modified throughout the whole life of the ATENA project, in order to follow the evolution of the project.

Possible revisions of the present Assurance Manager), a quality expert in the Project Management Teamcould be due to modifications

• Consortium Quality System Procedures• Role and responsibilities within the project

Additional revisions of the present QPindicators are calculated, for example because of improvement of measurement tools and procedures.

When revisions involve only project IQP, they do not represent a reason to re

1.3 Document Structure

The document is made of several chapters,

• Chapter 1 is the present introduction• Chapter 2 sums up the key

chapters

• Chapter 3 describes the quality objectives, defining thquality of important aspects of the projects

• Chapter 4 sums up the organias stakeholders in the quality process

• Chapter 5 describes the activities of the Quality Management System in ATENA• Chapter 6 describes the rules for producing the expected deliverables, and the process to

be adopted when modifying, reviewing and validat



Classification PU

is provided to European Commission (in the following Quality Plan (QP) of the H2020 Innovation Action shortly

This document collects the measures, roles and procedures for assessing the quality of project deliverables and the visibility and the impact of the project results, for the whole lifetim

to the following Quality Assurance requirements:

This QP is applicable to all the consortium activities relevant to the GA [2]

This is a live document, that will be modified throughout the whole life of the ATENA project, in order to follow the evolution of the project.

present QP are under responsibility of the , a quality expert in the Project Management Team. The update

of any of:

Consortium Quality System Procedures

Role and responsibilities within the project

Additional revisions of the present QP may be due to changes to the way the key performance indicators are calculated, for example because of improvement of measurement tools and

olve only project Internal Procedures and are not relevant to the contentreason to re-issue the QP itself.

Document Structure

is made of several chapters, which respectively deals with:

is the present introduction

sums up the key points related to ATENA project, and needed for the following

bes the quality objectives, defining the KPIs to quantitatively measure the quality of important aspects of the projects

sums up the organisation of ATENA project, introducing some key roles involved as stakeholders in the quality process

describes the activities of the Quality Management System in ATENA

describes the rules for producing the expected deliverables, and the process to be adopted when modifying, reviewing and validating a deliverable in ATENA.


Page 5 of 22

is provided to European Commission (in the following referred as EC) to shortly named ATENA - Grant

This document collects the measures, roles and procedures for assessing the quality of project deliverables and the visibility and the impact of the project results, for the whole lifetime of the

the following Quality Assurance requirements:

[2] that are ruled according

This is a live document, that will be modified throughout the whole life of the ATENA project, in

of the project PAM (Program . The updates to the QP

may be due to changes to the way the key performance indicators are calculated, for example because of improvement of measurement tools and

not relevant to the content of the

points related to ATENA project, and needed for the following

e KPIs to quantitatively measure the

ation of ATENA project, introducing some key roles involved

describes the activities of the Quality Management System in ATENA

describes the rules for producing the expected deliverables, and the process to ing a deliverable in ATENA.


• Chapter 7 is a listing of • Chapter 8 contains the bibliographic references

1.4 Acronym and symbols

Acronym or symbols

Explanation

CA Consortium Agreement

CI Critical Infrastructure

EC European Commission

GA Grant Agreement

IACS Industrial and Automation Control

ICT Information & Communication Technology

KPI Key Process Indicator

PAM Program Assurance Manager

PC Project Coordinator

PM Project Manager

PMT Project Management Team

QA Quality Assurance

QAR Quality Audit Report

QP Quality Plan

SC Steering Committee

SCADA Supervisory Control and Data Acquisition

WBS Work Breakdown Structure

WP Work Package



Classification PU

is a listing of quality records established for ATENA.

contains the bibliographic references

Acronym and symbols

onsortium Agreement

Critical Infrastructure

European Commission

Grant Agreement

ustrial and Automation Control System

Communication Technology

Key Process Indicator

Program Assurance Manager

rdinator

Project Manager

Project Management Team

Quality Assurance

Audit Report

Steering Committee

Supervisory Control and Data Acquisition

akdown Structure

Work Package


Page 6 of 22


2 Project DescriptionNote: The project description (with the list of envisaged WPs, deliverables and milestones) is inserted for the sake of completenessto give a minimal context to the project, readers are invited to refer to the

Over recent years, Industrial and Automation Control Systems (IACSs) and Supervisory Control and Data Acquisition (SCADA) systems adopted in known as Critical Infrastructures, and referred in the following as due to the increasing number of interconnected distributed devices, sthe large amount of information exchanged among system components. With the emergency of such an “Internet of Things” generation of IACS, the boundaries to be protected have grown well beyond that of the single or aggregatedtime, new ICT paradigms provide interesting new features for flexibly and efficiently managing, monitoring and controlling devices and data traffic, but they may introduce new threats that potentially can impact CIs. The aim of ATENA project is to provide new tools and models capable of protecting the whole value chain of CIs, while preserving their efficient and flexible management.

ATENA will develop an innovative framework of tools and processes to incresilience of CIs, by combining new anomaly detection algorithms and risk assessment methodologies within a distributed cyberintegrated market-ready ICT networked components and advancedalgorithms for both correct static CI configuration and fast dynamic CI reaction in presence of adverse events.

More in detail, ATENA objectives are:

1. Develop a Unified Modelling Framework and to improve resilience across CIs against threats of their IACSs and infrastructures;

2. Develop methodologies and technologies for increasing autocontroled CIs;

3. Develop new anomaly detedistributed ICT-controled CI environment

4. Develop a suite of integrated marketreaction in the presence of adverse events in industrial distributed

5. Validate the ATENA models and tool suite in significant Use Cases.

The ATENA project has an overall length of 36 months.

2.1 Work Breakdown Structure

The WBS of the project is shortly

WP no. WP Title

WP1 Project Management

WP2

Resilience & Efficiency for flow prediction across CIs against adverse events on their IACS



Classification PU

scription The project description (with the list of envisaged WPs, deliverables and milestones)

is inserted for the sake of completeness and easy reference for following sectionto give a minimal context to the quality plan. For a detailed and official

to refer to the Grant Agreement of the Innovation Action

Over recent years, Industrial and Automation Control Systems (IACSs) and Supervisory Control quisition (SCADA) systems adopted in industry-related Essential Services (traditionally

known as Critical Infrastructures, and referred in the following as CIs)) have become more complex due to the increasing number of interconnected distributed devices, sensors and actuators and to the large amount of information exchanged among system components. With the emergency of such an “Internet of Things” generation of IACS, the boundaries to be protected have grown well beyond that of the single or aggregated-plant, typical of the mono-operator vision. At the same time, new ICT paradigms provide interesting new features for flexibly and efficiently managing, monitoring and controlling devices and data traffic, but they may introduce new threats that

n impact CIs. The aim of ATENA project is to provide new tools and models capable the whole value chain of CIs, while preserving their efficient and flexible management.

ATENA will develop an innovative framework of tools and processes to incresilience of CIs, by combining new anomaly detection algorithms and risk assessment methodologies within a distributed cyber-physical environment, and will provide a suite of

ready ICT networked components and advanced tools embedding innovative algorithms for both correct static CI configuration and fast dynamic CI reaction in presence of

More in detail, ATENA objectives are:

Unified Modelling Framework with ad-hoc models to control physical fimprove resilience across CIs against threats of their IACSs and

Develop methodologies and technologies for increasing auto-reconfiguring

Develop new anomaly detection algorithms and risk assessment methodologies within controled CI environment;

Develop a suite of integrated market-ready ICT networked components for detection and resence of adverse events in industrial distributed systems;

Validate the ATENA models and tool suite in significant Use Cases.

The ATENA project has an overall length of 36 months.

Work Breakdown Structure

shortly displayed in the following table:

Lead Beneficiary Start Month

Management FNM 1

Resilience & Efficiency models for flow prediction across CIs against adverse events on their ENEA 1


Page 7 of 22

The project description (with the list of envisaged WPs, deliverables and milestones) and easy reference for following section, in order

and official description of the Grant Agreement of the Innovation Action [2].

Over recent years, Industrial and Automation Control Systems (IACSs) and Supervisory Control related Essential Services (traditionally

have become more complex ensors and actuators and to

the large amount of information exchanged among system components. With the emergency of such an “Internet of Things” generation of IACS, the boundaries to be protected have grown well

operator vision. At the same time, new ICT paradigms provide interesting new features for flexibly and efficiently managing, monitoring and controlling devices and data traffic, but they may introduce new threats that

n impact CIs. The aim of ATENA project is to provide new tools and models capable the whole value chain of CIs, while preserving their efficient and flexible management.

ATENA will develop an innovative framework of tools and processes to increase security and resilience of CIs, by combining new anomaly detection algorithms and risk assessment

physical environment, and will provide a suite of tools embedding innovative

algorithms for both correct static CI configuration and fast dynamic CI reaction in presence of

hoc models to control physical flow efficiency improve resilience across CIs against threats of their IACSs and of their related ICT

reconfiguring capability of ICT-

ction algorithms and risk assessment methodologies within a

ready ICT networked components for detection and systems;

Start Month End Month

36

36


WP no. WP Title

WP3 IACS design for security

WP4 Distributed Awareness

WP5 Distributed Mitigation and Resiliency in interdependent scenario

WP6 Development and componentsIntegration

WP7 Validation and evaluation

WP8 Project disseminatcommercial strategy

2.1.1 Deliverables

A long list of deliverables with their deadline deliverables). Please refer to that list

2.1.2 Milestones

The list of the milestones is described in the milestones in [4]). Please refe

MS no. Milestone title

MS1

Quality, training, dissemination and communication plans ready

MS2 SoTA interim assessment

MS3 1st project review

MS4 Consolidated ATENA system requirements and specifications

MS5 2nd project review

MS6 1st release of the ATENA

software components



Classification PU

Lead Beneficiary Start Month

IACS design for security CRAT 1

Distributed Awareness UC 5

ributed Mitigation and in interdependent UNIROMA3 3

and components FNM 9

and evaluation IEC 3

issemination and commercial strategy ITRUST 1

Table 1: Work Package List

with their deadline is described in [4] (section 1.3.2 WT2 List of . Please refer to that list when evaluating the KPIs in section 3.1

milestones is described in the following table (see also sectioner to the original list in [4] for the verification c

WPs involved Who Due Month

Quality, training, dissemination and communication plans

WP1, WP8 FNM 3

SoTA interim assessment WP2, WP3, WP5 FNM 6

WP1, WP2, WP3, WP5, WP6, WP8

FNM 12

Consolidated ATENA system requirements and

WP1, WP2, WP3, WP4, WP5, WP7, WP8

FNM 20

WP3, WP4, WP5, WP6, WP7

FNM 24

release of the ATENA software components

WP3, WP4, WP5, WP6

FNM 30


Page 8 of 22

Start Month End Month

30

30

30

36

36

36

(section 1.3.2 WT2 List of 3.1.

section 1.3.4 WT4 List of criteria:

Month Delivery Documents

D1.1, D8.1

D2.1, D3.1, D5.1

D1.2, D1.3, D1.4, D2.2, D2.3, D2.4, D3.2, D5.2, D5.6, D6.1, D8.2, D8.3, D8.4, D8.5

D1.5, D1.6, D1.7, D2.5, D3.3, D3.4, D3.6, D4.1, D4.2, D4.3, D5.3, D5.4, D7.1, D8.6, D8.7, D8.8, D8.9, D8.10, D8.11

D3.5, D4.4, D4.5, D5.7, D6.2, D7.2, D7.3

D3.7, D3.8, D3.9, D4.6, D4.7, D4.8, D5.5, D5.8, D5.9,


MS no. Milestone title

MS7

1st release of the integrated tools suite, 2nd release of the software components

MS8 Final project review



Classification PU

WPs involved Who Due Month

release of the uite,

release of the software components

WP2, WP6 FNM 33

WP1, WP2, WP6, WP7, WP8

FNM 36

Table 2: Milestone List


Page 9 of 22

Month Delivery Documents D6.3

D2.6, D6.4, D6.5, D6.6

D1.8, D1.9, D1.10, D2.7, D2.8, D6.7, D6.8, D7.4, D7.5, D8.12, D8.13, D8.14, D8.15, D8.16, D8.17, D8.18


3 Quality ObjectivesAs regards quality objectives,

• Governance metrics: the indicators for effective governance & project management;• Service metrics: the indicators

Each quality objective will be identified by a KPI and defined byScope, Description, Class, Basic Time of measuring, Time of reportin

(Unique) ID - Name (a short title for the KPI)

Scope – (the aim and the scope of the KPI

Description – (a description of the KPI

Class

(Type of KPI, according to a defined classification; in this case class is Governance or Service)

Time of measuring: (when the basic data Formula: (formula to calculate the KPITime of reporting: (when the

Notes: (additional comments; e.g.,additional hints for calculation)

Table 3:

3.1 Governance metrics

The governance metrics directly measure the effectiveness in WP1. Indirectly, they also measure the effectiveness in all the other WPs, because all the WPs have to provide the PC with the envisaged deliverables within scheduled deadlines.

KPI01 - Delivery to PC

Scope Check the compliance withleaders and PCversion of the deliver

Description Average delay (the envisaged deliverable

Class Basic Data

Governance

DaySub:

DayDead:

NoDeliverables



Classification PU

Objectives , metrics have been identified and summarised in

: the indicators for effective governance & project management;

the indicators defined for the service provision.

will be identified by a KPI and defined by a set of elementsBasic Measurable Data, Unit of measure, Formula, reporting, Notes) according to the following annotated table

(a short title for the KPI)

(the aim and the scope of the KPI - purpose of the metrics)

of the KPI’s measurements)

Basic Data Unit

(Data element useful for the calculation of the KPI)

(Type of Measure for the calculated KPI)

(when the basic data elements are measured) (formula to calculate the KPI starting from the basic data elements(when the KPI is reported)

e.g., explanations on the decisions behind the KPI definition

Table 3: Annotated schema for KPI description

Governance metrics

The governance metrics directly measure the effectiveness in WP1. Indirectly, they also measure effectiveness in all the other WPs, because all the WPs have to provide the PC with the

envisaged deliverables within scheduled deadlines.

compliance with the deadlines of the internal schedulers and PC, establishing when each WP leader has to provide

version of the deliverables to the Project Coordinator (PC)

delay (number of working days late with respect to the schedule) ondeliverables to PC

Unit

Date of the email to submit the deliverable to PC (notification)

Scheduled deadline for delivery (notification)

s: Number of expected deliveries in reporting time periods M1-11, M12-18, M18-36

Working days


Page 10 of 22

ed in two classes:

: the indicators for effective governance & project management;

a set of elements (ID, Name, , Formula, Acceptance criteria,

annotated table schema:

Acceptance Criteria

of Measure for the calculated

(Measurable criteria for establishing if the KPI is accepted or not)

elements)

on the decisions behind the KPI definition or possibly

The governance metrics directly measure the effectiveness in WP1. Indirectly, they also measure effectiveness in all the other WPs, because all the WPs have to provide the PC with the

the internal schedule agreed among WP has to provide the reviewed/validated

the schedule) on delivering

Unit Acceptance Criteria

Working days

KPI01


Time of measuring: at delivery time

Formula: K

Time of reporting (month): M11, M18, M36

Notes: The average number of days is calculated for the prstarting from the data elements related to each single deliverablethe schedule do not compensatemaintain the average number of days per single WP

KPI02 - Delivery to EC

Scope Check compliance with Commission) in the Grant Agreement final version of each

Description Maximum delay (number of the envisaged deliverable

Class Basic Data

Governance

DaySub:

DayDead:

noDeliverables:

Time of measuring: on submissionFormula: KPI02Time of reporting (month): M11, M18, M36

Notes: The number of days is calculated the data elements related to each single deliverabledo not compensate for delay on the submission of other deliverables. No real interest to maintain the average number of days per single WP.

KPI03 - Risk Management

Scope Check the periodic revision of the risks (intended as risks and opportunities) associated to the ATENA project

Description Periodic (at least quarterly) Project Management Team

Class Basic Data

Governance

Events: distinct works on the implementationmitigation act



Classification PU

at delivery time

KPI01 = ∑ ��;�� M11, M18, M36 (in the planned reports for project review

The average number of days is calculated for the project as a whole in each reporting periods, starting from the data elements related to each single deliverable. Deliverables submitted in advance the schedule do not compensate for delay on the submission of other deliverables.

ge number of days per single WP.

Table 4: KPI01 – Delivery to PC

compliance with the deadlines of the schedule promised to EC in the Grant Agreement [2], establishing when the PC

each deliverable to the EC

elay (number of working days late with respect to the schedule) on delivering envisaged deliverable to EC

Unit

Date of submission of the deliverable to EC (on Participant Portal) Scheduled deadline for the delivery according to GA

number of expected deliveries in reporting time periods M1-11, M12-18, M18-36

Working days

submission of the deliverable KPI02 = max( DaySub – DayDead ; 0) M11, M18, M36 (in the planned reports for project review)

The number of days is calculated for the project as a whole in each reporting period, starting from the data elements related to each single deliverable. Deliverables submitted in advance

on the submission of other deliverables. No real interest to maintain the er of days per single WP.

Table 5: KPI02 – Delivery to EC

Check the periodic revision of the risks (intended as risks and opportunities) associated to the ATENA project

(at least quarterly) occurrence of risk management activities inside the ATENA Project Management Team

Unit

distinct times (sessions) when the PMT works on the revision of the critical mplementation risks and their related mitigation actions (occurring in a quarter)

Number of times


Page 11 of 22

for project review)

in each reporting periods, Deliverables submitted in advance of

on the submission of other deliverables. No real interest to

promised to EC (European e PC is asked to provide the

days late with respect to the schedule) on delivering

Acceptance Criteria

Working days KPI02 = 1


Time of measuring: at the end of the sessionFormula: KPI03Time of reporting (month): M11, M18, M36

Notes: The number of times is calcuwhen PMT worked on the risks during a past quarter

3.2 Service Metrics

The service metrics for ATENAbe measured according to KPIs that willpage 39 [5]:

KPI Means

KPI04 Project website

KPI05 Social media

KPI06 Press releases &newsletters

KPI07 Scientific publications

Table 7: KPI04, KPI05, KPI06, KPI07

The table above is inserted in a format that is as similar as possible to [5] in order to show the fact that promised metrics and acceptance criteria are kept unchanged wrt the GA.

The described KPI04, KPI05, KPI06, KPI07 are distinct occurrences of the relevant metric of interest. For this reason, hopefully the reader will forgive us if we decide to avoid content.

In addition, a KPI measuring the website (u

KPI08 - Website Unavailability

Scope – Check the availability of the ATENA public website

Description – How long (in hours) the website is not accessible from the Internet

Class Basic Data

Service

HoursDown: number of hours when the Project website is down

noHours: calendar hours in reporting time periods M1



Classification PU

at the end of the session of revision of the risks KPI03 = ∑ times the PMT was working on risks (during the quarter)M11, M18, M36 (in the planned reports for project review)

s is calculated in each reporting period, counting thewhen PMT worked on the risks during a past quarter.

Table 6: KPI03 – Risk Management

Metrics

TENA will measure the communication effectiveness in the WP8 and will according to KPIs that will be coherent with the values claim

Metric Poor

Number of Website views (by year)

10

Communication effectiveness

entioned Table 8 page 39 in order to show the fact that promised metrics and acceptance criteria are kept unchanged wrt

very simple to calculate, being simply the count of distinct occurrences of the relevant metric of interest. For this reason, hopefully the reader will

four distinct KPI description tables just to repeat the same

in the following table:

How long (in hours) the website is not accessible from the Internet

Acceptance Criteria

Percentage KPI08


Time of measuring: before M11, M18 and M36, with the periodicity allowed bmeasurement tools chosen by the web site administrator

Formula: KPI04 = HoursDown Time Reporting (month): M11, M18, M36

Notes: This KPI is measured since the day the web site is publiclin working hours, not in working days, in order to correctly cope with events when a very short lapse of service occurs in a day, while the website is fully working in the rest of the day. Measured unavailability may be due to a number of reasons, external (e.g., electric blackmaintenance). Whenever possible with automatic measurement tools chosen by the web site administrator, a differentiated measure will be reported

Table 8:

3.3 Measurements

The measurements of KPIs will be calculateddata made available by other project WP8 leader (Service Metrics)



Classification PU

before M11, M18 and M36, with the periodicity allowed bmeasurement tools chosen by the web site administratorKPI04 = HoursDown / noHours M11, M18, M36 (in the planned reports for project review)

This KPI is measured since the day the web site is publicly declared as ready (M3). It is calculated in working hours, not in working days, in order to correctly cope with events when a very short lapse of service occurs in a day, while the website is fully working in the rest of the day. Measured unavailability may be due to a number of reasons, external (e.g., electric black-out) or internal (e.g., extraordinary maintenance). Whenever possible with automatic measurement tools chosen by the web site administrator, a differentiated measure will be reported

Table 8: KPI08 - Website Unavailability

Measurements

ts of KPIs will be calculated by the PAM with the support of the relevant basic data made available by other project partners, in particular the PC (Governance Metrics) and

Metrics).


Page 13 of 22

before M11, M18 and M36, with the periodicity allowed by the automatic measurement tools chosen by the web site administrator

(in the planned reports for project review)

y declared as ready (M3). It is calculated in working hours, not in working days, in order to correctly cope with events when a very short lapse of service occurs in a day, while the website is fully working in the rest of the day. Measured unavailability

out) or internal (e.g., extraordinary maintenance). Whenever possible with automatic measurement tools chosen by the web site

with the support of the relevant basic the PC (Governance Metrics) and the


4 Organization and Responsi

4.1 Project Organization

The project organisation for ATENA is Management Structure, page 44)of completeness of the Quality Plan

4.2 Responsibilities

The organisation is composed by:

• A Governance Structure;• An Operative Structure.

The Governance Structure has in charge al

• Project Manager (PM): he/she is a representative of the the overall technical and administrative responsibility for the project and is the contact with the European Commission (EC)even if formally speaking this is not correct.



Classification PU

Organization and Responsibilities

Project Organization

ation for ATENA is thoroughly described in [5] (section, page 44). Here a graphical view of the organisation

of the Quality Plan:

Figure 1: Operational Structure

bilities

ation is composed by:

A Governance Structure;

An Operative Structure.

The Governance Structure has in charge all the management activities and

Project Manager (PM): he/she is a representative of the Project Coordinator (PC)the overall technical and administrative responsibility for the project and is the contact with the European Commission (EC). For the sake of simplicity, the PM is often named as PC, even if formally speaking this is not correct.


Page 14 of 22

(section B.3.2.2 Operational ation is shown for the sake

agement activities and is composed by:

Project Coordinator (PC) who has the overall technical and administrative responsibility for the project and is the contact with

r the sake of simplicity, the PM is often named as PC,


• Project Management Teamand comprising the following functions

o Administrative and Financial Managemento Quality Management o Risk Management o Innovation and IPR Managemento Dissemination (including standardio Communication o Exploitation o Technical & Scientific Cooo Data Control and Ethical Aspects

It is worth noting that, according to the present QP naming,ATENA is done by the PAM Quality of the ATENA project and is part of PMT.

• Steering Committee (SC): a collegial body, chairerepresentative from each partner of the consortium, overall progress of the project, milestones, and solve

The Operative Structure has in charge all the

• WP Leaders Team (WPL Team)WP Leader has the responsibility activity and the compliance with the envisaged dates

• WP Team (WP Team): team is responsible for the implementation of the WP’



Classification PU

Project Management Team (PMT): a team of people supporting the PM in the daily duties the following functions (sometimes appointed to the

Administrative and Financial Management

Innovation and IPR Management Dissemination (including standardisation)

Technical & Scientific Coordination (also representing WP Leaders) Data Control and Ethical Aspects

, according to the present QP naming, the Quality Management of the PAM (Program Assurance Manager) who is the responsible for the

Quality of the ATENA project and is part of PMT.

eering Committee (SC): a collegial body, chaired by the PM andrepresentative from each partner of the consortium, with the responsibility to oversee the overall progress of the project, verify the compliance of project advances with

possible consortium-level problems.

The Operative Structure has in charge all the technical WP activities and is composed

WP Leaders Team (WPL Team): a team composed of the seven technical WP leaders;responsibility for the proper management and execution of the WP

and the compliance with the envisaged dates;

WP Team (WP Team): a team composed by the partners involved in the specific WPteam is responsible for the implementation of the WP’s activities.


Page 15 of 22

supporting the PM in the daily duties the same person):

Leaders)

the Quality Management of ) who is the responsible for the

d by the PM and including one delegate the responsibility to oversee the

ompliance of project advances with the

WP activities and is composed by:

seven technical WP leaders; each agement and execution of the WP

composed by the partners involved in the specific WP; the


5 Activities for the QQuality Management implemented and keep under control all the items concerned with the quality of

The Quality Management is implemented through a

• Quality Planning: methodologies to ensure the environment and how to satisfthis activity is in the Quality Plan (QP) document;

• Quality Assurance: applying systematically quality to all activities of the project, to ensure requirements are met; preventing defects by recurand to define corrective measures;

• Quality Measurement and Control:whether they comply with standards, to prevent potential problemeliminate the causes of unsatisfactory performance by eliminating the roots of identified defects;

• Quality Analysis & Improvement:set the necessary actions to achieve results and the continuous processes improvem

5.1 Quality Plan

During the execution of ATENAmanagement, planning and development processes.

The audits consist in the examination of a representative sample of documentation and/or planned materials produced by these processes, with the following purposes:

• Verify that every deliverableadded to the list of planned deliverables because of EC’s request or PMT’s decision,and is subject to the established review and tests, by the competent offices in accordance with GA requirements and standards

• Check the effective reach of planned milestones;• Identify nonconformity

corrections or corrective actions, verifying thei

• Inform the PC about the quality status of the program.

The audits are documented in the Quality Audit Report (QAR).

The PAM keeps under continual control the auditof changes of the timing of the pro

The following is a tentative agreement with PAM and PC.

Audit No. Scope

#1 Verify Ma

#2 Verify Management Activity & Delivery Verify Activity of



Classification PU

Activities for the Quality Management Systemt implemented in ATENA is made up of activities performed to coordinate, lead

and keep under control all the items concerned with the quality of project and processes.

implemented through a set of activities:

identifying the quality objectives, the quality standards, the methodologies to ensure the measure and the monitoring of processes;environment and how to satisfy them; planning review and inspection. The description

in the Quality Plan (QP) document;

applying systematically quality to all activities of the project, to ensure requirements are met; preventing defects by recurring audits in order to evaluate quality and to define corrective measures;

Quality Measurement and Control: measuring and monitoring results to determine whether they comply with standards, to prevent potential problem

he causes of unsatisfactory performance by eliminating the roots of identified

Quality Analysis & Improvement: monitoring, measurement and analysis of processes,set the necessary actions to achieve results and the continuous processes improvem

ng the execution of ATENA, the PAM involved in the ATENA project performmanagement, planning and development processes.

The audits consist in the examination of a representative sample of documentation and/or materials produced by these processes, with the following purposes:

Verify that every deliverable (report or other kind of documentation), required by added to the list of planned deliverables because of EC’s request or PMT’s decision,and is subject to the established review and tests, by the competent offices in accordance

h GA requirements and standards;

Check the effective reach of planned milestones;

y and/or lacks of processes and delivery and start corrections or corrective actions, verifying their application and effectiveness;

Inform the PC about the quality status of the program.

The audits are documented in the Quality Audit Report (QAR).

PAM keeps under continual control the audit planning, making the necessary updates in case of changes of the timing of the project schedule.

tentative timing table of the audits, but possible revisions are .

Scope Month

Verify Management Activity & Delivery M7 MS1 and MS2)

Verify Management Activity & Delivery Verify Activity of critical WPs

M21 (after milestones MS3 and MS4)


Page 16 of 22

uality Management System is made up of activities performed to coordinate, lead

and processes.

identifying the quality objectives, the quality standards, the the monitoring of processes; creating the quality

review and inspection. The description of

applying systematically quality to all activities of the project, to ensure ring audits in order to evaluate quality

and monitoring results to determine whether they comply with standards, to prevent potential problems or once they occur to

he causes of unsatisfactory performance by eliminating the roots of identified

monitoring, measurement and analysis of processes, to set the necessary actions to achieve results and the continuous processes improvement.

performs internal audits on

The audits consist in the examination of a representative sample of documentation and/or other materials produced by these processes, with the following purposes:

documentation), required by the GA or added to the list of planned deliverables because of EC’s request or PMT’s decision, exists and is subject to the established review and tests, by the competent offices in accordance

and/or lacks of processes and delivery and start opportune r application and effectiveness;

planning, making the necessary updates in case

ossible revisions are defined in

Month

(after milestones MS1 and MS2)

M21 (after milestones MS3 and MS4)


#3 Verify Management Activity & DeliveryVerify Activity of WP

5.2 Internal Project

Recurrently and within the jointly agreed period of timethe PC organizes a project review where the Committee members are involved.

The agenda of this review is presystematically covered, namely: deliverables, KPI, risk analysis

5.3 Verification and

PAM checks and reviews any project deliverables before submission verification is reported in an internal note and sent to the Project Coordinator for the opportreview.

The Verification Process is described

5.4 Analysis and Improvement

The Consortium will identify, collect and analyze data to demonstrate the adequacy and effectiveness of quality management and to evaluacontinue the effectiveness of the quality management system.

In order:

• to maintain a precise technical controor remarks from EC, highlighted by technical bodies corpreports for tracking in a single database the state of technical problems and the activated actions for their resolution;

• KPI results will be recorded and analyzed to identify areas for improvement

These activities aim to continual improvement of the processes and, in general, of the factors which determine the success and its results.

The measures will be in input to the scheduled



Classification PU

Verify Management Activity & Delivery Verify Activity of WPs

M31MS5 and MS6

Table 9: Audit Timetable

Internal Project Reviews

ecurrently and within the jointly agreed period of time (after the quality audits mentioned in the PC organizes a project review where the Project Management Te

members are involved.

The agenda of this review is pre-established and ensures that all following domains are systematically covered, namely: contractual aspects, planning and progress achieved, s

sk analysis and assessment.

and Approval of Deliverables

PAM checks and reviews any project deliverables before submission to EC. The result of the is reported in an internal note and sent to the Project Coordinator for the opport

Process is described in section 6.5.

Analysis and Improvement

The Consortium will identify, collect and analyze data to demonstrate the adequacy and effectiveness of quality management and to evaluate where improvements can be made to continue the effectiveness of the quality management system.

to maintain a precise technical control on deliverables, any productsremarks from EC, highlighted by technical bodies corporate or EC, will be formalized on

for tracking in a single database the state of technical problems and the activated actions for their resolution;

will be recorded and analyzed to identify areas for improvement

continual improvement of the processes and, in general, of the factors which determine the success and its results.

The measures will be in input to the scheduled Internal Project Reviews (see section


Page 17 of 22

1 (after milestones MS5 and MS6)

after the quality audits mentioned in 5.1), ect Management Team and the Steering

established and ensures that all following domains are planning and progress achieved, status of

to EC. The result of the is reported in an internal note and sent to the Project Coordinator for the opportune

The Consortium will identify, collect and analyze data to demonstrate the adequacy and te where improvements can be made to

l on deliverables, any products or technical problems orate or EC, will be formalized on

for tracking in a single database the state of technical problems and the activated

will be recorded and analyzed to identify areas for improvement.

continual improvement of the processes and, in general, of the factors

(see section 5.2).


6 Documentation and Data ControlThe documents issued by the Consortivalidated by the:

• WP Leader (with the help of internal reviewers chosen among experts in the consortium)• Program Assurance Manager;• Project Coordinator.

After the conclusion of the verification/approval process, the document is submitted to EC.

The author of the documentdefined in this chapter.

The document modifications are described synthetically in the identifies the modified parts and the occurred changedefined as completely reviewed, indicating

6.1 Document Classification

Characteristic Rule

Code The code related to the deliverable (

File Naming Convention for internal documents

(intermediate versions)

_DocTitleYYYY

ACR: Reference of editor (

File Naming Convention for official deliverable (intermediate versions)

__ DocTitle: Title of the deliverable YYYY-MM-DD: Date of upload to the project

transmission (usually inside the Consortium by ACR: Reference of editor (short partner’s name)

___ DX.X: Code of the deliverable DocTitle: Title of the deliverable YYYYMMDD: Date of upload to the project

transmission (usually inside the Consortium by ACR: Reference of editor (short partner’s name)

Default version : M.N where: M: Major Version = 0,1,…. N : Minor Version = 1,2,…. New document as first version = 0.1

Draft: the document is incomplete. Final Draft: the version considered is finished by the author(s) and delivered

to the internal reviewer partners for final comments/remarks.Validation: the document is ready to be submitted for quality (the internal

reviewer partners have given all the comments and merged them into a ready-to-send version

Final: the document passed the quality checks and submitted to EC or to other recipient (AFTER CHANGING THE FILENAME, THE VERSION AND THE FORMAT IN ORCOMPLY WITH EC RULES AND/OR OPPORTUNITY REASONS)

Table 10: Classification Document


Page 18 of 22

um, in relation to the ATENA project, must be checked and

(with the help of internal reviewers chosen among experts in the consortium);

After the conclusion of the verification/approval process, the document is submitted to EC.

for checking that the document respects the rules

register. This description If the document is widely modified, it is

the review motivations.

2.1.1)

file server or date of usually inside the Consortium by email)

project file server or date of usually inside the Consortium by email)

finished by the author(s) and delivered partners for final comments/remarks.

ed for quality (the internal given all the comments and the editor

version). passed the quality checks and is ready to be

(AFTER CHANGING THE AND THE FORMAT IN ORDER TO

AND/OR OPPORTUNITY


6.2 Template

ATENA documents must follow the structure and physical layout agreed in the templates defias a joint work of WP1 and WP8project web site (https://www.atena

6.3 Standard and tools

All the documents must be written in English.

All the document deliverablesOffice 2007© or later (or compatible toolsExcel© (.xlsx).

6.4 Modification and Update

The document modifications are described synthetically in the description allows a simple identification of the modified parts and the comprehension of the change motivations to the recipients of the new review of the

If the document is widely modified, it is defined as completely reviewed, indicatingreview motivations.

After modification, the document follow

6.5 Review Procedures

In the following steps the process

• The PC and the WP leader will define the titentative names of internal expert reviewers

• The WP leader (also known asto WP partners asking

• The involved partners may contribute by providing their proposals and contributions to the WP leader acting as editor, maintaining unchanged the Major.

• Whenever the WP leader merges the partners’ conthe WP leader decides to sent this new version to the partners with a different version numbering (increasing the Major numbering by one, putting the Minor to zero; or leaving unchanged the Major numbering and incrany time, the WP leader will have the master of the deliverable.

• When the deliverable is mature enough (at any time in the production of the deliverable, and possibly more times dprovides internal expert reviewers with a

• The remarks originating from the partners.

• The WP leader – asking contall the remarks providing adequate explanations leader is responsible for updating the document to take th

• The WP leader provideCoordinator within the agreed deadline;



Classification PU

ts must follow the structure and physical layout agreed in the templates defias a joint work of WP1 and WP8. The templates are made available for Consortium partners

https://www.atena-h2020.eu).

Standard and tools

ments must be written in English.

s are produced by means of the standard toocompatible tools): Microsoft© Word© (.docx), PowerPoint

and Update

The document modifications are described synthetically in the Change Logdescription allows a simple identification of the modified parts and the comprehension of the change motivations to the recipients of the new review of the document.

If the document is widely modified, it is defined as completely reviewed, indicating

After modification, the document follows the review procedures.

Review Procedures

the process of internal review is described:

The PC and the WP leader will define the time schedule for the deliverabletentative names of internal expert reviewers;

also known as Lead Beneficiary) prepares a Draft asking for comments and contributions;

The involved partners may contribute by providing their proposals and contributions to the WP leader acting as editor, maintaining unchanged the Major.Minor numbering.

Whenever the WP leader merges the partners’ contributions and edit a new stable version, the WP leader decides to sent this new version to the partners with a different version numbering (increasing the Major numbering by one, putting the Minor to zero; or leaving unchanged the Major numbering and increasing the Minor numbering by one, as usual). At any time, the WP leader will have the master of the deliverable.

deliverable is mature enough (at any time in the production of the deliverable, and possibly more times during the preparation, if it seems worth doing it

internal expert reviewers with a Final Draft version, within the agreed deadline

The remarks originating from the internal review must be notified to the

asking contributions to WP partners when needed all the remarks providing adequate explanations and modifications

is responsible for updating the document to take the accepted remarks into account;

ovides the merged version (with status Validation) Coordinator within the agreed deadline;


Page 19 of 22

ts must follow the structure and physical layout agreed in the templates defined for Consortium partners in the

are produced by means of the standard tools included in Microsoft© , PowerPoint© (.pptx), and

Change Log register. This description allows a simple identification of the modified parts and the comprehension of the

If the document is widely modified, it is defined as completely reviewed, indicating, however, the

me schedule for the deliverable, and the

Draft 0.1 version and send it

The involved partners may contribute by providing their proposals and contributions to the inor numbering.

tributions and edit a new stable version, the WP leader decides to sent this new version to the partners with a different version numbering (increasing the Major numbering by one, putting the Minor to zero; or leaving

easing the Minor numbering by one, as usual). At

deliverable is mature enough (at any time in the production of the deliverable, seems worth doing it) the WP leader

, within the agreed deadline;

review must be notified to the WP leader and WP

ributions to WP partners when needed - collects and analyses and modifications when required. The WP

e accepted remarks into account;

the merged version (with status Validation) to the Project


• The Project Coordinator has to decide whether or not to involve partners in the Security Advisor Board according to the content and the nature of the

• If needed, the Security Advisor Board is asked to public deliverables that are possibly revealing sensitive information)

• The PAM assesses the quality review, conventions required by ECConsortium (e.g. deleting the internal history change log)now Final and a new versioning is especially defined fostarting again from 1.0 independently from possible higher numbers in non-Final) versions and, in case of future submissions, increasing this special ECversion and updating the history change log

• The Project Coordinator



Classification PU

The Project Coordinator has to decide whether or not to involve partners in the Security Advisor Board according to the content and the nature of the deliverable;

Security Advisor Board is asked to assess the sensitivity aspectspublic deliverables that are possibly revealing sensitive information)

assesses the quality review, and applies if needed the nameconventions required by EC or considered as adequate for publishing

(e.g. deleting the internal history change log). The status of the deliverable is and a new versioning is especially defined for deliverables submitted to EC

starting again from 1.0 independently from possible higher numbers in and, in case of future submissions, increasing this special EC

version and updating the history change log of previously submitted Final deliverables

ject Coordinator delivers the Final document to the EC.


Page 20 of 22

The Project Coordinator has to decide whether or not to involve partners in the Security deliverable;

assess the sensitivity aspects (only for the public deliverables that are possibly revealing sensitive information);

if needed the name, version and format or considered as adequate for publishing externally to the

. The status of the deliverable is bles submitted to EC - so

starting again from 1.0 independently from possible higher numbers in intermediate (i.e. and, in case of future submissions, increasing this special EC-related

viously submitted Final deliverables;


7 Quality RecordsIn the table below are reported the quality records established for this

Document

Review Reports

KPI Reports

Test Plan

Test Report

Audit Report



Classification PU

Quality Records In the table below are reported the quality records established for this project.

To be delivered

Review Reports No

Yes

Yes

No

Audit Report No

Table 11: Quality Records


Page 21 of 22

project.

To be delivered


8 References [1] ISO 9001:2008 — Quality management systems

http://www.iso.org/iso/cata

[2] ATENA Grant Agreement Number 700581

[3] ATENA Consortium Agreement 700581

[4] ATENA Grant Agreement Number 700581numbering in [2] (pagg. 85

[5] ATENA Grant Agreement Number 700581numbering in [2] (pagg. 212



Classification PU

Quality management systems — Requirementshttp://www.iso.org/iso/catalogue_detail?csnumber=46486

ATENA Grant Agreement Number 700581, signed agreement, 2016

NA Consortium Agreement 700581 IA under Horizon 2020, internal agreement, 2016

ATENA Grant Agreement Number 700581 - Annex 1 (part A), document included with separ(pagg. 85-147), 2016

ATENA Grant Agreement Number 700581 - Annex 1 (part B), document included with separate (pagg. 212-301), 2016


Page 22 of 22

Requirements. On line (June 2016):

, internal agreement, 2016

document included with separate

, document included with separate