www.eu-etics.org

INFSOM-RI-026753

Alberto Di Meglio, Project Director, CERNAlberto Di Meglio, Project Director, CERNETICS 1ETICS 1stst EU Review EU ReviewCERN, Geneva - 9 February 2007CERN, Geneva - 9 February 2007

Status of the ProjectStatus of the Project

INFSOM-RI-026753

Introduction

• General Status of the project, providing an overview of:General Status of the project, providing an overview of:– Review AgendaReview Agenda– Project GoalsProject Goals– AchievementsAchievements– Workpackages Status SummaryWorkpackages Status Summary– Deliverables and milestonesDeliverables and milestones– Financial statusFinancial status– Manpower levelsManpower levels

1st ETICS Review – Status of the Project - CERN, 9 February 2007 2

INFSOM-RI-026753

Review Agenda

1st ETICS Review – Status of the Project - CERN, 9 February 2007 3

14:00 - Demo Matteo Selmi (INFN)Paolo Fabriani (Engineering)

15:00 - Dissemination Andrea Manieri (Engineering)15:45 - Future Plans Alberto Di Meglio (CERN)16:30 - Reviewers Closed Session followed by Feedback

MorningMorning

AfternoonAfternoon

INFSOM-RI-026753

Project Goals

To provide within a two year programme:To provide within a two year programme:

• A professionally managed, integrated system of services and A professionally managed, integrated system of services and resources for running automated builds and test suites for resources for running automated builds and test suites for distributed software projectsdistributed software projects

• A repository of packages, test reports and interoperability A repository of packages, test reports and interoperability information that development projects can use to develop, information that development projects can use to develop, validate and certify their productsvalidate and certify their products

• A forum where to discuss and promote software quality initiatives A forum where to discuss and promote software quality initiatives and collect common software engineering tools and processes, and collect common software engineering tools and processes, software configuration information and documentation.software configuration information and documentation.

1st ETICS Review – Status of the Project - CERN, 9 February 2007 4

INFSOM-RI-026753 1st ETICS Review – Status of the Project - CERN, 9 February 2007 5

The ETICS Partners

Build system, software configuration, service

infrastructure, dissemination, EGEE,

gLite, project coord.

Software configuration, service infrastructure, dissemination

Web portals and tools, quality process, dissemination, DILIGENT

Test methods and metrics, unit testing tools, EBIT

The Condor batch system, distributed testing tools, service infrastructure, NMI

INFSOM-RI-026753

Project Structure

1st ETICS Review – Status of the Project - CERN, 9 February 2007 6

INFSOM-RI-026753

Project Structure

1st ETICS Review – Status of the Project - CERN, 9 February 2007 7

Project Management Board Project Technical Committee

Project Manager (PM)

WP2 WP3

WP4 WP5

INFSOM-RI-026753

What has ETICS delivered?• InfrastructureInfrastructure

– Three resource sites at Three resource sites at CERN, , INFNand and UoW with more that 150 CPUs with more that 150 CPUsand more than 15 platformsand more than 15 platforms

– A A Service Level Agreement and anddocumented deployment proceduresdocumented deployment procedures

• ServicesServices– A set of web services and tools for configuring,A set of web services and tools for configuring,

building and testing softwarebuilding and testing software– Web and multiplatform CLI clientsWeb and multiplatform CLI clients– An initial repository of An initial repository of reports and and packages

• Community SupportCommunity Support– Build and test support for more than 10Build and test support for more than 10

projects and in particular EGEE (gLite, mpi, projects and in particular EGEE (gLite, mpi, GridWay), DILIGENT and OMII-EuropeGridWay), DILIGENT and OMII-Europe

• Dissemination and TrainingDissemination and Training– Three major training events (Three major training events (EGEE’06,,

DILIGENT TCOM, INFN-CNAF)DILIGENT TCOM, INFN-CNAF)– A Build &Test Community Group in OGFA Build &Test Community Group in OGF

1st ETICS Review – Status of the Project - CERN, 9 February 2007 8

INFSOM-RI-026753 1st ETICS Review – Status of the Project - CERN, 9 February 2007 9

The ETICS Timeline

Jan 06 Jun 06 Dec 06

•Kick-off

•1st Review

All-hands meeting(Budapest)

All-hands meeting(Bologna)

April 2006ETICS Alpha programme started (first gLite components registered)

July 2006ETICS Beta programme started (gLite 3.1 managed only with ETICS),DILIGENT using for testing, OMII-Europe project created

February 2007ETICS Service v1.0 Released

September 2006ETICS Pre-release programme started, first major training event in Geneva

OGF CCG Launched

INFSOM-RI-026753

• GoalsGoals– To ensure execution of the project in conformity with the To ensure execution of the project in conformity with the

Commission contract and the Consortium Agreement and interface Commission contract and the Consortium Agreement and interface with the Commission and report of project progresswith the Commission and report of project progress

– To maintain the technical coherence of the project activities as a To maintain the technical coherence of the project activities as a whole, organise and coordinate the production and fulfilment of whole, organise and coordinate the production and fulfilment of deliverables and milestonesdeliverables and milestones

– To ensure adequate collaboration among the WPs and resolve To ensure adequate collaboration among the WPs and resolve internal conflicts, manage risks, and perform contingency planning internal conflicts, manage risks, and perform contingency planning so that the project reaches its goalsso that the project reaches its goals

1st ETICS Review – Status of the Project - CERN, 9 February 2007 10

WP1: Administrative and Technical Management

INFSOM-RI-026753

WP1: Administrative and Technical Management

• Major AchievementsMajor Achievements– Set up of the project structure, communications channels and Set up of the project structure, communications channels and

reporting toolsreporting tools– Set up and coordination of the Project Management Board and the Set up and coordination of the Project Management Board and the

Project Technical Committee, organization with the Partners of the Project Technical Committee, organization with the Partners of the project All Hands meeting in Budapest and Bolognaproject All Hands meeting in Budapest and Bologna

– Production of the project Quality Assurance and Progress Monitoring Production of the project Quality Assurance and Progress Monitoring Plan (deliverable Plan (deliverable D1.1), the Project Management Plan and the ), the Project Management Plan and the Activity Plan, the Quarterly and , the Quarterly and Year 1 Reports Reports

– Participation to several international events (like the EGEE Participation to several international events (like the EGEE User Forum and and Conference, , IGSS 2006, , eIRG and IST events) to and IST events) to showcase ETICS and promote collaboration with other projectsshowcase ETICS and promote collaboration with other projects

• IssuesIssues– Considerable effort had to be spent in ensuring that the monthly Considerable effort had to be spent in ensuring that the monthly

effort reports and the project deliverables are submitted within effort reports and the project deliverables are submitted within deadlines. In the future reminders will be sent soonerdeadlines. In the future reminders will be sent sooner

1st ETICS Review – Status of the Project - CERN, 9 February 2007 11

INFSOM-RI-026753

• GoalsGoals– To set up and maintain the grid services and computing resources To set up and maintain the grid services and computing resources

where ETICS testing, integration and validation activities are where ETICS testing, integration and validation activities are performed across the resource centresperformed across the resource centres

– To set up and maintain the software repository, possibly in To set up and maintain the software repository, possibly in collaboration with other resource providers and implement the collaboration with other resource providers and implement the packages management strategies during build/test executionpackages management strategies during build/test execution

– To define and publish the Service Level Agreements with resource To define and publish the Service Level Agreements with resource providers and service usersproviders and service users

1st ETICS Review – Status of the Project - CERN, 9 February 2007 12

WP2: Infrastructure and Service Management

INFSOM-RI-026753

• Major AchievementsMajor Achievements– Deployment and maintenance of the three resource sites at CERN, INFN Deployment and maintenance of the three resource sites at CERN, INFN

and University of Wisconsin with more than 150 CPUs running a variety of and University of Wisconsin with more than 150 CPUs running a variety of more than 15 different platformsmore than 15 different platforms

– Production of the project Service Level Agreement (deliverable Production of the project Service Level Agreement (deliverable D2.1) and ) and documented deployment documented deployment procedures for all ETICS services for all ETICS services

– Set up of the ETICS repository infrastructure and procedures for deploying Set up of the ETICS repository infrastructure and procedures for deploying the ETICS service componentsthe ETICS service components

• IssuesIssues– The time zone difference and distance between US and Europe makes The time zone difference and distance between US and Europe makes

sometimes difficult to have internal meeting. The face-to-face all-hands sometimes difficult to have internal meeting. The face-to-face all-hands meeting have provided a reliable mitigation to this issuemeeting have provided a reliable mitigation to this issue

– Production use of ETICS by pilot projects (gLite and DILIGENT) has started Production use of ETICS by pilot projects (gLite and DILIGENT) has started sooner than expected putting strong pressure on the teamsooner than expected putting strong pressure on the team

1st ETICS Review – Status of the Project - CERN, 9 February 2007 13

WP2: Infrastructure and Service Management

INFSOM-RI-026753

• GoalsGoals– To provide a configuration information schema able to capture the To provide a configuration information schema able to capture the

configuration information of software developed by research project and configuration information of software developed by research project and allowing coherent storage of such information in the ETICS database allowing coherent storage of such information in the ETICS database

– To provide a configuration, build and integration system based on existing To provide a configuration, build and integration system based on existing or adapted tools able to perform build/integration tasks to be executed by or adapted tools able to perform build/integration tasks to be executed by the ETICS facilitythe ETICS facility

– To collaborate with WP2, WP4 and WP5 in deploying the tools, integrating To collaborate with WP2, WP4 and WP5 in deploying the tools, integrating the different ETICS services, clients and administration tools and defining the different ETICS services, clients and administration tools and defining metrics and generating reports, especially in the field of software metrics and generating reports, especially in the field of software interoperability and standard complianceinteroperability and standard compliance

1st ETICS Review – Status of the Project - CERN, 9 February 2007 14

WP3: Software Configuration Tools and Methodologies

INFSOM-RI-026753

• Major AchievementsMajor Achievements– Definition of a data model to describe configuration information and Definition of a data model to describe configuration information and

build/test operation in a secure, multiplatform environment. Implementation build/test operation in a secure, multiplatform environment. Implementation in a relational database system (MySQL)in a relational database system (MySQL)

– Design and development of a web service to coordinate the work between Design and development of a web service to coordinate the work between the database and the various build/test tools and clientsthe database and the various build/test tools and clients

– Integration and extension of build tools and clients from EGEE and other Integration and extension of build tools and clients from EGEE and other projects to work in the distributed, multiplatform environments that ETICS projects to work in the distributed, multiplatform environments that ETICS must support (as described in D3.1)must support (as described in D3.1)

– Initial analysis of interoperability and compliance requirements (deliverable Initial analysis of interoperability and compliance requirements (deliverable D3.2)D3.2)

• IssuesIssues– Not all resources available for the first three months. Partially compensated Not all resources available for the first three months. Partially compensated

by reallocating some tasks to WP5by reallocating some tasks to WP5– Interoperability metrics definition/implementation delayed as a Interoperability metrics definition/implementation delayed as a

consequence. The activity has now started in January, new personnel being consequence. The activity has now started in January, new personnel being recruited to compensate initial delaysrecruited to compensate initial delays

1st ETICS Review – Status of the Project - CERN, 9 February 2007 15

WP3: Software Configuration Tools and Methodologies

INFSOM-RI-026753

• GoalsGoals– To collect requirements for unit, functional and regression testingTo collect requirements for unit, functional and regression testing– To deploy, adapt and maintain the software testing tools in addition to To deploy, adapt and maintain the software testing tools in addition to

collect and adapt the related testing methodologycollect and adapt the related testing methodology– To identify quality metrics, and evaluate the different methods and tools To identify quality metrics, and evaluate the different methods and tools

based on them, provide measurement collectors for the reporting enginebased on them, provide measurement collectors for the reporting engine

1st ETICS Review – Status of the Project - CERN, 9 February 2007 16

WP4: Testing Tools and Methodologies

INFSOM-RI-026753

• Major AchievementsMajor Achievements– Analysis of more than 50 different testing methodologies and 20 test tools Analysis of more than 50 different testing methodologies and 20 test tools

and products (documented in D4.1). and products (documented in D4.1). – Provision of Test Plan templates to help other projectsProvision of Test Plan templates to help other projects– Analysis of testing requirements of projects using ETICS, their Analysis of testing requirements of projects using ETICS, their

implementation in the system, especially in the NMI framework; definition of implementation in the system, especially in the NMI framework; definition of test metrics and prototype implementation of test plugins (documented in test metrics and prototype implementation of test plugins (documented in D4.2)D4.2)

• IssuesIssues– It has been difficult to identify open source unit test, coverage and metrics It has been difficult to identify open source unit test, coverage and metrics

analysis tools for the C and C++ programming languages. Most existing analysis tools for the C and C++ programming languages. Most existing tools have commercial licenses. Initial focus has been given to Java and tools have commercial licenses. Initial focus has been given to Java and Python, which in any case constitute a large fraction of the code of the Python, which in any case constitute a large fraction of the code of the current pilot projectscurrent pilot projects. The ‘plug-in’ mechanism offers the possibility to user . The ‘plug-in’ mechanism offers the possibility to user to use commercial tools with their own licenseto use commercial tools with their own license

1st ETICS Review – Status of the Project - CERN, 9 February 2007 17

WP4: Testing Tools and Methodologies

INFSOM-RI-026753

• GoalsGoals– To set up the project portal for the wide aggregation of user community and To set up the project portal for the wide aggregation of user community and

to supply the ETICS servicesto supply the ETICS services– To disseminate project results and organise dissemination and training To disseminate project results and organise dissemination and training

events; promote awareness about the need for quality in research projects events; promote awareness about the need for quality in research projects software development activitiessoftware development activities

– To investigate and collaborate with similar initiatives to the definition, To investigate and collaborate with similar initiatives to the definition, applicability and scope of quality assurance policies and standardsapplicability and scope of quality assurance policies and standards

– To investigate the feasibility of a certification process for software To investigate the feasibility of a certification process for software developed by research projects and outline a roadmap for its developed by research projects and outline a roadmap for its implementationimplementation

1st ETICS Review – Status of the Project - CERN, 9 February 2007 18

WP5: Dissemination, Certification and Web Tools

INFSOM-RI-026753

• Major AchievementsMajor Achievements– Design and implementation of the ETICS project identity (logo, web site, Design and implementation of the ETICS project identity (logo, web site,

templates, posters, flyers)templates, posters, flyers)– Design and implementation of the ETICS web portal to give users access to Design and implementation of the ETICS web portal to give users access to

the ETICS services (Configuration Manager, Repository and Report the ETICS services (Configuration Manager, Repository and Report Browsers, administrative applications)Browsers, administrative applications)

– Organization of three training events and related materialOrganization of three training events and related material– Proposition and organization of a Community Group on Software Proposition and organization of a Community Group on Software

Certification in the Open Grid Forum (OGF), which acts as the ETICS Certification in the Open Grid Forum (OGF), which acts as the ETICS Expert BoardExpert Board

• IssuesIssues– In the first quarter some WP3 tasks have been reallocated to WP5. This In the first quarter some WP3 tasks have been reallocated to WP5. This

issue together with the need to follow the OGF procedures has slowed issue together with the need to follow the OGF procedures has slowed down some other tasks like the certification process feasibility study. The down some other tasks like the certification process feasibility study. The activity will ramp up in year 2, but we may have to shift D5.7 by two months activity will ramp up in year 2, but we may have to shift D5.7 by two months (currently foreseen for PM20)(currently foreseen for PM20)

1st ETICS Review – Status of the Project - CERN, 9 February 2007 19

WP5: Dissemination, Certification and Web Tools

INFSOM-RI-026753

Relations with Other Projects

1st ETICS Review – Status of the Project - CERN, 9 February 2007 20

• ETICS has spent considerable effort in establishing relationships ETICS has spent considerable effort in establishing relationships with other projects:with other projects:– EGEEEGEE: SA3 (certification), JRA1 (development), NA4 (applications): SA3 (certification), JRA1 (development), NA4 (applications)– DILIGENTDILIGENT: integration and testing: integration and testing– OGFOGF: co-chair of a Community Group on Build, Test and Certification of grid : co-chair of a Community Group on Build, Test and Certification of grid

softwaresoftware– OMII-EuropeOMII-Europe: provider of the build and test infrastructure: provider of the build and test infrastructure– EUChinaGridEUChinaGrid: implementation of plugins for IPv6 compliance tests: implementation of plugins for IPv6 compliance tests– ICEAGEICEAGE: participation to the Grid Computing School 2006: participation to the Grid Computing School 2006– CERN openlabCERN openlab: co-organizer with EGEE of the HP workshop on gLite : co-organizer with EGEE of the HP workshop on gLite

industrial readinessindustrial readiness

• ETICS has actively taken part to the EC/IST activities:ETICS has actively taken part to the EC/IST activities:– eIRG Workshops (Linz, Espoo)eIRG Workshops (Linz, Espoo)– IST Workshops and Concertation Meetings (Bordeaux, Helsinki)IST Workshops and Concertation Meetings (Bordeaux, Helsinki)

INFSOM-RI-026753

Deliverables and milestones

• All deliverables/milestones (PM1-12) doneAll deliverables/milestones (PM1-12) done– 15 deliverables and 8 milestones in 12 months15 deliverables and 8 milestones in 12 months

• All deliverables submitted within deadlinesAll deliverables submitted within deadlines– only one exception (D4.1, two weeks late over the summer)only one exception (D4.1, two weeks late over the summer)

1st ETICS Review – Status of the Project - CERN, 9 February 2007 21

INFSOM-RI-026753

Deliverables List

1st ETICS Review – Status of the Project - CERN, 9 February 2007 22

Del. no.

Deliverable name WP no.

Lead participant

Delivery date(PM)

Actual delivery date

D1.1 Project Quality and Progress Monitoring Plan 1 CERN PM03 12/05/2006D2.1 Service Level Agreement document 2 UoW PM03 12/05/2006D3.1 Configuration, build and integration specifications 3 INFN PM03 12/05/2006D5.1 ETICS portal – Community support 5 ENG PM03 12/05/2006D5.2 Promotional Material 5 CERN PM03 12/05/2006D5.3 Dissemination and training plan 5 ENG PM04 15/06/2006D5.4 Plan for using and disseminating knowledge (first draft) 5 CERN PM06 14/08/2006D2.2 Infrastructure installation and usage documentation 2 UoW PM06 14/08/2006D4.1 Requirements and specifications for unit, functional and

regression testing 4 4DSOFT PM06 31/08/2006

D5.5 ETICS portal – testing support service (prototype) 5 ENG PM12 24/01/2006D2.3 Status of certification, integration and validation test bed

setup (prototype)2 UoW PM12 24/01/2006

D3.2 Interoperability reports specifications, configuration, build and integration system

3 INFN PM12 24/01/2006

D4.2 Distributed test execution system (prototype) 4 4DSOFT PM12 24/01/2006D5.6 Plan for using and disseminating knowledge (second

draft)5 CERN PM12 24/01/2006

D1.2 First year project report 1 CERN PM12 24/01/2006

•

INFSOM-RI-026753

Financial status

1st ETICS Review – Status of the Project - CERN, 9 February 2007 23

• The provisional Financial Report covering the 1st reporting The provisional Financial Report covering the 1st reporting period shows a total cost of 603 664 € and requests a total EC period shows a total cost of 603 664 € and requests a total EC contribution of 579 633 €. It represents 42% of the total contribution of 579 633 €. It represents 42% of the total maximum EC contribution as per the Annex I of the Contract.maximum EC contribution as per the Annex I of the Contract.

• Internal Cost Claims (Form C) are being submitted by all Internal Cost Claims (Form C) are being submitted by all partners to the coordinator and should be ready by the partners to the coordinator and should be ready by the February, 15February, 15thth deadline. Commercial companies have longer and deadline. Commercial companies have longer and stricter internal auditing procedures, which makes it difficult to stricter internal auditing procedures, which makes it difficult to have official data in short periods of timehave official data in short periods of time

• The breakdown of expenses is as follows:The breakdown of expenses is as follows:– Direct Costs: 76%Direct Costs: 76%– Indirect Costs: 24%Indirect Costs: 24%– Personnel: 89%Personnel: 89%– Travel, subsistence and other costs: 11%Travel, subsistence and other costs: 11%

INFSOM-RI-026753

Manpower levels• PPT tool to monitor project effort consumption (timesheets)PPT tool to monitor project effort consumption (timesheets)

– 28 people registered across Europe and USA28 people registered across Europe and USA– Total equivalent of ~ 10 FTE/yearTotal equivalent of ~ 10 FTE/year

Consumed EffortVs.Total Planned Effort

at Dec 2006

Funded Unfunded Total

45% 66% 49%

1st ETICS Review – Status of the Project - CERN, 9 February 2007 24

INFSOM-RI-026753

Summary

• ETICS is closely following the planned work program and it has ETICS is closely following the planned work program and it has delivered a first fully functional prototype of the ETICS Build and delivered a first fully functional prototype of the ETICS Build and Test Service and a fully managed infrastructureTest Service and a fully managed infrastructure

• It has exceeded the expected number of supported projects and It has exceeded the expected number of supported projects and users for Year 1 and has 3 projects (EGEE, DILIGENT and OMII-users for Year 1 and has 3 projects (EGEE, DILIGENT and OMII-Europe) using its services for production tasksEurope) using its services for production tasks

• ETICS has established strong relationships with other projects ETICS has established strong relationships with other projects and initiatives in the field of software quality assurance and and initiatives in the field of software quality assurance and certification and its rapidly establishing itself as a reliable service certification and its rapidly establishing itself as a reliable service provider and partnerprovider and partner

• The project is close to the expected levels of effort and budget The project is close to the expected levels of effort and budget and all the tools and procedures to monitor the progress and and all the tools and procedures to monitor the progress and assess and resolve issues are in placeassess and resolve issues are in place

1st ETICS Review – Status of the Project - CERN, 9 February 2007 25

INFSOM-RI-026753 1st ETICS Review – Status of the Project - CERN, 9 February 2007 26

Thanks

http://www.eu-etics.org