www.eu-etics.org infsom-ri-026753 elisabetta ronchieri infn cnaf etics 2 nd eu review (cern) 15...
TRANSCRIPT
www.eu-etics.org
INFSOM-RI-026753
Elisabetta RonchieriElisabetta RonchieriINFN CNAFINFN CNAF
ETICS 2ETICS 2ndnd EU Review (CERN) EU Review (CERN)15 February 200815 February 2008
WP3 - Software Configuration WP3 - Software Configuration Tools and MethodologiesTools and Methodologies
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Contents
ETICS 2nd EU Review - CERN - 15 February 2008 2
• GoalsGoals
• AchievementsAchievements
• ChallengesChallenges
• WP3 in NumbersWP3 in Numbers
• ConclusionsConclusions
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Contents
ETICS 2nd EU Review - CERN - 15 February 2008 3
• GoalsGoals
• AchievementsAchievements
• ChallengesChallenges
• WP3 in NumbersWP3 in Numbers
• ConclusionsConclusions
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Goals
ETICS 2nd EU Review - CERN - 15 February 2008 4
G1. Provide a configuration, build and integration system able to describe software tasks to be executed by the ETICS facility.
G2. Provide a configuration information schema able to capture the configuration information of software developed by software project and a coherent storage of such information in the ETICS database
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Goals
ETICS 2nd EU Review - CERN - 15 February 2008 5
G3. Web Server: the heart of the ETICS system – Provide a logic for the entire ETICS system– Handle access to the ETICS system in secure mode– Handle metadata, artefacts, and user requests– Perform complex queries on behalf of the clients– Handle build/test remotely– Support build/test locally
G4. Command Line Interfaces (CLIs): one of the ETICS clients– Allow users to register their software, specify configuration
information (i.e., edit metadata)– Handle access to Web Service in secure mode– Checkout code and metadata– Build/Test code locally and remotely– Tag code and metadata– Produce reports and different format of artefacts
INFSOM-RI-026753 EGEE and gLite are registered trademarks
ETICS CLIs, Build/Test Web Serivce and metadata
ETICS Portal (myETICS, Build/Test, Repository, Administration)
ReportDB
Metronome Execution Engine
Metronome ClientWrapper
WNs ETICS Infrastructure
Repository Service
Build/TestArtefacts
ProjectDB
Clients
Web Service
Via browser
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Contents
ETICS 2nd EU Review - CERN - 15 February 2008 7
• GoalsGoals
• AchievementsAchievements
• ChallengesChallenges
• WP3 in NumbersWP3 in Numbers
• ConclusionsConclusions
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Achievements
8ETICS 2nd EU Review - CERN - 15 February 2008
A1. Authentication and Authorization• Performed authentication using standard x.509 certificates in Web
Service and CLIs• Registered information of users (e.g., name and distinguish name)
in the ETICS data model • Based authorization on role definition
– i.e., defined roles to descriminate the operations users can perform– e.g., clients connecting without certificate are mapped as Guest
• Allowed or denied operations to the users by Web Service
Role Description
Administrator Super user
Module Administrator Handles individual projects
Developer runs remote builds
Integrator runs remote builds, registers artefacts in the repository
Tester runs remote tests, stores results in the report repository
Release Manager Defines configurations at project level, and project releases
Guest Read-only access
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Achievements
9ETICS 2nd EU Review - CERN - 15 February 2008
A2. Software Metadata definition• Based schema version on Common Information Model (CIM) • Defined formal entities to express project structure,
build/test/checkout configurations, security information, dependencies, environment variables, properties, users, roles– E.g., software project structure and configuration structure
– Project, Subsystem and Component referred as Modules– More than one Configurations is allowed for each Module – More than one Platforms is allowed for each Configuration– Set Hierarchy in Project and Subsystem Configurations
Project Subsystem Component Component ... Subsystem ... Component .....
Configuration Platform 1 VCS Command Build Command Test Command Dependency1 Property1 Environment1 Platform n VCS Command Hierarchy1
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Achievements
10ETICS 2nd EU Review - CERN - 15 February 2008
A3. Build/Test• Handled the downloading of code and metadata information• Supported the build and test locally and remotely• Generated platform specific artefacts (e.g., RPMS, MSI, Debian,
tarballs) during the build of software• Generated build and test reports • Supported the browsing of software information• Supported the editing of software metadata
– e.g., the possibility to create metadata, modify/remove them and copy existing metadata
• Supported co-scheduling by the editing functionality– DTDT {node1, node2, …, nodeN} {node1, node2, …, nodeN}
– Set hierarchy in DT configurationSet hierarchy in DT configuration
– NodeNode {service1, service2, …, serviceN} {service1, service2, …, serviceN}– Set services in Node configurationSet services in Node configuration
• Handled code and metadata tag
Project DestrebutedTest Node Node ... Subsystem ... Component .....
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Achievements
11ETICS 2nd EU Review - CERN - 15 February 2008
A4. Interoperability and Compliance to Standards • Metrics
– Identified four interoperability metrics1. Number of 3rd Party Components (No3C)
2. Number of Generic Components (NoGC)
3. Number of Architectural Revisions (NoAR)
4. Number of Versions (NoV)
– Used configuration information stored in the ETICS database
• Collectors– Provided information collectors for the reporting engine
– Compliance to standards:
Ipv6
WS-compliance– Interoperability
Metrics
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Contents
ETICS 2nd EU Review - CERN - 15 February 2008 12
• GoalsGoals
• AchievementsAchievements
• ChallengesChallenges
• WP3 in NumbersWP3 in Numbers
• ConclusionsConclusions
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Challenges
ETICS 2nd EU Review - CERN - 15 February 2008 13
C1. C1. Improve the data model in order to guarantee a better Improve the data model in order to guarantee a better response time from the backend response time from the backend– CIM does not scale to large software applicationsCIM does not scale to large software applications
– Started a revision of the data modelStarted a revision of the data model
C2. C2. Improve the CLI performancesImprove the CLI performances– Current version based on CIM and SOAP Current version based on CIM and SOAP – As the size of a project increases, the performance of the client As the size of a project increases, the performance of the client
decreases decreases to handle large XML documentsto handle large XML documents– SubstantiallySubstantially performance improvements by using newer version of the performance improvements by using newer version of the
Python SOAP enginePython SOAP engine (called ZSI) (called ZSI) in handling large XML documentsin handling large XML documents– Started tests in the ETICS clientStarted tests in the ETICS client
C3. C3. Improve the calculation of the configuration propertiesImprove the calculation of the configuration properties– SubstantiallySubstantially performance improvements performance improvements by making better use of by making better use of
cachingcaching
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Contents
ETICS 2nd EU Review - CERN - 15 February 2008 14
• GoalsGoals
• AchievementsAchievements
• ChallengesChallenges
• WP3 in NumbersWP3 in Numbers
• ConclusionsConclusions
INFSOM-RI-026753 EGEE and gLite are registered trademarks
WP3 in Numbers
15ETICS 2nd EU Review - CERN - 15 February 2008
DeliverablesDeliverables::
Milestones:Milestones:
Del. N. Del. Name Delivery Date
D3.1 Configuration, build and integration specifications PM03
D3.2 Interoperability reports specifications, configuration, build and integration system PM12
D3.3 Final evaluation report and configuration, build and integration system (final release) PM20
Mil. N. Mil. Description Delivery Date
M3.1 Configuration, build and integration system deployed in the ETICS resource pools (prototype) PM12
M3.2 Configuration, build and integration system deployed in the ETICS resource pools (final release) PM22
INFSOM-RI-026753 EGEE and gLite are registered trademarks
WP3 in Numbers
16ETICS 2nd EU Review - CERN - 15 February 2008
OS Version Architecture Compiler
Fedora Core 3 ia32 gcc 3.4.4
5 ia32 gcc 4.1.1
6 ia32, x86_64 gcc 4.1.1
Scientific Linux 3 ia32, x86_64 gcc 3.2.3
5 Ia32 gcc 4.1.1
Scientific Linux Cern 3 ia32, x86_64 gcc 3.2.3, gcc 3.4.5
4 ia32, ia64, x86_64 gcc 3.4.6
Red Hat Enterprise Linux 3 ia32 gcc 3.2.3
4 ia32, x86_64 gcc [3.4.3, 3.4.6], gcc [3.4.5, 3.4.6]
SUSE 9 ia32, x86_64 gcc [3.3.1, 3.3.5], gcc 3.3.3
10 ia32 gcc [4.0.2, 4.1.0]
Solaris 10 ia32 gcc 3.4.3
Debian 3 ia32, x86_64 gcc 3.3.5
4 ia32, x86_64 gcc 4.1.2
centOS 4 ia32, x86_64 gcc [3.4.4, 3.4.6], gcc 3.4.6
5 ia32, x86_64 gcc 4.1.2, gcc 4.1.1
Darwin/Mac OS 772 powerpc gcc 3.3.3
880 powerpc gcc 4.0.1
891 ia32 gcc 4.0.1
8100 powerpc gcc 4.0.0
ubuntu 5 Ia32 gcc 4.0.2
6 Ia32 gcc 4.0.3
7 ia32 gcc 4.1.2
Win32 71
aix 52 powerpc gcc 3.3.3
MeasurementsMeasurements• Supported Platforms Supported Platforms 4141
– Fedora Core 4Fedora Core 4
– Scientific Linux 3Scientific Linux 3
– Scientific Linux Scientific Linux Cern 5Cern 5
– Red Had Enterprise Red Had Enterprise Linux 5Linux 5
– SUSE 5SUSE 5
– Solaris 1Solaris 1
– Debian 4Debian 4
– centOS 5centOS 5
– Darwin/Mac OS 4Darwin/Mac OS 4
– win32 1win32 1
– aix 1aix 1
– ubuntu 3ubuntu 3
INFSOM-RI-026753 EGEE and gLite are registered trademarks
WP3 in Numbers
17ETICS 2nd EU Review - CERN - 15 February 2008
Measurements:Measurements:– Supported languagesSupported languages 7 7
– C, C++, Python, Java, Perl, sh, csh
– Implemented RequirementsImplemented Requirements 97.56% 97.56% D3.1– Total number of open and closed bugs over timeTotal number of open and closed bugs over time
– Ratio of closed/open bug over timeRatio of closed/open bug over time
020406080
100120140160
nu
mb
er o
f b
ug
s
2006 2007 years
open
closed
Ratio Year
0.90 2006
0.67 2007
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Contents
ETICS 2nd EU Review - CERN - 15 February 2008 18
• GoalsGoals
• AchievementsAchievements
• ChallengesChallenges
• WP3 in NumbersWP3 in Numbers
• ConclusionsConclusions
INFSOM-RI-026753 EGEE and gLite are registered trademarks
Conclusions
19ETICS 2nd EU Review - CERN - 15 February 2008
• Most important achievements A1. Authorization and Authentication
A2. Software Metadata Definition
A3. Build/Test
A4. Interoperability and compliance to standards
• Three outstanding challengesC1. Data Model Structure
C2. CLI Performances
C3. CLI Property Calculation
• Planned in ETICS II activities to address the challenges