centre de calcul de l’institut national de physique
TRANSCRIPT
![Page 1: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/1.jpg)
Centre de Calcul de l’Institut National de Physique Nucléaire et de Physique des Particules
eTRIKS Project
Sneak Peek FJPPL Computing Workshop @ CCIN2P3, March 11th 2015
Benjamin Guillon, Systems Engineer
![Page 2: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/2.jpg)
Agenda
FJPPL Computing Workshop 11/03/2015 2
Platform Overview
Platform Usage
Next Steps
A cloud based platform
Project Modules
Key Services
Hosted Projects
Platform Monitoring
Usage Statistics
Project Overview
Translational research
The eTRIKS project
![Page 3: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/3.jpg)
Translational Research
FJPPL Computing Workshop 11/03/2015 3
Patients with diseases
WGS RNAseq Mass Spec Imaging RT Sensing
Bioassays: measurements on genes,
molecules, organs
Combine clinical observations and bioassay techniques ◦ provide more efficient research of treatments
+
![Page 4: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/4.jpg)
The eTRIKS Project
FJPPL Computing Workshop 11/03/2015 4
A knowledge management platform ◦ Store, curate, analyze and export data
◦ Host multiple IMI projects (and others…)
Increasing the efficiency of translational research ◦ Reduced costs: one platform to rule them all
◦ Cross study analyses
◦ Private/Public Translational Research IMI support
◦ Open agreed standards across the IMI TR projects
◦ Open source and interoperable software: TranSMART
![Page 5: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/5.jpg)
Quick provisioning
Horizontal Scalability
Resources utilization efficiency
A cloud based platform
FJPPL Computing Workshop 11/03/2015 5
![Page 6: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/6.jpg)
Project Modules (1)
FJPPL Computing Workshop 11/03/2015 6
Physical host
Virtual machines
1 Project = n VMs + 1 DB instance
Database server
User raw data DB Instance
iSCSI Volume SSH gateway
Curation ETL
tranSMART worker(s)
© Julien Carpentier
![Page 7: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/7.jpg)
Project Modules (2)
FJPPL Computing Workshop 11/03/2015 7
Physical host
Virtual machines
Project A
1 Project = n VMs + 1 DB instance
Project B Project D Project E
Database server
User raw data DB Instance
iSCSI Volume SSH gateway
Curation ETL
tranSMART worker(s)
© Julien Carpentier
![Page 8: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/8.jpg)
The eTRIKS web portal
Key Services (1)
FJPPL Computing Workshop 11/03/2015 8
![Page 9: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/9.jpg)
TranSMART
Key Services (2)
FJPPL Computing Workshop 11/03/2015 9
![Page 10: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/10.jpg)
Galaxy
Key Services (3)
FJPPL Computing Workshop 11/03/2015 10
![Page 11: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/11.jpg)
Data Curation Environment ◦ Dedicated curation VM
◦ ETL tools
◦ Direct database access
◦ Complete Linux environment for curators
Data Storage Services ◦ Various types (Block storage, Databases)
◦ 240TB of raw storage space
◦ Focused on reliability
Key Services (4)
FJPPL Computing Workshop 11/03/2015 11
![Page 12: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/12.jpg)
Oncotrack ◦ Oncology research
◦ www.oncotrack.eu
Abirisk ◦ Drug immunization research
◦ www.abirisk.eu
Public Server: the eTRIKS showroom ◦ Accessible from the portal
◦ Open access
◦ Public data from clinical studies on various diseases
More projects joining soon…
Hosted Projects
FJPPL Computing Workshop 11/03/2015 12
![Page 13: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/13.jpg)
Multiple levels of monitoring
Platform Monitoring
FJPPL Computing Workshop 11/03/2015 13
• Application access metrics
• Various information about users
• Time spent using the applications
End-users
• Apache web servers
• Tomcat application servers and the Java Virtual Machine
• PostgreSQL databases
• SSH gateways
• Compute and storage metrics
• CPU, disk I/O, memory and network
• Storage space capacity
• Electrical consumptions
Infrastructure
Services
![Page 14: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/14.jpg)
Users on the public server
Usage Statistics (1)
FJPPL Computing Workshop 11/03/2015 14
0
50
100
150
200
250
300
350
2014 Jul 2014 Aug 2014 Sep 2014 Oct 2014 Nov 2014 Dec 2015 Jan
Visites Unique visitors
![Page 15: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/15.jpg)
Users on the public server
Usage Statistics (2)
FJPPL Computing Workshop 11/03/2015 15
0
200
400
600
800
1000
1200
1400
1600
1800
2014 Jul 2014 Aug 2014 Sep 2014 Oct 2014 Nov 2014 Dec 2015 Jan
Data Transfered (MB)
![Page 16: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/16.jpg)
Database cumulated sizes
Usage Statistics (3)
FJPPL Computing Workshop 11/03/2015 16
89
2,5
24
Loads in gigabytes (GB)
Public Abirisk Oncotrack
Addition of the TranSMART v1.1, v1.2 (and Galaxy when available) databases.
~ 115GB / 100TB
![Page 17: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/17.jpg)
Disk space usage
Usage Statistics (4)
eTRIKS Annual Meeting 2015 11/03/2015 17
75
47 19
94
Usage in gigabytes (GB)
Public
Abirisk
Oncotrack
Common
Addition of all the projects disk storage used (excluding databases).
~ 235GB / 100TB
![Page 18: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/18.jpg)
FJPPL Computing Workshop 11/03/2015 18
Next Steps
Multi-site federated cloud ◦ Horizontal scaling
◦ Fail-over and high availability
◦ Data backup
◦ Legal issues
Application breakdown ◦ From a monolithic to a modular architecture
◦ Increase horizontal scalability for the backends
PostgreSQL, R, Tomcat.
◦ Mutualize frontends
Apache, HAProxy.
Docker support ◦ On top of Openstack?
◦ For dissemination purposes
Automate services deployment ◦ Use puppet to deploy and maintain services
![Page 19: Centre de Calcul de l’Institut National de Physique](https://reader036.vdocuments.site/reader036/viewer/2022071612/6156fac9a097e25c764fa939/html5/thumbnails/19.jpg)
Any questions?
FJPPL Computing Workshop 11/03/2015 19
Dr. Pengfei Liu ◦ eTRIKS software security developer from CCIN2P3
◦ ISGC 2015 Taiwan (Taipei) - March 15th - 20th 2015
http://event.twgrid.org/isgc2015/
Next week in …