e-infrastructure & data service
DESCRIPTION
TRANSCRIPT
Riccardo Rotondo [email protected]
e-Infrastructure & Data Services
Features and implementations
Catania, 14/03/2014
2
Outline Requirements e-Infrastructure Service Data Service Workflow Javadocs Implementations Demo
3
e-Infrastructure & Data Service We designed the architecture to provide a
Service accessible from your application. Modular attachable resources available to the
SG thanks to the straight coupled service that allows read/write data from several e-Infrastructures.
4
e-Infrastructure Service Four different kind of e-Infrastructure
identified: Local Remote Grid Cloud
Possibility of defining different Policy and Quota according e-infrastructures/users availability/needs
Default and favorite e-Infrastructures selectable
Disaster recovery and (cross) file replica.
5
e-Infrastructure Service If you want to copy a file on a file system
you plug-in an external storage to it.
HFS
ext4
NFS
FAT
6
e-Infrastructure Service On a SG now you are able to plug-in an e-
Infrastructure.
Local Storage
Remote Storage
Grid Storage
Cloud Storage
7
Data Services Business logic related to file management
totally independent from the underneath technology
All the common features associated to file provided: create, copy, delete, move and sharing.
For each features, we implemented a different Storage Interface that translates the single commands according to the technology
All data are stored in different tables in order to let users organise files on a virtual file system.
8
Data Service
Local Storage
Remote Storage
Grid Storage
Cloud Storage
Different implementations
of the same storage interface
9
Upload workflow
1. Sign in
Database
5. File Upload
4. e-Infra found
7. Update
6. Upload on e-Infras
2. Upload request
? 3. e-Infra Discovery
10
Data Management on variouse-Infrastructures
Dec ’13: release
0.1
Riccardo Rotondo
Service Implementation Package name Namespace Entity
Interface type Finder Ordered value One-to-many relation Many-to-many relation Fake entity
Tutorial on Science Gateways, Roma, 04.06.2013
service.xml
12
Database created (automatically by service buider)mysql> show tables like 'eInfrSrv%';
+---------------------------------+
| Tables_in_lportal61 (eInfrSrv%) |
+---------------------------------+
| eInfrSrv_File |
| einfrsrv_files_tags |
| einfrsrv_files_virtualpaths |
| einfrsrv_infrastructure |
| einfrsrv_properties |
| einfrsrv_storageurls |
| einfrsrv_tag |
| einfrsrv_virtualpath |
+---------------------------------+
File
Directory
File url on storage
e-Infrastructures
File tags
Properties (reffered to e-Infra)
13
Javadocs
http://www.catania-science-gateways.it/javadocs/einfrsrv-portlet/index.html
14
Model & Service Implementation
Custom model implementation
Custom service implementation
15
Main features e-Infrastructure:
Define several e-Infrastructures; Associate each e-Infra with Policy and Quota
Storage interface: A standard interface defined to support any
storage solution simply implementing the abstract method
Directory: File tree data organisation
16
e-Infrastructure Service (it.infn.ct.einfrsrv.sb.service.impl.InfrastructureLocalServiceImpl)
InfrastructuregetDefaultInfrastructure(long companyId, long userId, java.lang.String portletId, java.lang.String type)
17
Data Service Storage (it.infn.ct.einfrsrv.data.storage)
java.lang.String getStoragePath() boolean isURLAvailable(java.lang.String url) java.lang.String putFile(java.lang.String filePath)
java.lang.String putFile(java.lang.String source, java.lang.String destination)
boolean writeFileToStream(java.lang.String filePath, java.io.OutputStream out)
boolean removeFile(java.lang.String filePath)
18
Data Service File (it.infn.ct.einfrsrv.sb.service.impl.FileLocalServiceImpl)
boolean upload(long companyId, java.lang.String companyName, long userId, java.lang.String portletId, Infrastructure infraParam, java.lang.String fileName, java.lang.String source, java.lang.String virtualPathParam, java.lang.String[] tagsParam)
java.util.List<File> listFiles(long userId, java.lang.String virtualPathParam)
boolean writeFileToStream(long userId, long fileId, java.io.OutputStream out)
java.net.URL getURL(long userId, long fileId)
19
Data Service Directory (it.infn.ct.einfrsrv.sb.service.impl.VirtualPathLocalServiceImpl)
VirtualPathmakeDir(java.lang.String parentPath, java.lang.String path, long userId)
20
Portlet accessing the service Add to liferay-plugin-package.properties:
required-deployment-contexts=einfrsrv-portlet
21
Source & Binary Sourceforge: http
://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/HEAD/tree/
Rel 0.1: http://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/HEAD/tree/tags/release-0.1/
Binaryhttps://sourceforge.net/projects/ctsciencegtwys/files/eInfrastructureService/Release/0.1/
22
Demo Whishing to see e-Infra & Data service in
action? Request access to GARR Science Gateway (https://sgw.garr.it)
Remember the service is still under development so your feedback is welcome!
23
Questions ?