e-infrastructure & data service

Riccardo Rotondo riccardo.rotondo@ct.infn.it

e-Infrastructure & Data Services

Features and implementations

Catania, 14/03/2014

2

Outline Requirements e-Infrastructure Service Data Service Workflow Javadocs Implementations Demo

3

e-Infrastructure & Data Service We designed the architecture to provide a

Service accessible from your application. Modular attachable resources available to the

SG thanks to the straight coupled service that allows read/write data from several e-Infrastructures.

4

e-Infrastructure Service Four different kind of e-Infrastructure

identified: Local Remote Grid Cloud

Possibility of defining different Policy and Quota according e-infrastructures/users availability/needs

Default and favorite e-Infrastructures selectable

Disaster recovery and (cross) file replica.

5

e-Infrastructure Service If you want to copy a file on a file system

you plug-in an external storage to it.

HFS

ext4

NFS

FAT

6

e-Infrastructure Service On a SG now you are able to plug-in an e-

Infrastructure.

Local Storage

Remote Storage

Grid Storage

Cloud Storage

7

Data Services Business logic related to file management

totally independent from the underneath technology

All the common features associated to file provided: create, copy, delete, move and sharing.

For each features, we implemented a different Storage Interface that translates the single commands according to the technology

All data are stored in different tables in order to let users organise files on a virtual file system.

8

Data Service

Local Storage

Remote Storage

Grid Storage

Cloud Storage

Different implementations

of the same storage interface

9

Upload workflow

1. Sign in

Database

5. File Upload

4. e-Infra found

7. Update

6. Upload on e-Infras

2. Upload request

? 3. e-Infra Discovery

10

Data Management on variouse-Infrastructures

Dec ’13: release

0.1

Riccardo Rotondo

Service Implementation Package name Namespace Entity

Interface type Finder Ordered value One-to-many relation Many-to-many relation Fake entity

Tutorial on Science Gateways, Roma, 04.06.2013

service.xml

12

Database created (automatically by service buider)mysql> show tables like 'eInfrSrv%';

+---------------------------------+

| Tables_in_lportal61 (eInfrSrv%) |

+---------------------------------+

| eInfrSrv_File |

| einfrsrv_files_tags |

| einfrsrv_files_virtualpaths |

| einfrsrv_infrastructure |

| einfrsrv_properties |

| einfrsrv_storageurls |

| einfrsrv_tag |

| einfrsrv_virtualpath |

+---------------------------------+

File

Directory

File url on storage

e-Infrastructures

File tags

Properties (reffered to e-Infra)

13

Javadocs

http://www.catania-science-gateways.it/javadocs/einfrsrv-portlet/index.html

14

Model & Service Implementation

Custom model implementation

Custom service implementation

15

Main features e-Infrastructure:

Define several e-Infrastructures; Associate each e-Infra with Policy and Quota

Storage interface: A standard interface defined to support any

storage solution simply implementing the abstract method

Directory: File tree data organisation

16

e-Infrastructure Service (it.infn.ct.einfrsrv.sb.service.impl.InfrastructureLocalServiceImpl)

InfrastructuregetDefaultInfrastructure(long companyId, long userId, java.lang.String portletId, java.lang.String type)

17

Data Service Storage (it.infn.ct.einfrsrv.data.storage)

java.lang.String getStoragePath() boolean isURLAvailable(java.lang.String url) java.lang.String putFile(java.lang.String filePath)

java.lang.String putFile(java.lang.String source, java.lang.String destination)

boolean writeFileToStream(java.lang.String filePath, java.io.OutputStream out)

boolean removeFile(java.lang.String filePath)

18

Data Service File (it.infn.ct.einfrsrv.sb.service.impl.FileLocalServiceImpl)

boolean upload(long companyId, java.lang.String companyName, long userId, java.lang.String portletId, Infrastructure infraParam, java.lang.String fileName, java.lang.String source, java.lang.String virtualPathParam, java.lang.String[] tagsParam)

java.util.List<File> listFiles(long userId, java.lang.String virtualPathParam)

boolean writeFileToStream(long userId, long fileId, java.io.OutputStream out)

java.net.URL getURL(long userId, long fileId)

19

Data Service Directory (it.infn.ct.einfrsrv.sb.service.impl.VirtualPathLocalServiceImpl)

VirtualPathmakeDir(java.lang.String parentPath, java.lang.String path, long userId)

20

Portlet accessing the service Add to liferay-plugin-package.properties:

required-deployment-contexts=einfrsrv-portlet

21

Source & Binary Sourceforge: http

://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/HEAD/tree/

Rel 0.1: http://sourceforge.net/p/ctsciencegtwys/einfsrv/sources/HEAD/tree/tags/release-0.1/

Binaryhttps://sourceforge.net/projects/ctsciencegtwys/files/eInfrastructureService/Release/0.1/

22

Demo Whishing to see e-Infra & Data service in

action? Request access to GARR Science Gateway (https://sgw.garr.it)

Remember the service is still under development so your feedback is welcome!

23

Questions ?