![Page 1: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/1.jpg)
POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan
Carlos Castrejon, U. de Grenoble,
Christine Collet, Grenoble INP, Rafael Lozano,
ITESM-CCM
http://vargas-solar.imag.fr
![Page 2: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/2.jpg)
DATA MANAGEMENT WITH RESOURCES CONSTRAINTS
STORAGE SUPPORT
Systems
ARCHITECTURE & RESOURCES AWARE
RAM
Algorithms
Efficiently manage and exploit data sets according to given specific storage, memory and computation resources 2
![Page 3: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/3.jpg)
DATA MANAGEMENT WITHOUT RESOURCES CONSTRAINTS
3 Costly manage and exploit data sets according to unlimited storage, memory and computation resources
Systems Algorithms
COSTAWARE
ELASTIC
![Page 4: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/4.jpg)
DEALING WITH HUGE AMOUNTS OF DATA
Peta 1015
Exa 1018
Zetta 1021
Yota 1024
RAID
Disk
Cloud
4
![Page 5: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/5.jpg)
DEALING WITH HUGE AMOUNTS OF DATA
5
Peta 1015
Exa 1018
Zetta 1021
Yota 1024
RAID
Disk
Cloud
Concurrency
Consistency
Atomicity
Relational
Graph
Key value
Columns
![Page 6: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/6.jpg)
6
![Page 7: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/7.jpg)
7
![Page 8: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/8.jpg)
8
![Page 9: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/9.jpg)
9
![Page 10: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/10.jpg)
DATA ALL AROUND IN THE ERA OF THE CLOUD
POLYGLOT PERSISTENCE
PUTTING POLYGLOT PERSISTENCE IN PRACTICE
CONCLUSION AND OUTLOOK
ROADMAP
10
![Page 11: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/11.jpg)
POLYGLOT PERSISTENCE
11
• Polyglot Programming: applications should be written in a mix of languages to take advantage of different languages are suitable for tackling different problems
• Polyglot persistence: any decent sized enterprise will have a variety of different data storage technologies for different kinds of data • a new strategic enterprise application should no longer be built
assuming a relational persistence support
• the relational option might be the right one - but you should seriously look at other alternatives
M. Fowler and P. Sadalage. NoSQL Distilled: A Brief Guide to the Emerging World of Polyglot Persistence. Pearson Education, Limited, 2012.
![Page 12: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/12.jpg)
12
![Page 13: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/13.jpg)
13
![Page 14: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/14.jpg)
WHEN IS POLYGLOT PERSISTENCE PERTINENT?
14
• Application essentially composing and serving web pages
• They only looked up page elements by ID, they had no need for transactions, and no need to share their database
• A problem like this is much better suited to a key-value store than the corporate relational hammer they had to use
• Scaling to lots of traffic gets harder and harder to do with vertical scaling
• Many NoSQL databases are designed to operate over clusters
• They can tackle larger volumes of traffic and data than is realistic with a single server
![Page 15: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/15.jpg)
15
![Page 16: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/16.jpg)
16
![Page 17: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/17.jpg)
DATA ALL AROUND IN THE ERA OF THE CLOUD
POLYGLOT PERSISTENCE
PUTTING POLYGLOT PERSISTENCE IN PRACTICE
CONCLUSION AND OUTLOOK
ROADMAP
17
![Page 18: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/18.jpg)
(Katsov-2012)
Use the right tool for the right job…
How do I know which is the right tool for the right job?
18
![Page 19: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/19.jpg)
PROBLEM
19
• How to specify data requirements for cloud environments?
• For a set of data requirements, how to choose an appropriate combination of cloud storage system implementation and deployment provider?
• How to generate/manage everything that’s required to work with the selection that I make?
![Page 20: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/20.jpg)
EXISTING SOLUTIONS
20
• Integration of cloud storage platforms (Livenson-2011)
• Cloud Data Management Interface (CDMI) (SNIA-2011) proxy to integrate blob and queue data stores
• Data integration over NoSQL stores (Curé-2011)
• Integration of relational and NoSQL databases (Document, column) • Focus on efficient answering of queries
• Storage provider selection (Ruiz-2011, Ruiz-2012) • Characterize storage providers features (Ex: performance, cost) • Specify requirements for application datasets (Ex: expected size, access
latency, concurrent clients) • Based on the previous information, an assignment of datasets to different
storage systems is proposed
![Page 21: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/21.jpg)
EXISTING SOLUTIONS
21
• Modeling as a Service (Bruneliere-2010)
• Deploy and execute model-driven services over the Internet (SaaS)
• Design and deploy applications in the cloud (Peidro-2011)
• Promotes graphical models to capture cloud requirements
• Models automatically deployed to PaaS and IaaS environments
• Application design/execution in multiple clouds (Ardagna-2012)
• MDE quality-driven method for design, development and operation
• Monitoring and feedback system
![Page 22: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/22.jpg)
LIMITATIONS OF EXISTING SOLUTIONS
22
• Support for a limited set of cloud storage interfaces
• Data integration can be highly based on the relational model
• Limited information for the selection of data storage systems
• Consideration for high-level cloud models (SaaS) but limited support for low-level models (PaaS and IaaS)
![Page 23: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/23.jpg)
OBJECTIVES
23
• Provide adequate notations and environments to characterize cloud data storage requirements
• Selection of cloud data storage implementations and deployment providers
• Management of the required artifacts to work with different combinations of cloud storage implementations and providers
![Page 24: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/24.jpg)
MODEL2ROO: APPROACH
24
UML class diagram
Spring Roo
Java web App Spring Data
Graph database
Relational database
High-level abstractions
Low-level abstractions
http://code.google.com/p/model2roo/
![Page 25: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/25.jpg)
EXSCHEMA: APPROACH
http://code.google.com/p/exschema/
25
Metalayer
Struct Relationship
AttributeSet
* *
*
*
* *
*
***
*
*
*
*
Declarations analyzer
Updates analyzer
Repositories analyzer
Schema1 Schema2Schema3
Set
Attribute
implementation : Spring RepositorySet
Attribute
name : fr.imag.twitter.domain.UserInfoStruct
Attribute
name : userId
Attribute
name : firstName
Attribute
name : lastName
Set
Attribute
implementation : Spring RepositorySet
Attribute
name : fr.imag.twitter.domain.TweetStruct
Attribute
name : text
Attribute
name : userId
Set
Attribute
implementation : Neo4jStruct
Attribute
name : fr.imag.twitter.domain.UserStruct
Attribute
name : nodeId
Attribute
name : userName
Attribute
name : userId
Attribute
name : password
Relationship
start end
Attribute
name : followers
Schema1 Schema2Schema3
Set
Attribute
implementation : Spring RepositorySet
Attribute
name : fr.imag.twitter.domain.UserInfoStruct
Attribute
name : userId
Attribute
name : firstName
Attribute
name : lastName
Set
Attribute
implementation : Spring RepositorySet
Attribute
name : fr.imag.twitter.domain.TweetStruct
Attribute
name : text
Attribute
name : userId
Set
Attribute
implementation : Neo4jStruct
Attribute
name : fr.imag.twitter.domain.UserStruct
Attribute
name : nodeId
Attribute
name : userName
Attribute
name : userId
Attribute
name : password
Relationship
start end
Attribute
name : followers
Schema1 Schema2Schema3
Set
Attribute
implementation : Spring RepositorySet
Attribute
name : fr.imag.twitter.domain.UserInfoStruct
Attribute
name : userId
Attribute
name : firstName
Attribute
name : lastName
Set
Attribute
implementation : Spring RepositorySet
Attribute
name : fr.imag.twitter.domain.TweetStruct
Attribute
name : text
Attribute
name : userId
Set
Attribute
implementation : Neo4jStruct
Attribute
name : fr.imag.twitter.domain.UserStruct
Attribute
name : nodeId
Attribute
name : userName
Attribute
name : userId
Attribute
name : password
Relationship
start end
Attribute
name : followers
Spring Data
![Page 26: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/26.jpg)
DATA ALL AROUND IN THE ERA OF THE CLOUD
POLYGLOT PERSISTENCE
PUTTING POLYGLOT PERSISTENCE IN PRACTICE
CONCLUSION AND OUTLOOK
ROADMAP
26
![Page 27: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/27.jpg)
WRAP UP
27
• Polyglot persistence is about using different data storage technologies to handle varying data storage needs • can apply across an enterprise or within a single application
• increases complexity in programming and operations, so the advantages of a good data storage fit need to be weighed against this complexity
• will come at a cost - but it will come because the benefits are worth it if used appropriately
• Encapsulating data access into services reduces the impact of data storage choices on other parts of a system
![Page 28: P PERSISTENCE FOR ADDRESSING MANAGEMENT …...POLYGLOT PERSISTENCE FOR ADDRESSING DATA MANAGEMENT ON THE CLOUD Genoveva Vargas-Solar, CNRS, LIG-LAFMIA, Juan Carlos Castrejon, U. de](https://reader035.vdocuments.site/reader035/viewer/2022062506/5f02a7007e708231d40555a1/html5/thumbnails/28.jpg)
Contact: Genoveva Vargas-Solar, CNRS, LIG-LAFMIA
http://vargas-solar.imag.fr
http://code.google.com/p/exschema/
Want to put cloud data managemet in practice ? cf. http://vargas-solar.imag.fr/academika/cloud-data-management/
28