exschema
DESCRIPTION
Overview of the ExSchema projectTRANSCRIPT
ExSchemaDiscovering Schemas from Polyglot
Persistence Applications
Juan Castrejón - Université de Grenoble
2
ObjectiveDiscover schemas from the source code
of polyglot persistence applications
Document DB
Column-Family DB
GraphDB
Source code
RelationalDB
Key-Value DB
3
Why?Polyglot persistence applications
are becoming widespread
Schema-less datastores
Non-standard APIsImplicit schemas described
in the source code
But for their development and maintenance,software engineers have to deal with…
4
How?
Declarations Analyzer
Updates Analyzer
Repositories Analyzer
MetaLayer Representation
ExSchema
Neo4j API MongoDB API HBase API
JPA API Spring Data CouchDB API
Application source code
Annotations Analyzer
Analyze project structure and update operations
5
MetaLayerSet
Struct
Attribute
Relationship
*
*
*
**
**
**
Based on: P. Atzeni, F. Bugiotti, and L. Rossi. Uniform access to non-relational database systems: the SOS platform. In CAiSE’12, volume 7328 of LNCS, pages 160–174. Springer, 2012.
6
ResultsPDF file
Spring Roo scripts(JPA, MongoDB, Neo4j)
7
Demonstrationimport org.neo4j.graphdb.Node;import org.neo4j.graphdb.Relationship;
class ActorImpl implements Actor { private static final String NAME_PROPERTY = "name”;
private final Node underlyingNode;
public void setName( final String name ) { underlyingNode.setProperty( NAME_PROPERTY, name ); }
public Role createRole( final Actor actor, final Movie movie, final String roleName ) final Node actorNode = ((ActorImpl) actor).getUnderlyingNode(); final Node movieNode = ((MovieImpl) movie).getUnderlyingNode(); final Relationship rel = actorNode.createRelationshipTo( movieNode, RelTypes.ACTS_IN );…} https://github.com/neo4j-examples/imdb
Neo4j
Declaration
Update
8
Demonstration Neo4j
PDF file
9
Demonstration Neo4j
Spring Roo script
10
Demonstration Spring Data Neo4j
https://github.com/neo4j-examples/cineasts
import org.springframework.data.annotation.Indexed;import org.springframework.data.graph.annotation.NodeEntity;import org.springframework.data.graph.annotation.RelatedToVia;import org.springframework.data.graph.core.Direction;
@NodeEntitypublic class Actor { @Indexed(indexName="actor_id") private String id; private String name; @RelatedToVia(type=Participation.RELATIONSHIP_TYPE, direction=Direction.OUTGOING , elementClass=Participation.class) Iterable<Participation> participations;…}
Declaration
Annotation
11
Demonstration Spring Data Neo4j
PDF file
12
Demonstration
Spring Roo script
Spring Data Neo4j
13
Demonstration Spring Data MongoDB
http://vargas-solar.imag.fr/academika/cloud-data-management/
import javax.persistence.ManyToOne;import org.springframework.roo.addon.layers.repository.mongo.RooMongoEntity;
@RooMongoEntitypublic class Post { private String idContact; private String postGeoStamp; private Date postTimeStamp; private String idPost;
@ManyToOne private Content content;…}
Declaration
Annotation
14
Demonstration Spring Data MongoDB
http://vargas-solar.imag.fr/academika/cloud-data-management/
import fr.imag.mynet.domain.Post;import java.util.List;import org.springframework.roo.addon.layers.repository.mongo.RooMongoRepository;
@RooMongoRepository(domainType = Post.class)public interface PostRepository {
List<fr.imag.mynet.domain.Post> findAll();}
Repository
15
Demonstration Spring Data MongoDB
PDF file
Demonstration Spring Data MongoDB
Spring Roo script
17
Demonstrationimport org.apache.hadoop.hbase.client.HTable;import org.apache.hadoop.hbase.client.Put;import org.apache.hadoop.hbase.util.Bytes;
public class PutExample { public static void main(String[] args) throws IOException { HTable table = new HTable(conf, "testtable"); Put put = new Put(Bytes.toBytes("row1"));
put.add(Bytes.toBytes("colfam1"), Bytes.toBytes("qual1"), Bytes.toBytes("val1")); put.add(Bytes.toBytes("colfam1"), Bytes.toBytes("qual2"), Bytes.toBytes("val2")); table.put(put); …}
HBase
https://github.com/larsgeorge/hbase-book/tree/master/ch03
Declaration
Update
18
Demonstration HBase
PDF file
19
Demonstrationimport com.fourspaces.couchdb.Database;import com.fourspaces.couchdb.Document;import com.fourspaces.couchdb.Update;
public class UpdateTest { Database foo;
public void createTestDB() { Document testDoc = new Document(); testDoc.put("_id", "test_data"); testDoc.put("Field1", "Default"); testDoc.put("Field2", "Default");
foo.saveDocument(testDoc);…}
CouchDB
https://github.com/mbreese/couchdb4j
Declaration
Update
20
Demonstration
PDF file
CouchDB
21
Demonstration MongoDB+JPA
https://github.com/SpringSource/cloudfoundry-samples/tree/master/cross-store
import javax.persistence.Entity;import org.springframework.data.mongodb.crossstore.RelatedDocument;
@Entitypublic class Customer {
@Id private Long id; private String firstName; private String lastName;
@RelatedDocument private SurveyInfo surveyInfo;…}
Declaration
Annotation
22
Demonstration MongoDB+JPA
PDF file
23
Demonstration MongoDB+JPA
Spring Roo script
24
Test applicationsNeo4j: - https://github.com/neo4j-examples/cineasts.git - https://github.com/neo4j-examples/imdb.git - https://github.com/neo4j-examples/java-astar-routing.git - https://github.com/neo4j-examples/java-dijkstra.git - https://github.com/neo4j-examples/java-tree-traverse.git - MyNetContacts (http://vargas-solar.imag.fr/academika/cloud-data-management/)MongoDB: - https://github.com/mongolab/mongodb-driver-examples.gitHBase: - https://github.com/larsgeorge/hbase-book.git (ch03) - https://github.com/SpringSource/spring-hadoop-samples.git (original-samples/hbase-crud)CouchDB: - https://github.com/mbreese/couchdb4j.gitRelational: - Indvalid-core (http://www.indvalid.com/)Relational + MongoDB: - https://github.com/SpringSource/cloudfoundry-samples.git (cross-store) - MyNet (http://vargas-solar.imag.fr/academika/cloud-data-management/) - Indvalid-dao (http://www.indvalid.com/) Neo4j + MongoDB + Relational: - twitter-spring - twitter-polyglot(Based on: P. Atzeni, F. Bugiotti, and L. Rossi. Uniform access to non-relational database systems: the SOS platform. In CAiSE’12, volume 7328 of LNCS, pages 160–174. Springer, 2012)
Industrial application
25
LimitationsBased on project structure and update operations
Based on programming styles of test applications
(Queries and get operations not considered)
(Heavily relies on local variables)
Limited associations between entities(Besides Neo4j’s relationships and MongoDB cross-store)
26
Future workAnalysis of queries and get operations
Support additional languages besides Java
Increase support for different programming styles
27
Implementation
Eclipse JDTEclipse AST
Spring Data
https://code.google.com/p/exschema/
