Hybrid Data ArchitectureUnlock the full potential of your business-critical applications

Dr. Jayaraj Poroorhttp://jayaraj.poroor.com

Data architecture defines Software architecture

“Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowcharts; they'll be obvious.”

Fred BrooksComputer Pioneer & Turing Award* Winner

* Turing Award is Computer Science equivalent of Nobel Prize

Benefits of proper data architecture

● Stability● Scalability● Performance● Security● Reduced code complexity● Extensibility

Data Architecture: Good vs Bad

Application Requirements

Wrong Data Architecture

Application Requirements

Right Data Architecture

Not all application data are equal

The same application may have diverse data requirements.

BasicDomainData

Time-taggedLog Data

RelationshipData

UnstructuredText

SessionData

Force fit all data into the same data store?● Poor performance/scalability● Complex application code

○ Slow/buggy

Application’s diverse data requirements.

A single data store won’t fit all requirements

Hybrid Data Architecture

● Use the right kind of data store for each different kind of data requirement.

Data stores: sheer variety1. Key-value data stores, e.g., Memcached, Redis, Amazon SimpleDB, Amazon S3. 2. Distributed key-value data stores, e.g., Riak, Amazon DynamoDB.3. In-process key-value data stores, e.g., Level-DB, Berkeley DB, MapDB.4. Unstructured/semi-structured search databases, e.g., ElasticSearch, Apache SOLR.5. Time series databases, e.g., TempoDB, InfluxDB, OpenTSDB.6. Column family databases, e.g., Apache HBase, Cassandra, Google BigTable (Cloud Datastore).7. Document databases, e.g. CouchBase, CouchDB, MongoDB, Terrastore, RavenDB.8. Graph databases, e.g., Neo4J, InfiniteGraph, InfoGrid, OrientDB, FlockDB.9. SQL databases, e.g., MySQL, MariaDB, PostgreSQL.

10. In-process (embedded) SQL databases, e.g., HSQLDB, Apache Derby, Sqlite.11. XML databases, e.g., ExistDB, Sedna, BaseX.12. Logic-based data stores, e.g., Datomic, Jena.

About Me● 19+ years of experience in industry and

research settings.○ Holds PhD in Computer Science

● Has been architecting software systems for 14 years.

● Hands-on experience with diverse programming languages/platforms in web/mobile/embedded worlds.

Hybrid Data Architecture consultancy services● Understanding your application requirements and developing a hybrid data

architecture and design that best suites your application needs.○ I can advise at CTO level or directly work with tech leads.

● Giving hands-on training sessions to the developers to help them come up to speed on various database technologies.

● Helping developers integrate hybrid data access into the application code.

● Helping your sysadmin get the various databases installed and configured in your servers or in a cloud platform such as the AWS.

Real project cases(1) Migrating from pure MySQL to hybrid MySQL/Amazon

S3 improved an application’s performance by a factor of 5 and reduced the main transactional database size by a factor of 50.

(2) Migrating from pure MySQL-based system to a hybrid architecture of MySQL/ElasticSearch/Neo4J reduced the application code complexity significantly and reduced development/testing time.

Real project cases (2)(3) Employing a hybrid architecture of

MySQL/Redis/InfluxDB a highly scalable and stable IoT back-end application was developed in 3 month’s time by 2 developers.

(4) Search performance significantly improved for a web application by employing a hybrid architecture of MySQL/ElasticSearch.

(5) Employing Mapdb at the client-side greatly enhanced user experience, allowing disconnected operation.

Thank You

Realize the full potential of your business-critical applications through hybrid data architecture.

jayaraj.poroor@gmail.comhttp://jayaraj.poroor.com