towards understanding the performance of distributed ...€¦ · page 2 06.11.2019 | symposium on...

Towards Understanding the Performance of Distributed Database Management Systems in Volatile EnvironmentsJörg Domaschka and Daniel SeyboldInstitute of Information Resource ManagementUlm University | Ulm | Germany

06.11.2019 | Symposium on Software Performance | Towards Understanding the Performance of Distributed DBMS in Volatile Environments

Current Trends of Data-intensive Applications

Web 2.0 Big Data IoT

appl

icat

ion

dom

ains

&

requ

irem

ents

appl

icat

ion

arch

itect

ures

Infr

astr

uctu

res




appl

icat

ion

dom

ains

&

requ

irem

ents

appl

icat

ion

arch

itect

ures

infr

astr

uctu

res

performance scalability

elasticity availability










appl

icat

ion

dom

ains

&

requ

irem

ents

appl

icat

ion

arch

itect

ures

Infr

astr

uctu

res






appl

icat

ion

dom

ains

&

requ

irem

ents

appl

icat

ion

arch

itect

ures

infr

astr

uctu

res










appl

icat

ion

dom

ains

&

requ

irem

ents

appl

icat

ion

arch

itect

ures

infr

astr

uctu

res



https://www.gartner.com/en/documents/3941821/the-future-of-the-dbms-market-is-cloud

https://www.gartner.com/en/documents/3941821/the-future-of-the-dbms-market-is-cloud


Contribution

How to operate distributed DBMS in the cloud?

Insights in operating DBMS on cloud resources:

distributed DBMS impact factors

cloud resource impact factors

selected DBMS and cloud resource-centric evaluation results


Contribution

How to operate distributed DBMS in the cloud?

Insights in operating DBMS on cloud resources:

distributed DBMS impact factors


selected DBMS and cloud resource-centric evaluation results

the results summarizes the insights of a series of DBMS evaluation publications


Distributed DBMS Impact Factors



data models1 sharding & scale1,2 replication & consistency2

1Mazumdar, S., Seybold, D., Kritikos, K., & Verginadis, Y. (2019). A survey on data storage and placement methodologies for cloud-big data ecosystem. Journal of Big Data, 6(1), 15. 2Domaschka, J., Hauser, C. B., & Erb, B. (2014, September). Reliability and availability properties of distributed database systems. In 2014 IEEE 18th International Enterprise Distributed Object Computing Conference (pp. 226-233). IEEE.



data models sharding & scale replication & consistency

RDBMS

NewSQL

NoSQL




RDBMS

NewSQL

NoSQLcluster size

architecture

shardingmechanism




RDBMS

NewSQL

NoSQLcluster size

architecture

shardingmechanism

a*

a

consistency model replication

mechanism

replication factor




RDBMS

NewSQL

NoSQLcluster size

architecture

shardingmechanism

a*

a

consistency model replication

mechanism

replication factor

> 220 NoSQL & 20 NewSQL DBMS on the market1

1http://nosql-database.org/

http://nosql-database.org/



DBMS impact factors

cluster size

sharding

factor

consistency model

range hash

replication

ACID BASE scope

data model

RDBMS NewSQL NoSQL …

client-side consistency

architecture

single master-slave

multi-master

scalability – elasticity – availability performance

configurable

predefined


Cloud Resource Impact Factors



provider1 resource type2 resource characteristics3

1Baur, D., Seybold, D., Griesinger, F., Masata, H., & Domaschka, J. (2018, May). A provider-agnostic approach to multi-cloud orchestration using a constraint language. In Proceedings of the 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (pp. 173-182). IEEE Press.2Seybold, D., Hauser, C. B., Eisenhart, G., Volpert, S., & Domaschka, J. (2018, August). The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS. In European Conference on Parallel Processing (pp. 93-105). Springer, Cham.3Seybold, D., Hauser, C. B., Volpert, S., & Domaschka, J. (2017, October). Gibbon: An availability evaluation framework for distributed databases. In OTM Confederated International Conferences" On the Move to Meaningful Internet Systems" (pp. 31-49). Springer



provider1 resource type resource characteristics

1Baur, D., Seybold, D., Griesinger, F., Masata, H., & Domaschka, J. (2018, May). A provider-agnostic approach to multi-cloud orchestration using a constraint language. In Proceedings of the 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (pp. 173-182). IEEE Press.

public private

resource offerings



provider resource type resource characteristics

public

bare metal

private VMcontainer

storagesizing

resource offerings




bare metal

VMcontainer

storagesizing

interferences

failures

public private

resource offerings




bare metal

VMcontainer

storagesizing

interferences

failures

public private

resource offerings

> 20.000 public cloud resource offerings1

1https://cloudharmony.com/

https://cloudharmony.com/




characteristicsresource type

interferences failuresVM bare metal

provider

AWS … OpenStack container


configurable

predefined

storage

HDD SSD remote

sizing sizing


How to operate distributed DBMS in the Cloud?




configurable

predefined



range hashVM bare metal

provider

AWS …Open-Stack

container

storage

HDD SSD remote

DBMS impact factors

cluster size

sharding

factor

consistency model

range hash

replication

ACID BASE scope

data model

RDBMS

NewSQL

NoSQL …


architecture

master-slave

mult-master


Evaluation Scenario: Client-Consistency & Cluster Size


dynamic

static



range hashVM bare metal

provider

AWS …Open-Stack

container

storage

HDD SSD remote

DBMS impact factors

sharding

factor

consistency model

range hash

replication

ACID BASE scope

data model

RDBMS

NewSQL

NoSQL …


architecture

master-slave

mult-master

cluster size



evaluation environment

provider: OpenStack Ulmresource: VMsizing: 2 cores – 4GB memory – SSD storage

DBMS: Apache Cassandraversion: 3.11

Workload: YCSBType: write-heavy

complete evaluation details1

1Seybold, D., Keppler, M., Gründler, D., & Domaschka, J. (2019, April). Mowgli: Finding your way in the DBMS jungle. In Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (pp. 321-332). ACM.









consistency – performance impact









scalability





DBMS: Couchbaseversion: 5.0.1









DBMS: Couchbaseversion: 5.0.1





scalability


Lessons Learned: Client-Consistency & Cluster Size

minor changes to the DBMS configuration may have significant performance impact

scalability depends on (resources1), DBMS runtime configuration and workload properties



Evaluation Scenario: Resource Types


configurable

predefined



range hash

VM bare metal

provider

AWS …Open-Stack

container

storage

HDD SSD remote

DBMS impact factors

cluster size

sharding

factor

consistency model

range hash

replication

ACID BASE scope

data model

RDBMS

NewSQL

NoSQL …


architecture

master-slave

mult-master


Evaluation Scenario: Resource Types



DBMS: MongoDBversion: 3.6.3



1Seybold, D., Hauser, C. B., Eisenhart, G., Volpert, S., & Domaschka, J. (2018, August). The Impact of the Storage Tier: A Baseline Performance Analysis of Containerized DBMS. In European Conference on Parallel Processing (pp. 93-105). Springer, Cham.

physical

container

DBMS

physical

container

DBMS

physical

VM

DBMS

physical

VM

container

DBMS


Lessons Learned: Resource Types

virtualization reduces DBMS performance

storage location is an important and challenging decision for operating DBMS in the cloud

DBMS in containers on VM introduce neglectable overhead compared to operational benefits


Conclusion & Outlook


Conclusion

DBMS evaluation need to consider DBMS, cloud resource and workload characteristics

comprehensive DBMS evaluations are technically challenging, time consuming and error prone

Tool support is required!


Conclusion

DBMS evaluation need to consider DBMS, cloud resource and workload characteristics

comprehensive DBMS evaluations are technically challenging, time consuming and error prone

Tool support is required!

Mowgli Framework: fully automates DBMS evaluations and enables reproducible and portable evaluations!


Outlook

Advanced distributed DBMS evaluations: complex DBMS workloads – DBMS elasticity and availability – self-hosted DBMS vs. DBaaS

Automated DBMS operation in the cloud


Thank you!

The research leading to these results has received funding from the EC's Framework Programme HORIZON 2020 under grant agreement number 731664 (MELODIC) and 732667 (RECAP).

Mowgli Software:https://omi-gitlab.e-technik.uni-ulm.de/mowgli

Release 0.1:https://zenodo.org/record/3341512#.XcFnRehKiUk

DBMS Evaluation Data Sets:

Performance & Scalability:https://zenodo.org/record/3518786#.XcFnf-hKiUk

Elasticity:https://zenodo.org/record/3362279#.XcFnmehKiUk

containerized DBMS https://github.com/omi-uulm/Containerized-DBMS-Evaluation

https://omi-gitlab.e-technik.uni-ulm.de/mowgli

https://zenodo.org/record/3341512#.XcFnRehKiUk

https://zenodo.org/record/3518786#.XcFnf-hKiUk

https://zenodo.org/record/3362279#.XcFnmehKiUk

https://github.com/omi-uulm/Containerized-DBMS-Evaluation

towards understanding the performance of distributed ...€¦ · page 2 06.11.2019 | symposium on...

Documents