gain a mongodb advantage with the percona memory engine...• data from the mongodb query profiler...

©2017Percona1

VadimTkachenko

GainaMongoDBAdvantagewiththePerconaMemoryEngineWebinarDecember6th,2017

CTO,Percona

©2017Percona2

Me• CTOatPercona

• LeadingSoftwareEngineeringatPercona

• Technology&(database,hardware,filesystems)PerformanceEnthusiast

• Currentareasofinterest:MySQL,MongoDBandClickHouse

• @VadimTk

©2017Percona3

MemoryEngine• https://www.percona.com/software/mongo-database/percona-

memory-engine-for-mongodb

• StorageEngineinPerconaServerForMongoDB• BasedonwiredTiger• butwithoutwritestothestorage• HighWriteThroughput• HighReadThroughput• Predictable,lowlatencies

©2017Percona4

MemoryEngineusecases

©2017Percona5

ApplicationcacheReplaceservicessuchasmemcached andcustomapplication-leveldatastructureswiththefullpowerofMongoDBfeatures.

©2017Percona6

Typicalarchitecture

Databaselayer(MongoDB,MySQL,etc)

Cachinglayer(memcached or

Redis)

Key-valuestoreLimitedcapabilities

©2017Percona7

Betterway

Databaselayer(MongoDB,MySQL,etc)

CachinglayerPerconaMemory

Engine

• Documentstore• Flexiblequerylanguage• Replication/sharding• FamiliarMongoDBdeployment

©2017Percona8

SessionmanagementDecreaseapplicationresponsetimesbykeepingactiveusersessionsinmemory.

©2017Percona9

TransientruntimestateStoreapplicationstateful runtimedatathatdoesn'trequireon-diskstorage.

©2017Percona10

Multi-tierobjectsharingFacilitatesharingofdatainmulti-tier/multi-languageapplications.

©2017Percona11

ApplicationtestingReduceturnaroundtimeforautomatedapplicationtests.

©2017Percona12

SophisticateddatamanipulationIncreaseperformancefordatamanipulationoperationssuchasaggregationandmapreduction.

©2017Percona13

Real-timeAnalyticsUsesin-memorycomputinginsituationswhereresponsetimeismorecriticalthanpersistence.

©2017Percona14

MemoryEnginedeployments

©2017Percona15

•  Fortestinganddevelopment•  Usefulforsizingandquickresets•  NoRedundancy•  NoPersistence•  Limitedtoserversize

Standalone

PSMDB Memory

©2017Percona16

./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150

Standaloneconfig

PSMDB Memory

©2017Percona17

•  Forproduction•  Failovercapability•  Dataredundancy•  NoPersistence•  Readscaling

ReplicaSet- Memory

PSMDB Memory

primary secondary

secondary

PSMDB Memory

PSMDB Memory

©2017Percona18

ReplicaSet- config

PSMDB Memory

primary secondary

secondary

PSMDB Memory

PSMDB Memory

./mongod --storageEngine inMemory -–dbpath /data/db -–inMemorySizeGB 150 --replSet repSet1

©2017Percona19

•  Forproduction•  Failovercapability•  Dataredundancy•  DataPersistence•  Readscaling•  Configurepermanentmemberasahiddenmember(i.e.hidden:trueandpriority:0)

ReplicaSet– withPersistentOption

PSMDB Memory

primary secondary

secondary

PSMDB Memory

PSMDB wiredTiger or MongoRocks

Permanentstorage

©2017Percona20

Replicatesetwithpermanentoption• Onlythemongod instancesrunningwiththeMemorystorage

enginecanbecometheprimary.• ClientsconnectonlytotheMemorystorageenginemongod

instances.• Evenifbothmongod instancesrunningMemorystorageengine

crashandrestart,theycansyncfromthememberrunningpermanentstorageengine.

• Thehiddenmongod instancerunningwithpermanentstoragepersiststhedatatodisk,includingtheuserdata,indexes,andreplicationconfigurationinformation.

©2017Percona21

Sharding – scalingformorememory

PSMDB Memory PSMDB Memory

mongos

Shard by id

©2017Percona22

Sharding – scalingmongos

Shard by id

PSMDB Memory

primary secondary

secondary PSMDB Memory

PSMDB wiredTiger or MongoRocks

PSMDB Memory

primary secondary

secondary PSMDB Memory

PSMDB wiredTiger or MongoRocks

•  Forproduction•  Failovercapability•  Dataredundancy•  DataPersistence• Write scaling

©2017Percona23

PSMDBextras

©2017Percona24

•  Dataretrievalbydocumentfields

PSMDB– secondaryindexes{"_id“ : “Vadim”,“visits“ : 45,”created“ : ISODate("2016-01-07T15:46:32.085Z”)

}

> db.coll.ensureIndex({visits:1});> db.coll.find({visits : {$gte :40}});

©2017Percona25

•  Toremovedataautomatically

PSMDB– TTLindexes{"_id“ : “Vadim”,“visits“ : 45,”created“ : ISODate("2016-01-07T15:46:32.085Z”)

}

> db.coll.createIndex({"created": 1},{expireAtferSeconds: 7200 });

©2017Percona26

Morefeatures• Authentication• IncludingExternalSASLAuthentication• Role-BasedAccessControl• Auditlogging• GeospatialIndexes• TextSearch

©2017Percona27

PSMDBMemoryEngine- summary

StoreType DocumentStore

Failover/HA ReplicaSet

Scaling Sharding

Persistence ExtraReplicawithwiredTiger orMongoRocks

CRUD Insert,find,update,remove

Expiredata TTL

Concurrency Multi-threading

©2017Percona28

Scalability– MultiThreading• Intel(R)Xeon(R)CPUE5-2683v3@2.00GHzwith28cores/56

threads• 256GBRAM• Sysbench 1.0witholtp_mongo.lua• Availableathttps://github.com/Percona-Lab/sysbench-mongodb-lua• 100GBdataset• Userthreadsfrom1to128• Operation:OLTPevent

©2017Percona29

©2017Percona30

Stability– MultiThreading• Intel(R)Xeon(R)CPUE5-2683v3@2.00GHzwith28cores/56

threads• 256GBRAM• Sysbench 1.0witholtp_mongo.lua• Availableatgithub.com/percona-labs• 100GBdataset• Userthreads256• Operation:OLTPevent

©2017Percona31

©2017Percona32

©2017Percona33

Predictableresponsetime130ms(withpracticallynovariation)256userthreads

©2017Percona34

MonitoringwithPerconaMonitoringandManagement(PMM)

©2017Percona35

PMM• NewsoftwarefromPercona• Freeandopen-sourceplatformformanagingandmonitoring

MySQL® andMongoDB® performance.• YoucanrunPMMinyourownenvironment• Providesthoroughtime-basedanalysisforMySQL,MariaDB® and

MongoDBserverstoensurethatyourdataworksasefficientlyaspossible.

©2017Percona36

PMMbenefits• Providesqueryandmetricinformationthatenables

administratorstooptimizedatabaseperformance• Displayscurrentqueriesandhighlightspotentialqueryissuesto

enablefasterissueresolution• Mapsqueriesagainstmetricstohelpmakeinformeddecisions

aboutcrucialdatabaseresources:platformneeds,systemgrowth,teamfocusandthemostimportantdatabaseactivities

©2017Percona37

PMMfeatures• Point-in-timevisibilityandhistoricaltrendingofdatabase

performance• DatafromtheMongoDBqueryprofiler• SpecializedMongoDBdashboardsforgraphingMMAPv1,InMemory,

MongoRocks• Ahistoricalviewofmetricsthatarecriticaltoadatabaseserver• Querymetrics,includingbytessent,locktime,rowssent,andmore• Best-of-breedtools,includingGrafana,Prometheus,andConsul,as

wellasPercona-developedqueryanalytics,administration,API,agentandexportercomponents

• Asingle,easytomanagevirtualappliance

©2017Percona38

©2017Percona39

©2017Percona40

PMM– QueryAnalytics

©2017Percona41

Questions?