Scaling the server side of

occasionally-connected,

mobile systems with MongoDB

Thomas Huber - Fertl EDV-Systeme

MongoBerlin

October 4th

source: istockphoto.com/epicurean

Introduction Occasionally Connected Systems

• For e. g. Exchange Server & Outlook

• Every user has an Inbox, Calendar …..

• If you get a new Laptop – install Outlook and synch your Exchange account (synch takes time)

• After synching it is possible to go offline and read all mails offline

• booorrriing -> what about DB replication?

Definitions around Replication

• Lazy / Optimistic Replication <-> Eager ReplicationEager Replication – all nodes online are in synch, the transaction-steps are atomic Optimistic Replication – changes are logged, if the node is online, all changes are synched with other nodes, if not online, the node will synch later on. Conflicts are possible.

• Master-Slave <-> Peer-to-Peer One master, one or more slaves, not symmetric, all synchronization through the master (no Slave-to-Slave-Synch) Peer-to-Peer – all instances are talking with each other

Here is our solution

• Lazy / Optimistic Replication Smartphones can be out of coverage – offline functionality for mobile appsrobust in terms of going out of coverage and coming backlost data package detection

• Master-SlaveOne webserver / webportal, the user is able to change data on the smartphone and in the webportal, data will be synched

• One Security-Credential for Web-Portal & SmartphoneAuthentication (Login/Pwd) and Authorization (Permissions) are shared

• But keep in mind -> Exchange Server & OutlookData in the „inbox“ is synched whith the smartphonenot all data of all users

• It is possible to connect 990 different mobile devices to one user account (mixed scenarios - Symbian, iPhone, Android)

Mobile-Assistance-Framework

WTF – WHERE IS MONGODB????!!!!!!

• To scale this replication-scenario you need cheap power

• 1996 - Microsoft published a scientific paper about big replicationscenarios – one among many „The dangers of replication and a solution”... a ten-fold increase in nodes and traffic gives a thousand foldincrease in deadlocks or reconciliations ...

• It is not that ugly ;)

• But mobile apps will change the characteristics in typical load scenarios – think about more writes – think about longer transactions – because of GPRS

http://blog.appstorehq.com/post/1212703421/mashable-crushed-us-heres-how-we-bounced-back

Mongo helps in two Dimensions

• Think about 6-channel-ECG – 200Hz 16bit resolution35MByte per person per day -> 12GB per year

• 10.000 participants one year -> 120TB

• First dimension – the size -> with every new shard MongoDB is able save more data

• Second dimension – ops per seconds scales with every new shard

• if the problem is equally distributed over the machines the number of ops per second scales (it is very important to chose the right parameter)

Custom Development – maximizing the things covered in the framework

Hybrid – MongoDB and SQL

Behind the scenes

We are experimentingwith the ratio

WebServer______________ MongoMachines

Put things on the right track

• The portal is interlinked with the SQL Server (stored procedures)

• The important functions have good caching mechanisms

• Even 5 Billion User Credentials is not a real problem for a good SQL Server

• If Data grows per User per day,MongoDB helps

Easiest NoSQL-Engine (as far as I know)to port from SQL to MongoDB

• But there are no joins – so we had to implement it in our App-Server

• Compared to CassandraDB - MongoDB is much easier to query but not that easy to up-scale – but this is not a problem for us

• Easy to query http://www.mongodb.org/display/DOCS/Advanced+Queries

Thank you for your attention!More informations here:

www.mhealth-it.com

www.crossgeneration.info

I‘m presenting at the CloudConf

November 18th