no sql - { if and else }
TRANSCRIPT
NoSqlFuture begins here
Author: Marjan Nikolovski
Date: 2011-02-03
Content
NoSql as movementPros and ConsCategoriesMongo DB showcaseDiscussion
NoSql as movement
“NoSQL is a term used to designate databases that differ from classic relational databases in some way”
SchemalessNo “joins”
NoSql as movement
Schemaless – means everything can be stored without order, type, structure
No “joins” – means normalization is not used to relate and retrieve the data
NoSql as movement
Invented to cover what RDBMS could not◦Fast write/read under heavy usage of a system◦Highly distribution of data◦Better architecture of distributed systems◦Fun
Cap Theorem
CAP Theorem !!!“It is impossible for a distributed computer system to
simultaneously provide all three of the following guarantees:
-Consistency (all nodes see the same data at the same time)
-Availability (node failures do not prevent survivors from continuing to operate)
-Partition Tolerance (the system continues to operate despite arbitrary message loss)
According to the theorem, a distributed system can satisfy any two of these guarantees at the same time, but not all three”
Cap Theorem - 2
Pros and Cons - 2
Complex objects High-Volume Low-Value data Real-time analysis Online games (MMRPG) Search engines Scalable horizontally(sharding) Redundancy Flexibility Faster development Geospatial
Transactions and distributed transactions Business Intelligence
PROS
CONS
Pros and Cons
What are we architecting ???What are our options ???Result …
Categories
Under NoSql four main categories exists:◦Key / Value◦BigTable◦Document◦Graph
Categories – Key / Value
We can see on this type as distributed hash table
Used for caching layersQuerying is done by key onlyHighly distributiveProducts:
◦Memcached◦Project Voldemort◦Amazon Dynamo
Categories - BigTable
BigTable are step forward from Key/Value stores
There is some order in these stores and querying can be done by more params
More like the RDBMS that we now with excluded foreign key concept and with support each row to have different columns
Products:◦Apache Cassandra◦Apache HBase
Categories – Document
Documents stores are json/xml based stores for objects
Highly queryableProducts
◦MongoDB◦RavenDB◦Apache CouchDB
Categories – Graph
Graph stores store data in vortex connected with zero or more vortexes.
Data is held in hashtable in each nodeRich objects are not supportedProducts
◦Neo4j◦AllegroGraph◦InfoGrid
Categories – size/complexity*Complexity = Sophistication of data modeling
Mongo DB showcase - 1
What is◦Document store◦Fast, scalable and available◦JSon – used for data hydratation◦Dynamic◦Stores data structured as documents instead of
row as seen in RDBMS◦Uses Query, Insert, Update and Remove for
data manipulation◦Used for adding data at high rates without
“choking” the system
Mongo DB showcase - 2
Internals of Core
◦ Map files in memory (virtual)◦ Memory is allocated by namespaces ◦ Free space comes at the end of the structure◦ Each document contains a header for traversal and
position, body (document data) and paddingReplication
◦ Master/Slave and Replication sets architecture◦ Both require Master but Replication set vote for new
master if the one dedicated for the job is unavailableCursors
◦ Results can be of type cursor which can be iterated until cursor have 0 elements left
Mongo DB showcase - 3
Documents in◦ Each document can be of size max to 4mb◦ Each document you insert into MongoDB will be assigned an
id consisted of: This is a 12-byte value consisting of 4 parts:
timestamp (4 bytes) machine identifier (3 bytes) process id (2 bytes) increment (3 bytes)
◦ in MongoDB, each document will contain not only the values, but the key names (~"column names" in relational db-speak) too. So if you have keys: “Username" and “Password", then those alone will add 16 bytes to each document.
◦ MongoDB automatically adds some padding to the documents to allow for documents to grow in size to try and reduce the need for documents to be moved around if they do grow
Mongo DB showcase - 4
Map reduce and◦Map/Reduce is done via javascript
Mongo DB showcase - 5
Simple usage◦On win32 start mongod.exe◦Start mongo.exe test client for testing
db.Workers.insert({_workerId: “936DA01F-9ABD-4d9d-80C7-
02AF85C822A8”,name: “XyZ”,
department: “services”, subdepartment: “.NET”});
db.Workers.find({ name: “XyZ” });
Mongo DB showcase - 6
Performance analyses
Gets Keys i1 r1 i2 r2 i3 r3 Average read Memory (kb) Memory (mb)
30000 10000 0.66 16.77 0.66 16.59 0.57 17.05 16.80333333
30000 100000 5.77 16.88 5.82 16.8 5.86 16.91 16.86333333 559776 546.65625
30000 1000000 58.08 19.97 66.42 17.28 62.45 16.63 17.96 1712296 1672.164063
30000 3000000 178.74 17.57 194.34 17.26 16.69 17.17333333 4257796 4158.003906
30000 10000000 1102.78 73.09 75.31 73.88 74.09333333 7648124 7468.871094
30000 10000000 54.34 36.9 29.49 40.24333333
30000 10000000 21.12 18.78 18.16 19.35333333
Notes r1, r2, r3 are 3 read samples
i1, i2, i3 are insert samples
Mongo DB showcase - 7
So how can we use it in .NET ???
Mongo DB showcase - 8
Get Mongodb-csharp driver◦ http://github.com/samus/mongodb-csharp/commits/master/
Let’s start
Suggestions
Check:◦ The high scalability blog◦ Apache Cassandra◦ RanvenDB, .Net implementation of document
store◦ mysql performance blog◦CAP theorem
Discussion
Sponsored by