i have a nosql toaster - connectjs - october 2016
TRANSCRIPT
I have a NoSQL Toaster.Why would I want a NoSQL database?
Matthew D. Groves @mgroves
Original slides by Matthew Revell - @matthewrevell
©2016 Couchbase Inc. 2
Where am I?
• Connect.Tech
• http://connect-js.com
• http://twitter.com/connect_js
©2016 Couchbase Inc. 3
Who am I?
• Matthew D. Groves
• Developer Advocate for Couchbase
• @mgroves on Twitter
• Podcast and blog: http://crosscuttingconcerns.com
• “I am not an expert, but I am an enthusiast.” – Alan Stevens
©2016 Couchbase Inc. 4
Tweet something, get a sticker!
Use #Couchbase
Tweet something interesting you’ve learned.Tweet a picture.
I’ve got stickers and a prize for the best tweet!
©2016 Couchbase Inc. 5
Couchbase and CouchDB
Couchbase is to CouchDBas
MySQL is to SQL Server
©2016 Couchbase Inc. 6
SQL is the norm, right?
https://commons.wikimedia.org/wiki/File:George_Wendt_at_the_41st_Emmy_Awards_cropped.jpg
~3200 BC: invention of writing in Mesopotamia
~300 BC: Library of Alexandria
~1041 AD: Movable type invented in China
1450: Movable type invented in Europe
1822: Babbage’s paper on the application of difference engines
1943: Colossus, the first programmable digital computer
1957-1960: IBM SABRE, the first commercial use of a database
1970: EF Codd proposes the relational database model
1974: Ingres released
1979: Commercial launch of Oracle database
1988: Object databases appear
1989: Microsoft SQL Server version 1
1991: HTTP v0.9
1995: MySQL’s initial release
2005: CouchDB released
2006: Google’s Big Table paper
2007: Amazon’s Dynamo paper
2008-2009: The NoSQL Cambrian explosion: Riak, MongoDB, Cassandra, Redis
2010: Couchbase arrives on the scene
NoSQL isn’t a very useful term
©2016 Couchbase Inc. 9
Easy?
©2016 Couchbase Inc. 10
Scalable?
https://commons.wikimedia.org/wiki/File:Dagwood_sandwich.jpg
©2016 Couchbase Inc. 11
Commodity Hardware
https://commons.wikimedia.org/wiki/File:Rear_of_rack_at_NERSC_data_center_-_closeup.jpg
©2016 Couchbase Inc. 12
ACID vs BASE?
https://commons.wikimedia.org/wiki/File:PH_Scale.svg
AtomicConsistentIsolatedDurable
Basic AvailabilitySoft-stateEventual consistency
©2016 Couchbase Inc. 13
Schema Schemaless
https://en.wikipedia.org/wiki/File:Star-schema-example.png
©2016 Couchbase Inc. 14
Normalized Denormalized
https://commons.wikimedia.org/wiki/File:Frown.JPG
©2016 Couchbase Inc. 15
First: Why NoSQL?
https://www.flickr.com/photos/wonderlane/13067014944
©2016 Couchbase Inc. 16
Round pegs, square holes
http://www.timesofbook.com/2013/09/blobfish-image-gallery.html#.V6iXwbgrJhE
©2016 Couchbase Inc. 17
Perhaps SQL is right for this project
©2016 Couchbase Inc. 18
Scaling can be hard
https://commons.wikimedia.org/wiki/File:Sandwiches_Vienna.jpg
©2016 Couchbase Inc. 19
Availability is hard
https://www.flickr.com/photos/jeepersmedia/14562425273/
CAP
©2016 Couchbase Inc. 20
Schemas can be hard
https://commons.wikimedia.org/wiki/File:MediaWiki_database_schema_1-17_(r82044).svg
©2016 Couchbase Inc. 21
Types of NoSQL databases
Document
Key/Value
Columnar
Graph
©2016 Couchbase Inc. 22
Convergence!
©2016 Couchbase Inc. 23
Document
???
©2016 Couchbase Inc. 25
What makes it a document database?
JSON
XML
BSON
©2016 Couchbase Inc. 28
The document database
understands the
format of the document
©2016 Couchbase Inc. 29
It can do
server-side stuff
Buckets
Documents
Views
N1QL
"ClientProductCode","Title","Quantity""TShirtCBS","Couchbase logo t-shirt (small)",29"TShirtCBM","Couchbase logo t-shirt (medium)",72"TShirtCBL","Couchbase logo t-shirt (large)",34"TShirtCBXL","Couchbase logo t-shirt (extra large)",12"StickerCBLogo","Couchbase logo sticker",256"PenCBLogo","Couchbase logo pen",365
class Program{
private static IBucket _bucket;
static void Main(string[] args){
SetupCouchbase();
LoadStockList();
ClusterHelper.Close();}
private static void SetupCouchbase(){
ClientConfiguration config = new ClientConfiguration();config.Servers = new List<Uri> {new Uri("couchbase://192.168.1.5") };ClusterHelper.Initialize(config);_bucket = ClusterHelper.GetBucket("default");
}
private static void LoadStockList(){
var csv = new CsvReader(File.OpenText("swag.csv"));while (csv.Read()){
var record = csv.GetRecord<dynamic>();var document = new Document<dynamic>{
Id = record.ClientProductCode,Content = new{
record.Title,record.Quantity
}};_bucket.Insert(document);Console.WriteLine($"Added document '{record.ClientProductCode}'");
}}
}
private static void SetupCouchbase(){
ClientConfiguration config = new ClientConfiguration();config.Servers = new List<Uri> {new Uri("couchbase://192.168.1.5") };ClusterHelper.Initialize(config);_bucket = ClusterHelper.GetBucket("default");
}
private static void LoadStockList() {var csv = new CsvReader(File.OpenText("swag.csv"));while (csv.Read()) {
var record = csv.GetRecord<dynamic>();var document = new Document<dynamic> {
Id = record.ClientProductCode,Content = new {
record.Title,record.Quantity
}};_bucket.Insert(document);Console.WriteLine($"Added document '{record.ClientProductCode}'");
}}
private static void GetDocument(){
var doc = _bucket.Get<dynamic>("StickerCBLogo");Console.WriteLine($"Document Key: {doc.Id}");Console.WriteLine($"Title: {doc.Value.title}");Console.WriteLine($"Quantity: {doc.Value.quantity}");
}
©2016 Couchbase Inc. 38
Typical Document Database Operations
• Upsert (aka set)• Inserts document if key doesn’t exist, replaces otherwise
• Insert (aka add)• Inserts document, error if it already exists
• Update (aka replace)• Updates document, error if it doesn’t exist
• Delete
• Get
©2016 Couchbase Inc. 39
N1QL
SELECT m.*
FROM `default` m
WHERE META(m).id = ‘StickerCBLogo’
SELECT m.*
FROM `default` m
WHERE m.quantity > 10
Great for Dev
Scales Easily
Consistently Fast
©2016 Couchbase Inc. 41
Use cases for Document Databases
• User profiles
• Session stores
• Content management
• General relational replacement• http://info.couchbase.com/15Q1TopTenUC.html
©2016 Couchbase Inc. 42
Key/Value Database
©2016 Couchbase Inc. 44
Key-Value Databases
In-memory caches
and
distributed data stores
©2016 Couchbase Inc. 45
Doesn’t care what your data is (mostly)
KEY1 { type: “json” }
KEY2 <data type=“xml”></data>
KEY3 Lorem ipsum
… 00100101000100101
KEYN
https://www.flickr.com/photos/dcmot/23168680069
Buckets
Key-Value Pairs
©2016 Couchbase Inc. 47
C# Riak
private static void DoStuffWithRiak(){
var cluster = RiakCluster.FromConfig("riakConfig");var client = cluster.CreateClient();
var actor = new RiakObject("actors", "James Bond","Sean Connery");client.Put(actor);
var actorBackOut = client.Get("actors", "James Bond");var str = System.Text.Encoding.UTF8.GetString(actorBackOut.Value.Value);Console.WriteLine($"Value: {str}");
}
©2016 Couchbase Inc. 49
Introducing: Eventual Consistency!
©2016 Couchbase Inc. 50
Eventual Consistency
https://www.flickr.com/photos/scottchene/7046992749https://commons.wikimedia.org/wiki/File:Timothy_Dalton_1987.jpghttps://commons.wikimedia.org/wiki/File:Sir_Roger_Moore_crop.jpg
©2016 Couchbase Inc. 51
There can be only one!
https://en.wikipedia.org/wiki/File:Highlander_film_Connor_MacLeod.jpg
©2016 Couchbase Inc. 52
So, fix it
https://www.flickr.com/photos/94086507@N00/84368105
Scales Well
Highly Available
Data Agnostic
Maybe some extra work
©2016 Couchbase Inc. 54
Use Cases for K/V
• Data variability
• Object caching
• Session storage
• Large object storage
©2016 Couchbase Inc. 55
Columnar
Keyspaces
Column Families
Rows
Columns
©2016 Couchbase Inc. 58
Keyspace
©2016 Couchbase Inc. 59
Column Family
CREATE KEYSPACE mykeyspaceWITH replication = { 'class': 'SimpleStrategy',
'replication_factor': '1'};
CREATE TABLE users (firstname text, lastname text, age int, email text, city text, PRIMARY KEY (lastname));
private static void DoStuffWithDataStax(){
var cluster = Cluster.Builder().AddContactPoint("127.0.0.1").Build();var session = cluster.Connect("mykeyspace");
session.Execute("insert into users (lastname, age, city, email, firstname) values ('Jones', 35, 'Austin', '[email protected]', 'Bob')");
var userBackOut = session.Execute("select * from users wherelastname='Jones'").First();
Console.WriteLine($"{userBackOut["firstname"]} {userBackOut["age"]}");}
Impedance Mismatch
Scales Well (Ops Headache)
Suited to Analytics
©2016 Couchbase Inc. 64
Use cases for Columnar
•Metering data•Really big data•Analytics
©2016 Couchbase Inc. 65
Graph
https://commons.wikimedia.org/wiki/File:The_protein_interaction_network_of_Treponema_pallidum.png
©2016 Couchbase Inc. 66
Leonhard Euler
©2016 Couchbase Inc. 67
7 Bridges of Königsberg
©2016 Couchbase Inc. 69
Use cases for Graph databases
• Social networks / dating sites
• Fraud detection
• Parcel routing
• Shopping recommendations
©2016 Couchbase Inc. 70
Say what?
RelationalDocument
Key/Value
Columnar
GraphObject
others…
©2016 Couchbase Inc. 71
Box arrow box arrow
Web LogicSQL
Riak
Couchbase
More Logic
Mobile
©2016 Couchbase Inc. 72
Couchbase, everybody!
http://couchbase.com/connect
http://info.couchbase.com/Connect16_Livestream_Registration.html
©2016 Couchbase Inc. 73
Where do you find us?
• developer.couchbase.com
• blog.couchbase.com
• forums.couchbase.com
• @couchbasedev
• #couchbase
• @mgroves
• @couchbase