dr. chuckcartledgedr. chuckcartledgedr. chuckcartledge dr ...ccartled/teaching/2015... ·...
TRANSCRIPT
1/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CS-695 NoSQL DatabaseMongoDB (part 1 of 2)
Dr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck CartledgeDr. Chuck Cartledge
8 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 20158 Oct. 2015
2/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Table of contents I
1 Miscellanea
2 Data presentation
3 DB comparisons
4 Origins and history
5 Data model
6 CRUDy stuff
7 Assignment #4
8 Midterm
9 Conclusion
10 References
3/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Corrections and additions since last lecture.
Corrected typos in lecture006
Assignment #04 is available.
4/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Homework #3
How to present wide ranging data
These comments apply to all axis(x, y, z, . . . ):
All data is important
Range of values is such thatsmall values are “lost”
Small values are important
In example, max. Y value“swamps” rest of Y values.
Sample data from homework #3.
5/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Homework #3
Solution is to change how data are scaled.
Simple case is when all data is greaterthan 0. Only slightly more complexwhen there are 0s.
If there are 0 values, fudge themjust a little
Plot the logarithms of the data.
Add custom labels to the axis.
Inform the reader that the data isplotted using logarithmic scales.
Now large and small values are visible.
Sample data from homework #3.
A less obvious technique is to normalize data, can lead to confusing
interpretations.
6/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
How different DBs compare to a RDBMS
We have some terms to compare now[2]
RDBMS K/V Columnar Doc.
DB. instance cluster cluster instancedatabase — namespace —table bucket table collectionrow key-value row documentrowid key — idcol. — col. fam. —schema — — databasejoin — — DBRef
7/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Where it came from and why.
Initially developed in 2007as part of a “platform as aservice” project
Shifted to open source in2009
Currently used by Craigslist,eBay, and Foursquare
8/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Where it came from and why.
MongoDB’s view in the universe
Many things are outside ofMongoDB’s concern andcontrol
Administrators establish andmaintain an environment
Once a client has beenauthenticated, the client istrusted to “do the rightthing”
9/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
What is in the backend?
JSON is our friend.
Document databases storedocuments in the value partof a key-value store
Document databases havekey-value stores where thevalue part is examinable
The JSON attributes arewhere the data lives
Image from [2].
10/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
What is in the backend?
JSON is our friend.
11/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
What is in the backend?
Sharding and replications
Different types ofauto-sharding[1]:
Range-based according to ashard key value
Hash-based according toMD5 hash of shard key
Location-based controlled bythe administrator
Sharding partitions a very largedatabases the into smaller, faster,more easily managed parts.
12/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
What is in the backend?
Availability1
A mongoDB installation isavailable for most commonmodern operating systems andlanguages:
Various *nix packages
Windows after Windows XP
Mac OSX 10.6+
Language drivers availablefor: Java, .NET, Ruby, PHP,JavaScript, node.js, Python,Perl, PHP, Scala, andothers.
There is a mongoDB shell.
1http://docs.mongodb.org/manual/installation/
13/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Pre-CRUDy stuff
The mongoDB already exists
The database is static andavailable at all times
The collection is a group ofidentifiers
Anyone can create anddelete a collection
No security.
14/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Pre-CRUDy stuff
How to access things:
MongoDB shell:mongo
curl commands:It appears that the curlinterface isn’t working. Thismay be an unintentionalconsequence of mulitpleconflicting uses of apache onport 8080.
15/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CRUDy nuts and bolts
Create a collection2:
mongo {create: <collection name>,
capped: <true|false>,autoIndexId: <true|false>,size: <max size>,
max: <max documents>,
flags: <0|1|2|3>,storageEngine: <document>
}db.runCommand( { create: ‘‘collection’’, capped:
true, size: 64 * 1024 } )
To see the collection:show dbs
2http://docs.mongodb.org/manual/reference/command/create/,http://docs.mongodb.org/master/reference/mongo-shell/
16/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CRUDy nuts and bolts
Create data
All mongo shell commands are executed against/with the currentdatabase. The database “test” is created by default and alwaysavailable.To use the test database, execute: use test
db.restaurants.insert(
{"address" : {"street" : "2 Avenue",
"zipcode" : "10075",
"building" : "1480",
"coord" : [ -73.9557413, 40.7720266 ],
},"borough" : "Manhattan",
"cuisine" : "Italian",
"grades" : [
{"date" :
ISODate("2014-10-01T00:00:00Z"),
"grade" : "A",
"score" : 11
},{"date" :
ISODate("2014-01-16T00:00:00Z"),
"grade" : "B",
"score" : 17
}],
"name" : "Vella",
"restaurant id" : "41704620"
})
17/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CRUDy nuts and bolts
Find data
Find all the documents in the collection:db.restaurants.find()
Find selected documents in the collection:<field1>: <value1>, <field2>: <value2>, ...
db.restaurants.find( “borough”: “Manhattan” )
Note: Appending the .pretty() function to the commandimproves readability.
18/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CRUDy nuts and bolts
Update data
There is both a logical and functional difference between updatingan attribute and updating a document.db.restaurants.update(
{ "name" : "Juni" },{$set: { "cuisine": "American New" },$currentDate: { "lastModified": true } } )
db.restaurants.update(
{ "restaurant id" : "41156888" },{ $set: { "address.street": "East 31st Street" } }})db.restaurants.update(
{ "restaurant id" : "41704620" },"name" : "Vella 2",
"address" : {"coord" : [ -73.9557413, 40.7720266 ],
"building" : "1480",
"street" : "2 Avenue",
"zipcode" : "10075"
}})
19/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CRUDy nuts and bolts
Removing a document
db.restaurants.remove( { "borough": "Manhattan" } )
db.restaurants.remove( { "borough": "Queens" },{ justOne: true } )
db.restaurants.remove( { } )
db.restaurants.drop()
20/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
CRUDy nuts and bolts
Aggregations
mongoDB supports a wide range of data aggregation operators3.db.restaurants.aggregate(
[
{ $match: { "borough": "Queens", "cuisine":
"Brazilian" },{ $group: { " id": "$address.zipcode" ,
"count": { $sum: 1 } } }];
3http://docs.mongodb.org/manual/meta/aggregation-quick-reference/,http://docs.mongodb.org/manual/reference/sql-aggregation-comparison/,http://docs.mongodb.org/manual/core/aggregation-introduction/
21/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
Words of explanation.
The full text is available at:http://www.cs.odu.edu/
~ccartled/
Teaching/2015-Fall/NoSQL/
Assignments/04/
In general terms:
1 Parse data
2 Create document database
3 Update documents based onnumber of movies filmed in eachcity
4 Query database
5 Create list of 10 most usedlocations
6 Solve TSP with 10 cities
7 Plot route on a map
8 List locations and movies
22/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
General notes and ideas
It will be about things wecovered in class. Including:
Information and ideas
Variouslectures/presentations
Different databasetechnologies
Open book, open notes (notopen neighbor)
It will be about Sarah and Hasta.Sarah will have an idea for Hasta.Your task will be to discuss whichDB technology to use and why.
23/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
What have we covered?
Reviewed assignment #04Covered mongoDB CRUDYstuff
Next time: continued CRUDy exploration
24/24
Miscellanea Data presentation DB comparisons Origins and history Data model CRUDy stuff Assignment #4 Midterm Conclusion
References I
[1] Mongodb architecture guide, MongoDB, 2015.
[2] Eric Redmond and Jim R Wilson, Seven databases in seven weeks,Pragmatic Bookshelf, 2012.