python and mongodb
TRANSCRIPT
![Page 1: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/1.jpg)
![Page 3: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/3.jpg)
Agenda
Introduction to MongoDB pymongo CRUD Aggregation GridFS Indexes ODMs
![Page 4: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/4.jpg)
Ola, I'm Norberto
Norberto Leite Technical Evangelist !
Madrid, Spain @nleite [email protected] http://www.mongodb.com/norberto
![Page 5: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/5.jpg)
![Page 6: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/6.jpg)
MongoDB
![Page 7: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/7.jpg)
MongoDB
GENERAL PURPOSE DOCUMENT DATABASE OPEN-SOURCE
![Page 8: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/8.jpg)
Fully Featured
![Page 9: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/9.jpg)
MongoDB Features
JSON Document Model with Dynamic Schemas
Auto-Sharding for Horizontal Scalability
Text Search
Aggregation Framework and MapReduce
Full, Flexible Index Support and Rich Queries
Built-In Replication for High Availability
Advanced Security
Large Media Storage with GridFS
![Page 10: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/10.jpg)
MongoDB Inc.
400+ employees 2,000+ customers
Over $311 million in funding 13 offices around the world
![Page 11: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/11.jpg)
THE LARGEST ECOSYSTEM
9,000,000+ MongoDB Downloads
250,000+ Online Education Registrants
35,000+ MongoDB User Group Members
35,000+ MongoDB Management Service (MMS) Users
750+ Technology and Services Partners
2,000+ Customers Across All Industries
![Page 12: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/12.jpg)
pymongo
![Page 13: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/13.jpg)
pymongo
• MongoDB Python official driver • Rockstart developer team
• Jesse Jiryu Davis, Bernie Hackett • One of oldest and better maintained drivers • Python and MongoDB are a natural fit
• BSON is very similar to dictionaries • (everyone likes dictionaries)
• http://api.mongodb.org/python/current/ • https://github.com/mongodb/mongo-python-driver
![Page 14: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/14.jpg)
pymongo 3.0
!
• Server discovery spec • Monitoring spec • Faster client startup when connecting to Replica Set • Faster failover • More robust replica set connections • API clean up
![Page 15: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/15.jpg)
Connecting
![Page 16: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/16.jpg)
Connecting#!/bin/python from pymongo import MongoClient !mc = MongoClient()
client instance
![Page 17: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/17.jpg)
Connecting#!/bin/python from pymongo import MongoClient !uri = 'mongodb://127.0.0.1' mc = MongoClient(uri)
![Page 18: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/18.jpg)
Connecting#!/bin/python from pymongo import MongoClient !uri = 'mongodb://127.0.0.1' mc = MongoClient(host=uri, max_pool_size=10)
![Page 19: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/19.jpg)
Connecting to Replica Set#!/bin/python from pymongo import MongoClient !uri = ‘mongodb://127.0.0.1?replicaSet=MYREPLICA' mc = MongoClient(uri)
![Page 20: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/20.jpg)
Connecting to Replica Set#!/bin/python from pymongo import MongoClient !uri = ‘mongodb://127.0.0.1' mc = MongoClient(host=uri, replicaSet='MYREPLICA')
![Page 21: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/21.jpg)
Database Instance#!/bin/python from pymongo import MongoClient mc = MongoClient() !db = mc['madrid_pug'] !#or !db = mc.madrid_pug
database instance
![Page 22: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/22.jpg)
Collection Instance#!/bin/python from pymongo import MongoClient mc = MongoClient() !coll = mc[‘madrid_pug’]['testcollection'] !#or !coll = mc.madrid_pug.testcollection
collection instance
![Page 23: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/23.jpg)
CRUD
http://www.ferdychristant.com/blog//resources/Web/$FILE/crud.jpg
![Page 24: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/24.jpg)
Operations
• Insert • Remove • Update • Query • Aggregate • Create Indexes • …
![Page 25: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/25.jpg)
CRUD
• Insert • Remove • Update • Query • Aggregate • Create Indexes • …
![Page 26: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/26.jpg)
Insert#!/bin/python from pymongo import MongoClient mc = MongoClient() !coll = mc['madrid_pug']['testcollection'] !!coll.insert( {'field_one': 'some value'})
![Page 27: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/27.jpg)
Find#!/bin/python from pymongo import MongoClient mc = MongoClient() !coll = mc['madrid_pug']['testcollection'] !!cur = coll.find_one( {'field_one': 'some value'}) !for d in cur: print d
![Page 28: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/28.jpg)
Update#!/bin/python from pymongo import MongoClient mc = MongoClient() !coll = mc['madrid_pug']['testcollection'] !!result = coll.update_one( {'field_one': 'some value'}, {"$set": {'field_one': 'new_value'}} ) #or !result = coll.update_many( {'field_one': 'some value'}, {"$set": {'field_one': 'new_value'}} ) !print(result) !
![Page 29: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/29.jpg)
Remove#!/bin/python from pymongo import MongoClient mc = MongoClient() !coll = mc['madrid_pug']['testcollection'] !!result = coll.delete_one( {'field_one': 'some value’}) !#or !result = coll.delete_many( {'field_one': 'some value'}) !print(result) !
![Page 30: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/30.jpg)
Aggregate
http://4.bp.blogspot.com/-‐0IT3rIJkAtM/Uud2pTrGCbI/AAAAAAAABZM/-‐XUK7j4ZHmI/s1600/snowflakes.jpg
![Page 31: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/31.jpg)
Aggregation Framework
• Analytical workload solution • Pipeline processing • Several Stages
• $match • $group • $project • $unwind • $sort • $limit • $skip • $out
!
• http://docs.mongodb.org/manual/aggregation/
![Page 32: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/32.jpg)
Aggregation Framework#!/bin/python from pymongo import MongoClient mc = MongoClient() !coll = mc['madrid_pug']['testcollection'] !!cur = coll.aggregate( [ {"$match": {'field_one': {"$exists": True }}} , {"$project": { "new_label": "$field_one" }} ] ) !for d in cur: print(d)
![Page 33: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/33.jpg)
GridFS
http://www.appuntidigitali.it/site/wp-‐content/uploads/rawdata.png
![Page 34: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/34.jpg)
GridFS
• MongoDB has a 16MB document size limit • So how can we store data bigger than 16MB? • Media files (images, pdf’s, long binary files …)
• GridFS • Convention more than a feature • All drivers implement this convention
• pymongo is no different • Very flexible approach • Handy out-of-the-box solution
![Page 35: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/35.jpg)
GridFS#!/bin/python from pymongo import MongoClient import gridfs !!mc = MongoClient() database = mc.grid_example !!gfs = gridfs.GridFS( database) !read_file = open( '/tmp/somefile', 'r') !gfs.put(read_file, author='Norberto', tags=['awesome', 'madrid', 'pug'])
call grids lib w/ database
![Page 36: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/36.jpg)
GridFS#!/bin/python from pymongo import MongoClient import gridfs !!mc = MongoClient() database = mc.grid_example !!gfs = gridfs.GridFS( database) !read_file = open( '/tmp/somefile', 'r') !gfs.put(read_file, author='Norberto', tags=['awesome', 'madrid', 'pug'])
open file for reading
![Page 37: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/37.jpg)
GridFS#!/bin/python from pymongo import MongoClient import gridfs !!mc = MongoClient() database = mc.grid_example !!gfs = gridfs.GridFS( database) !read_file = open( '/tmp/somefile', 'r') !gfs.put(read_file, author='Norberto', tags=['awesome', 'madrid', 'pug'])
call put to store file and metadata
![Page 38: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/38.jpg)
GridFSmongo nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show dbs grid_sample 0.246GB local 0.000GB nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show collections fs.chunks 258.995MB / 252.070MB fs.files 0.000MB / 0.016MB
database created
![Page 39: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/39.jpg)
GridFSmongo nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show dbs grid_sample 0.246GB local 0.000GB nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show collections fs.chunks 258.995MB / 252.070MB fs.files 0.000MB / 0.016MB 2 collections
![Page 40: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/40.jpg)
GridFSmongo nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show dbs grid_sample 0.246GB local 0.000GB nair(mongod-‐3.1.0-‐pre-‐) grid_sample> show collections fs.chunks 258.995MB / 252.070MB fs.files 0.000MB / 0.016MB
chunks collection holds binary data
files holds metada data
![Page 41: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/41.jpg)
Indexes
![Page 42: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/42.jpg)
Indexes
• Single Field • Compound • Multikey • Geospatial
• 2d • 2dSphere - GeoJSON
• Full Text • Hash Based • TTL indexes • Unique • Sparse
![Page 43: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/43.jpg)
Single Field Indexfrom pymongo import ASCENDING, MongoClient mc = MongoClient() !coll = mc.madrid_pug.testcollection !coll.ensure_index( 'some_single_field', ASCENDING )
indexed field indexing order
![Page 44: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/44.jpg)
Compound Field Indexfrom pymongo import ASCENDING, DESCENDING, MongoClient mc = MongoClient() !coll = mc.madrid_pug.testcollection !coll.ensure_index( [('field_ascending', ASCENDING), ('field_descending', DESCENDING)] )
indexed fields indexing order
![Page 45: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/45.jpg)
Multikey Field Indexmc = MongoClient() !coll = mc.madrid_pug.testcollection !!coll.insert( {'array_field': [1, 2, 54, 89]}) !coll.ensure_index( 'array_field')
indexed field
![Page 46: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/46.jpg)
Geospatial Field Indexfrom pymongo import GEOSPHERE import geojson !!p = geojson.Point( [-73.9858603477478, 40.75929362758241]) !coll.insert( {'point', p) !coll.ensure_index( [( 'point', GEOSPHERE )])
index type
![Page 47: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/47.jpg)
ODM and others
![Page 48: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/48.jpg)
Friends
• mongoengine • http://mongoengine.org/
• Motor • http://motor.readthedocs.org/en/stable/ • async driver • Tornado • Greenlets
• ming • http://sourceforge.net/projects/merciless/
![Page 49: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/49.jpg)
Let's recap
![Page 50: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/50.jpg)
Recap
• MongoDB is Awesome • Specially to work with Python
• pymongo • super well supported • fully in sync with MongoDB server
![Page 51: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/51.jpg)
MongoDB 3.0 is here!
![Page 52: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/52.jpg)
Go and Play!
https://www.mongodb.com/lp/download/mongodb-‐enterprise?jmp=homepage
http://www.mongodb.com/norberto
![Page 54: Python and MongoDB](https://reader033.vdocuments.site/reader033/viewer/2022042615/55c2aadabb61eb86508b4761/html5/thumbnails/54.jpg)