building a location-based platform with mongodb from zero
DESCRIPTION
TRANSCRIPT
Building a Location-based platform
with MongoDB from Zero
Raviteja DoddaCo-Founder & CTO, Pipal Tech Ventures
Oct 26th 2012
MongoDB Bangalore Conference
• Offers, New Arrivals & Rewards from 250 + brands and 12,000 + stores across the country.
DelightCircle
• Discover what’s nearby, Engage with your favorite brands, and Earn real rewards.
SMS: ‘@delight’ to 92431-92431
Website: DelightCircle.com
Some highlights
“DelightCircle is featured as one of the top 6 mobile apps across the country.”
NASSCOM Emerge AppFame Aug 2012
“DelightCircle is featured by Samsung App Store & Blackberry App World.”
Sep, Oct 2012 - present
“DelightCircle is an app to help you pick out the best deals in the shops around you, and also hand you points for just walking into them. A must-have for all ye Android and iOS shopaholics.”
ThinkDigit, Sep 2012
MongoDB & MongoEngine power the data storage & processing parts of DelightCircle backend
platform
What is Geo-spatial/Location
data?Any data which has a location (2D – lat/lng)
associated with it.
Some Examples
• The data can be of users.User = {‘loc’:[12.97,72.023], ‘name’:’xyz’, ‘email’:’..’, ..}
• The data can be of local businesses.Place = {‘loc’:[12.97,72.023], ‘name’:’mnz’, ‘categories’:’..’, ..}
• The data can be of Photos.Photo = {‘loc’:[12.97,72.023], ‘file’: File(), ‘taken_by’:’User()’, ..}
• and can be many more types, leading to the next generation of location-based applications.
Let’s Dive in !
Build the backend for a check-in based rewards app
Requirements
• Rewards should be searchable by users based on the location and some keywords.
• Users can check-in to places using the app, and earn rewards based on check-ins.
Ex: GET 10% OFF on every 2nd check-in.
• Rewards can be created by the merchant, and can be assigned to specific places of his business.
Why MongoDB ?
• Fast, Easy and Schema-less.
• Multiple Language Support & Faster development cycle.
• Document Object store – maps much more cleanly to OOP.
• Replica sets/Automatic failover of failed nodes.
• Support for Geo-spatial Indexes and Queries.
It cheats to make the math easier/faster by assuming a flat earth (where 1 degree of latitude or longitude is always the same).
Has methods to handle curvature of Earth (Spherical – if needed)
MongoDB – a database with real world solutions for real
world problems
Simple to use & flexible.
MongoEngine
Python MongoDB
MongoEngine• Document Object Mapper for Python and MongoDB
• Built on top of PyMongo & Very similar to popular Django ORM
• Add all your validation stuff in MongoEngine. $easy_install mongoengine
Structuring & indexing your data depends on the way you
want to query your data
Data modeling differs between use-cases.
Idname
location (2d-index)
tags, etc.
Place
Idname
email (index)mobileGenderage, etc.
UserId
User-refPlace-ref
time (index)action
Check-in
Idtitle
[{Place-ref, Loc}, {Place-ref, Loc}, …..
]tags, etc.
Reward
Place-ref
User-ref
[Place-ref]1 to n
1 to 1
1 to 1
Schema Design
Let’s start with Places
Place = {_id: ObjectId(),loc: [12.97,72.23], //suggested optionname: ‘A’,tags: [‘food’, ‘restaurant’]
}
Place= {_id: ObjectId(),loc: {x:12.97, y:72.23},name: ‘A’,tags: [‘electronics’, ‘mobiles’]
}
loc: {lon: 40.739, lat:73.992}, ensure that you follow the same order for consistency.
Place Model - MongoEngine
Class Place(Document):loc= GeoPointField() // Geo2D index is
automatically createdname = StringField(max_length=50)tags =
ListField(StringField(max_length=300))
P = Place(name=‘Pizza Hut’, loc=[12.97,77.9], tags=[‘pizza’, ‘restaurant’, ..])
p.save() // saves the document to Mongo
import mongoengineconnect(‘my_database’) // connecting to DB
Some Geospatial Queries
Find the closest 20 places to a given location.
JavaScript Shell
> db.places.find({‘loc’: {‘$near’: [12.62, 72.323]}}).limit(20)
Python (MongoEngine)
Place.objects(loc__near = [12.62, 72.323]).limit(20)
Results are sorted by distance, no need of an additional sort.
Bounds Queries
Find the closest 20 places to a given location that are within a distance of 5 km.
JavaScript Shell
> db.places.find( { loc : { $near : [12.62,72.323] , $maxDistance : 5/earth_radius } } ).limit(20)
Python (MongoEngine)
Place.objects(loc__within__distance =[ [12.62, 72.323], 5/earth_radius]).limit(20)
Results are not sorted by distance, hence faster
User & Check-ins data model
User = {_id: ObjectId(),name: ‘A’,email: ‘[email protected]’, etc.
}
Check-in = {_id: ObjectId(),place: { "$ref" : "Places", "$id" :
ObjectId()},user: { "$ref" : ”Users", "$id" :
ObjectId()},time: ISODate(),etc.
}
Check-ins - MongoEngine
Class Check-in(Document):user = ReferenceField(‘Users’,
required=True),place = ReferenceField(‘Places’,
required=True),time = DateTimeField(required = True)
meta = {‘ordering’: [‘-time’],‘indexes’: [(‘user’, ‘place’)]
}
A Compound Index is created on user and place, since most of our queries will contain both.
Rewards data model
If you want to query the rewards based on location
Reward = {_id: ObjectId(),name: ‘GET 10% OFF .. ‘,places: [{ "$ref" : "Places", "$id" :
ObjectId()}, { "$ref" : “Places", "$id" : ObjectId()}, .. ]
etc.}
Reward = {_id: ObjectId(),places: [ { place: { "$ref" : "Places",
"$id" : ObjectId()}, loc: {x:0, y:0}} , .. ]etc.
}
Rewards - MongoEngineClass Reward(Document):
name = StringField(required=True),places =
ListField(EmbeddedDocumentField(PlaceLocation))
tags = ListField(StringField())num_checkins_required =
IntField(default=1)
Class PlaceLocation(EmbeddedDocument):place = ReferenceField(‘Places’)loc = GeoPointField()
Find the rewards near a location which have tags – ‘food’ and ‘pizza’.
JavaScript Shell
db.Rewards.find({‘places.loc’:{‘near’:[12.62, 72.3]}, ‘tags’: {‘$all’: [‘food’, ‘pizza’]}}).limit(20);
Python (MongoEngine)
Reward.objects(places__loc__near =[12.62,72.323], tags__all = [‘food’, ‘pizza’]).limit(20)
You need to have a compound geo-spatial index defined on both keys together for better performance of the query.
Conclusion
1. Spatial is easy and fun on MongoDB !!2. You can now build your own check-in
application (or) also build your own field data entry system.
3. Running MongoDB on any cloud infrastructure is quite easy.
PS: WE ARE HIRING – looking for smart hackersRoles: Backend, iOS Developer, UI/UX Designer, etc.
Mail me at [email protected] (or) come talk to me.
Questions?
Thank You
Stay Hungry, Stay Foolish !!!
- Steve Jobs
[email protected], Facebook - @raviteja2007
www.DelightCircle.com