fast rest apis development with mongodb
DESCRIPTION
In an R&D company fast prototyping is vital to develop new projects or proofs of concept quickly and inexpensively. In this talk we will demonstrate how real fast and agile development can be achieved with MongoDB and dynamic languages, with examples and best practices. All the code shown is already uploaded to a public Git repository - https://github.com/pablito56/py-eshopTRANSCRIPT
WITH MONGODB
FAST REST APIS DEVELOPMENT
Pablo Enfedaque Vidal @pablitoev56
WHO? • Pablo Enfedaque
• Computers Engineer
• Tech Lead and R&D software engineer at Telefonica Digital
• Working with MongoDB for some years.
TELEFONICA DIGITAL. WHAT? • Telefonica
• Fifth largest telecommunications company in the world
• Operations in Europe (7 countries), the United States and Latin America (15 countries)
• Movistar, O2, Vivo, Terra, Tuenti, Jahjah, Tokbox, everything.me, Open Web Device…
TELEFONICA DIGITAL. WHAT? • Telefonica Digital
• Web and mobile digital contents and services division
• Product Development and Innovation unit
• Products & services development, research, technology strategy, user experience, deployment & operations…
• Around 70 different on going projects
OUR PROJECTS • Full product development, with life cycle and several deployments
• 20 people team, 1 year or more
• Pilot or small product to be deployed in a certain environment • 6 people team, 6 months
• Seedbed or proof of concept to be run with reduced set of users • 3 people team, 3 months
• Ten Fridays open exploratory project to work on your ideas • 2 people team, 10 days (consecutive Fridays)
SO…
FAST DEVELOPMENT IS REALLY CRUCIAL FOR US
HOW TO SPEED UP OUR DEVELOPMENTS? • Agile methodologies
• Lean startup
• eXtreme Programming
• Continuous Integration
• …
HOW TO SPEED UP OUR DEVELOPMENTS?
CHOOSE THE
RIGHT TECHNOLOGY (AT FIRST)
¿ RIGHT TECHNOLOGY ?
THE RIGHT TECHNOLOGY • Faster development with Dynamic Languages
• 3x
• 4x
• 10x
THE RIGHT TECHNOLOGY
THE SAME CAN BE STATED FOR MONGODB • 3x
• 4x
• 10x
THE RIGHT TECHNOLOGY • Several times faster development with Dynamic Languages
• Several times faster development with MongoDB
AND BOTH TOGETHER IS A WIN WIN
WHY? HOW?
LET’S SEE SOME EXAMPLES
ONLY DYNAMIC LANGUAGES?
JAVA VERSION public int[] getDims() {
if (this.dims != null) { return this.dims;}BasicDBObject query = new BasicDBObject();query.put("_id", "ctxt_dimensions");DBObject setup = setup_coll.findOne(query);BasicDBList dbl = (BasicDBList)setup.get("dims");this.dims = new int[dbl.size() + 2];BasicDBObject users_counter_ref = new BasicDBObject("_id", users_coll_name);BasicDBObject apps_counter_ref = new BasicDBObject("_id", apps_coll_name);dims[0] = (Integer)counters_coll.findOne(users_counter_ref).get("value") + 1;dims[1] = (Integer)counters_coll.findOne(apps_counter_ref).get("value") + 1;for (int i=0; i<dbl.size(); i++) { dims[i + 2] = (Integer)dbl.get(i); }return dims;
}
PYTHON VERSION def get_dims(self): ud = self.counters_coll.find_one({'_id': 'users'})['value'] ad = self.counters_coll.find_one({'_id': 'applications'})['value'] res = [ud, ad] res.extend(self.setup_coll.find_one({}, {'dims': 1})['dims']) return res
IT’S UP TO YOU…
THE RIGHT TECHNOLOGY
LET’S PLAY TO SPOT THE
DIFFERENCES
EXAMPLE: SPEAKER JSON {
"name": "Pablo Enfedaque",
"company": "Telefonica Digital",
"accepted": true,
"registration_date": "2012-03-15T14:35:05",
"num_talks": 1,
”votes": 4,
"email": "[email protected]"
}
EXAMPLE: DECODED JSON (PYTHON) {
"name": "Pablo Enfedaque",
"company": "Telefonica Digital",
"accepted": True,
"registration_date": datetime(2012, 3, 15, 14, 35, 5),
"num_talks": 1,
”votes": 4,
"email": "[email protected]"
}
EXAMPLE: MONGODB BSON {
"name": "Pablo Enfedaque",
"company": "Telefonica Digital",
"accepted": true,
"registration_date": ISODate("2012-03-15T14:35:05Z"),
"num_talks": 1,
”votes": 4,
"email": "[email protected]",
”_id": ObjectId("5142d08c5db1362abc2d208b”)
}
LOOKS PRETTY
STRAIGHT FORWARD, RIGHT?
SPEAKER CREATION decoded_input = json.loads(input_json)decoded_input['registration_date'] = datetime.strptime(decoded_input['registration_date'], "%Y-%m-%dT%H:%M:%S”)return dbconn['speakers'].insert(decoded_input)> ObjectId('5142d2845db1362bb3155322')
SPEAKER RETRIEVAL retrieved = dbconn['speakers'].find_one({'name': 'Pablo'}, {'_id': 0})retrieved['registration_date'] = retrieved['registration_date'].strftime("%Y-%m-%dT%H:%M:%S")return retrieved
IT IS REALLY
STRAIGHT FORWARD!
WHAT IF WE WANT TO
CHANGE SPEAKERS DATA?
EXAMPLE: SPEAKER JSON {
"name": "Pablo Enfedaque",
"company": "Telefonica Digital",
"position": "R&D SW Engineer",
"accepted": true,
"registration_date": "2012-03-15T14:35:05",
"num_talks": 1,
”votes": 4.3, WAS AN INTEGER
"email": "[email protected]"
}
SPEAKER CREATION decoded_input = json.loads(input_json)decoded_input['registration_date'] = datetime.strptime(decoded_input['registration_date'], "%Y-%m-%dT%H:%M:%S”)return dbconn['speakers'].insert(decoded_input)
SPEAKER RETRIEVAL retrieved = dbconn['speakers'].find_one({'name': 'Pablo'}, {'_id': 0})retrieved['registration_date'] = retrieved['registration_date'].strftime("%Y-%m-%dT%H:%M:%S")return retrieved
0 LINES CHANGED
INPUT VALIDATION NEEDED?
SPEAKER VALIDATION from rest_framework import serializersclass SpeakerSerializer(serializers.Serializer): name = serializers.CharField(max_length=150) company = serializers.CharField(max_length=150) position = serializers.CharField(required=False) accepted = serializers.BooleanField() registration_date = serializers.DateTimeField() num_talks = serializers.IntegerField() votes = serializers.FloatField() email = serializers.EmailField(max_length=150) def restore_object(self, attrs, instance=None): return attrs
SPEAKER CREATION decoded_input = json.loads(input_json)serializer = SpeakerSerializer(decoded_input)print dbconn['speakers'].insert(serializer.object)
SPEAKER RETRIEVAL retrieved = dbconn['speakers'].find_one({'name': 'Pablo'})serializer = SpeakerSerializer(retrieved)return serializer.object
DON’T LIKE TO WORK WITH DICTIONARIES / HASHES?
CUSTOMISE ATTRIBUTES ACCESS class AttrDict(dict): def __getattr__(self, name): try: return super(AttrDict, self).__getitem__(name) except KeyError, e: raise AttributeError(e) def __setattr__(self, name, value): if name in self: super(AttrDict, self).__setitem__(name, value) else: super(AttrDict, self).__setattr__(name, value)
USE DICTIONARIES AS OBJECTS decoded_input = json.loads(input_json)serializer = SpeakerSerializer(decoded_input)speaker_obj = AttrDict(serializer.object)print speaker_obj.companyprint speaker_obj['position']> Telefonica DigitalR&D SW Engineer
USE AN ORM?
NO
OBJECT-RELATIONAL MAPPER
NO RELATIONAL NO ORM
NEEDED
CONCLUSIONS
CONCLUSIONS • MongoDB + dynamic languages = fastest development speed
• 14 months project with Oracle à 3 months project with MongoDB
• REST API best practices
• Use JSON
• Use dictionaries / hashes
• Access dictionaries as objects
• No relational model à no ORM
• No other mappers
• Use decorators to handle AutoReconnect