re-architecting with mongodb
TRANSCRIPT
![Page 1: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/1.jpg)
RE-ARCHITECTING WITH MONGODB
FIXING A RELATIONAL MESS WITH A NON RELATIONAL DATABASE
![Page 2: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/2.jpg)
ABOUT US
MICHAEL LUJANVP OF ENGINEERING @ VIXLET
BOBBY LINCOLNCOO @ VIXLET
![Page 3: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/3.jpg)
A LITTLE ABOUT VIXLET
• LOCATED IN DOWNTOWN LA• A PASSION BASED SOCIAL NETWORK • SOCIAL NETWORKS FOR MLB, ATP,
LFC, SLIPKNOT, ETC
![Page 4: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/4.jpg)
CAPSULES
• CAPSULES ARE USED TO REPRESENT TOPICS
• USERS AND BRANDS CAN POST INTO THESE CAPSULES
![Page 5: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/5.jpg)
THE PULSE• THE PULSE IS WHERE YOU FIND
CONTENT FROM CAPSULES YOU HAVE SUBSCRIBED TO• SHOWS CAPSULE WHERE MEDIA IS
FROM• USER WHO POSTED CONTENT• WHETHER YOU HAVE LIKED THIS
CONTENT
![Page 6: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/6.jpg)
MEDIA RECORDS
• ORIGINALLY DESIGNED BY DEVELOPERS WITH RDBMS EXPERIENCE
• FOREIGN KEYS JOINED AT RUNTIME IN CODE
{”_id”: ”
53d682bba374b38a686e9eeb”,“capsuleId”:
“53d681f9a374b38a686e9ee0”,“creator”:
“55b815ea54517d544e3f5bee”,“type”: “text”,“caption” “test”,“statuses”: {
“created”: 1445621494381,“updated“: 1465410655512
}}
![Page 7: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/7.jpg)
ORIGINAL WORKFLOW
Feed MS Media MS Capsule MSoAuth MS
Capsule MS
Relationship MS
User MS
User MS
Relationship MS
![Page 8: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/8.jpg)
AVERAGE RESPONSE TIMES
Call Response Time AveragesPulse 4000msCapsule Feed 2000msProfile Feed 2500ms
![Page 9: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/9.jpg)
INSERT FUNNY PICTURE
![Page 10: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/10.jpg)
TIME TO RE-ARCHITECT
• MICRO MODULE ARCHITECTURE• SINGLE API LAYER BUT BROKEN UP INTO BITE SIZE MODULES• REMOVES HTTP OVERHEAD OF SOME MICRO-SERVICE ARCHITECTURE• ALLOWS FOR SINGLE LAYER TO BE BROKE UP INTO MICRO-SERVICES• ALLOWS FOR EACH PIECE TO BE OPTIMIZED (INCLUDING DATABASES)
![Page 11: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/11.jpg)
MEDIA CREATION
Service Layer
Create Relationship
sCreate Media
Record Add to feeds
Mongo Riak Riak
![Page 12: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/12.jpg)
SIMPLIFY THE ARCHITECTURE
• WORKER ARCHITECTURE• CRUD IS HANDLED BY SERVICE LAYER• ALL NON CRITICAL WORK IS HANDED OFF TO WORKERS• MAIN DISPATCHER HANDLES ALL REQUESTS • IMPROVES SPEED AND SIMPLICITY
![Page 13: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/13.jpg)
DISPATCHINATOR
• EVENT TRACKING• ERROR HANDLING / RETRY LOGIC
![Page 14: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/14.jpg)
NEW MEDIA CREATION
Service Layer
Media Module
Dispatcher Queue
Create Relationship Neo4j
Mongo
CassandraAdd to Feeds
![Page 15: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/15.jpg)
NEW MEDIA RECORDS
• COMPLETE CAPSULE AND CREATOR OBJECTS
• NO LONGER NEED TO JOIN ON FOREIGN KEYS
{ ”_id”: ” 53d682bba374b38a686e9eeb”, “capsule”: {
“id”: “53d681f9a374b38a686e9ee0”,“name”: “ATP Challenger Tour”,“cover”: “image.png”,“description”: “description”
}, “creator”: {
“id”: “55b815ea54517d544e3f5bee”,“username”: “atpworldtour”,“avatar”: “image.png”
}, “type”: “text”, “caption” “test”, “statuses”: {
“created”: 1445621494381,“updated“: 1465410655512
}}
![Page 16: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/16.jpg)
CAPSULE / USER UPDATES
Service Layer Module Dispatcher
Queue
Update Media Mongo
Mongo
![Page 17: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/17.jpg)
BUT THERE IS A PROBLEM
• WHAT HAPPENS WHEN THERE ARE 1,000 UPDATES? OR 1,000,000 UPDATES?
![Page 18: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/18.jpg)
FAN OUT ON WRITE WORKER
• KEEP COUNT OF MEDIA RECORDS PER CAPSULE• BREAK UP WORK INTO MANAGEABLE SIZES
![Page 19: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/19.jpg)
NEW UPDATE WORKFLOW
Service Layer Module Dispatcher
Queue
Fan Out Worker
MongoMongo Update
Media
![Page 20: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/20.jpg)
NEW LOOK UP WORKFLOW
Service Layer
Feed Module
Relationship Module
![Page 21: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/21.jpg)
NEW RESPONSE TIMES
Call Response Time AveragesPulse 200msCapsule Feed 150msProfile Feed 150ms
![Page 22: Re-Architecting with MongoDB](https://reader034.vdocuments.site/reader034/viewer/2022042611/58e65d731a28ab8d758b4c2d/html5/thumbnails/22.jpg)
ANY QUESTIONS???