propel to the cloud with open source
DESCRIPTION
Trusted Opinion CTO shares the experience in re-engineering of a social network. Major parts of the presentation: - lessons learned during the re-engirding process - open source frameworks used to implement a highly scalable web applications - system architecture solutions used 5 years ago and todayTRANSCRIPT
Propel to the Cloud with Open Source
or “How we re-engineered a social network” by Gene Leybzon, CTO, Trusted Opinion, Inc
Propel to the cloud by Gene Leybzon 2
Trusted Opinion Social Recommendation Network
Propel to the cloud by Gene Leybzon 3
Trusted Opinion in NumbersOver Million usersOver 100 CountriesOver 1.5 Million Bars and Restaurants with
reviewsOver 100,000 emails sent daily
Propel to the cloud by Gene Leybzon 4
“Legacy” architecture
Propel to the cloud by Gene Leybzon 5
How J2EE architecture became a liability instead of an assetBusiness needs:
Almost “on-the fly” code changes with immediate deployments (daily releases)
Extreme scalabilityConstant tuning based on A/B testingCan not afford expensive DB licensing feeParallelism in code development with no
bottlenecks in the develop-test-release cycles
Propel to the cloud by Gene Leybzon 6
“Legacy” hosting
Propel to the cloud by Gene Leybzon 7
Scalability and Performance ProblemsToo much work performed in Database (store
procedures did a lot of math on the data)Way too expensive to go to database cluster
while a single server (even 16 core) has it’s limits too
Overhead of J2EE+templates (slow performance)
Propel to the cloud by Gene Leybzon 8
Other Pain PointsManaging offshore development teamSignificant of effort in system/database
administrationToo slow to implement changesHard to implement detail user behavior
analyticsTime to be multilingual and integrate with
CDN to support international customers
Propel to the cloud by Gene Leybzon 9
SolutionComplete re-architecture from the top-to-
bottom and from the bottom-up Update to current Snapshot of the best
technologies
Propel to the cloud by Gene Leybzon 10
Wish list that became the PlanUse latest and greatest technologiesOpen source, and open source only!Scalability on demandLocal team for core software developmentBest analytics with customized A/B testing
frameworkGood performance and with even better
performance perception
Propel to the cloud by Gene Leybzon 11
Why Cloud?Scalability on demand Dynamic load Ease deployment/keeping configurationSimplify ITCool factor
Propel to the cloud by Gene Leybzon 12
Why Amazon?EC2Cloud FrontMapReduceLoad balancerPerformance monitoring
Propel to the cloud by Gene Leybzon 13
Job Ahead#lines of Java codeMigrating to the new database and with
completely different database schemaAdd language support (20 core languages)Add on-demand image uploading to CDN and
serving all static content from the CDN
Propel to the cloud by Gene Leybzon 14
ChallengesWhere to find right and motivated
developers?Where to rent the office How to create a quality code development
cultureHow to have fun along the way
Propel to the cloud by Gene Leybzon 15
LessonsPeople firstHappy developers write good codeIn-house recruitingFocus on the core, borrow the rest
Propel to the cloud presentation by Gene Leybzon
16
New Architecture
Propel to the cloud by Gene Leybzon 17
New Database Architecture
Propel to the cloud by Gene Leybzon 18
Analyzing log data
Propel to the cloud by Gene Leybzon 19
Semantic Web and RDFUsed at Trusted Opinion to store information
about Movies and Restaurants Data stored across all items as “triplets”Allows structured and semi-structured data
to be mixed, exposed, and shared across different applications
Took a lot of load from relational DBEasy to add new item categories
20
You can do it too!“Just do it Right”“Right” developers (treat them with
respect!)“Right” technology (keep an eye on
technology trends, use it fresh but ready)“Right” design (NO new frameworks, focus
on the product, not a framework)“Right” management team
Propel to the cloud by Gene Leybzon