one architecture - many countries - how we brought stylight abroud
DESCRIPTION
We detail certain steps we took in the engineering department to scale STYLIGHT from one to 10 countriesTRANSCRIPT
|Slide 1 | STYLIGHT | Proud to bleed purple | @stylight
ONE ARCHITECTURE – MANY COUNTRIESHow we scaled STYLIGHT to 10 countries
Slide 2 | STYLIGHT | Proud to bleed purple | @stylight
WHAT‘S STYLIGHT?The best way to shop fashion
Slide 3 | STYLIGHT | Proud to bleed purple | @stylight
WHAT‘S STYLIGHT?Internationalization
Slide 4 | STYLIGHT | Proud to bleed purple | @stylight
WHAT‘S STYLIGHT?Some Facts
• 100 employee• 10 countries (adding three more this year)• 5 years old
• 18 engineers / designers/ DevOps• 6 nationalities, all in Munich• SCRUM, 3 teams
• Website• Mobile• Interne Tools
Slide 5 | STYLIGHT | Proud to bleed purple | @stylight
TECHNOLOGY STACKTwo Systems
„Backend“• Enhancing product meta data• Internal Tools
• Curation• SEO• SEM• Invoicing
„Frontend“• Searchpages• User interactions• API-Endpoints for Mobile Apps
Slide 6 | STYLIGHT | Proud to bleed purple | @stylight
TECHNOLOGY STACKTwo Systems
„Backend“• Used to be php, now python• Tornado• Alembic• Celery + rabbitmq• Mysql• MongoDB
„Frontend“• Java• Jboss• Hibernate• Struts• Bootstrap / less• jQuery / handlebar• Solr• Percona Cluster
Slide 7 | STYLIGHT | Proud to bleed purple | @stylight
SWITCHING TO UTF8
Slide 8 | STYLIGHT | Proud to bleed purple | @stylight
SWITCHING TO UTF8From latin1 to utf8
• Didn’t thought about it / never really understood• “Charsets, well, isn’t that what defines which chars are equal on LIKE
statements”
• Idea: convert backend first, then do frontend
Backend• Convert database
• Solve unique constraint violations – mostly SKUs cause trouble• Switch character set on MySQL driver – now we can work in parallel• Adapt html headers• Scan for all functions that encode / decode utf8• Scan for functions that generate URLs
Slide 9 | STYLIGHT | Proud to bleed purple | @stylight
SWITCHING TO UTF8From latin1 to utf8
Frontend• Convert database
• Comparably easy, since we mostly store IDs• Switch MySQL driver – test surfing• Manually check all input fields – we used to sanitize inputs for charset errors• Switch systems where we interface external partners (i.e. mail provider)
Switch was easier than expected, mostly because coding in utf8 is as easy as in latin1
=> Why don’t you use utf8 from the beginning?
Slide 10 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTURE
Slide 11 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTUREInitial situation
• In the beginning: ec2• For three years now: provider in
Munich• Real hardware• Images through Cloudinary
(CloudFront / Akamai)• JS / CSS on CloudFront
Slide 12 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTURE1st Attempt
• 90% is served by the CDN (unprimed)• Still 80% with primed cache
=> AppServer stays in the existing data center
• We have little ajax, that helps• Works great for Europe
Slide 13 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTURESetup
Slide 14 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTURE2nd Attempt
Bring AppServer closer to the user
• Don’t get a local data center, use ec2 or alike• Low latency now between AppServer and end user• Can easily move solr too, MySQL now the bottle neck
Move database?• Until now the databases are synchronous (percona cluster)• Switch to asynchronous replication
• Trouble ahead with broken replication….• Read / Write split
• Requires different thinking, hard to implement=> Right now, we stick with our initial setup
Slide 15 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTUREMonitoring – Google Analytics
Slide 16 | STYLIGHT | Proud to bleed purple | @stylight
SERVER ARCHITECTUREMonitoring – New Relic
DE
AU
Slide 17 | STYLIGHT | Proud to bleed purple | @stylight
TRANSLATION
Slide 18 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEGit + online translation
master
development
weblate
release
cronnew properties
fetch translations
translating
git merging
actual work
Slide 19 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEcreate SSH key for weblate app
Slide 20 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEmake weblate github user + SSH key
Slide 21 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEgive weblate permissions on repos
Slide 22 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEcreate weblate project + subproject
Slide 23 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEweblate git config
Slide 24 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEweblate cronjob
Slide 25 | STYLIGHT | Proud to bleed purple | @stylight
WEBLATEDevelopment process
• Important to communicate that developers only work on the base language file:• Strings/en.lproj/Localizable.strings (iOS)• view/package.properties (Java)• values/strings.xml (Android)
• Directly modifying other language files, will most likely cause conflicts! let the translators do this work using Weblate!
Slide 26 | STYLIGHT | Proud to bleed purple | @stylight
THANKS FOR YOUR ATTENTION!
We‘re hiring!
Sebastian Schuon Co-Founder & CTO
http://www.stylight.com/Engineering/
Slide 27 | STYLIGHT | Proud to bleed purple | @stylight