building high scale systems using serverless architecture for a burning man event - jonathan zenou -...
TRANSCRIPT
![Page 1: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/1.jpg)
BUILDING HIGH SCALE SYSTEM USING SERVERLESS ARCHITECTURE
FOR A BURNING MAN EVENTJonathann Zenou
DevOps Days Tel Aviv 2017
![Page 2: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/2.jpg)
DISCLAIMER
• This is not a serverless introduction
• This is about replacing
• This is a not a complicated solution
![Page 3: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/3.jpg)
HELLO WORLD
• Jonathann Zenou
• Tel Aviv
• DevOps Engineer @ Oracle
![Page 4: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/4.jpg)
![Page 5: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/5.jpg)
![Page 6: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/6.jpg)
![Page 7: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/7.jpg)
![Page 8: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/8.jpg)
![Page 9: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/9.jpg)
![Page 10: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/10.jpg)
![Page 11: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/11.jpg)
![Page 13: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/13.jpg)
NUMBERS• Since 2014
• 3rd biggest
• 11000 participants
• 2500 volunteers
• 30 departments
![Page 14: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/14.jpg)
MIDBURN TECH
• 50 yearly volunteers
• Perks of being a Non-Profit organisation
• Open source
• Crucial
![Page 15: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/15.jpg)
TECHNOLOGY
• Production and Rnd in AWS
• SCM in Github (Open source !)
• Ruby, NodeJS and Java
![Page 16: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/16.jpg)
SYSTEMS• volunteers system
• profile system
• gate system
• …
• ticket selling queue system
![Page 17: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/17.jpg)
PROBLEMATIC
![Page 18: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/18.jpg)
TICKETS SELLING SYSTEM• Sold out
• Very high demand in very short time
• Lottery ?
• 2 flash events
• FIFS
![Page 19: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/19.jpg)
1ST ITERATION
• 2014
• Drupal + php + MySQL
• Management of the whole tickets process from Drupal
![Page 20: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/20.jpg)
BUT
![Page 21: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/21.jpg)
![Page 22: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/22.jpg)
1 ST OUTAGE
• ~5000 concurrent users
• ~20 sessions by user
![Page 23: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/23.jpg)
![Page 24: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/24.jpg)
REMODELLING
![Page 25: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/25.jpg)
![Page 26: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/26.jpg)
![Page 27: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/27.jpg)
REMODELLING
• One queuing system
• One selling system
![Page 28: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/28.jpg)
2ND ITERATION
• 2015
• Backend : Heroku + Reddis + Ruby
• Frontend : Javascript + S3
![Page 29: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/29.jpg)
Frontend
CSV
![Page 30: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/30.jpg)
![Page 31: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/31.jpg)
![Page 32: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/32.jpg)
![Page 33: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/33.jpg)
2015
![Page 34: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/34.jpg)
![Page 35: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/35.jpg)
2016
![Page 36: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/36.jpg)
![Page 37: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/37.jpg)
![Page 38: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/38.jpg)
2ND OUTAGE
![Page 39: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/39.jpg)
![Page 40: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/40.jpg)
![Page 41: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/41.jpg)
![Page 42: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/42.jpg)
![Page 43: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/43.jpg)
![Page 44: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/44.jpg)
• a lot of users enter the system
• Backend is swamped
• Users hit “refresh”
• Backend is even more swamped
![Page 45: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/45.jpg)
WHAT WE UNDERSTOOD ?
![Page 46: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/46.jpg)
![Page 47: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/47.jpg)
![Page 48: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/48.jpg)
REMODELLING
![Page 49: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/49.jpg)
3RD ITERATION
• 2017
• Frontend : still Javascript + S3
• Backend : …
![Page 50: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/50.jpg)
![Page 51: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/51.jpg)
CREATE AN API GATEWAY API AS AN AMAZON KINESIS
PROXYhttp://docs.aws.amazon.com/apigateway/latest/developerguide/
integrating-api-with-aws-services-kinesis.html
![Page 52: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/52.jpg)
![Page 53: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/53.jpg)
Amazon API Getaway
![Page 54: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/54.jpg)
Amazon Kinesis Firehose
![Page 55: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/55.jpg)
![Page 56: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/56.jpg)
![Page 57: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/57.jpg)
S3 STATIC SITE LOGIC
is the opened queue page up
?
No
wait 5 seconds and retry
![Page 58: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/58.jpg)
![Page 59: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/59.jpg)
![Page 60: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/60.jpg)
TESTING
![Page 61: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/61.jpg)
CROSSING FINGERS2nd sale of the year
![Page 62: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/62.jpg)
![Page 63: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/63.jpg)
![Page 64: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/64.jpg)
![Page 65: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/65.jpg)
NUMBERS• 15 minutes
• 3315 users
• 300 lucky users
• 4 seconds
• 3 000 000 requests
• 0 errors
![Page 66: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/66.jpg)
–Itamar Cohen, Midburn's COO
The Midburn community is composed of thousands of people.Not everyone can get a ticket and people can
get angry and frustrated from it . With this new system we ensure that at least if someone does not
get a ticket it’s not because of a technical issue .
![Page 67: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/67.jpg)
![Page 68: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/68.jpg)
![Page 70: Building High Scale Systems Using Serverless Architecture for a Burning Man Event - Jonathan Zenou - DevOpsDays Tel Aviv 2017](https://reader038.vdocuments.site/reader038/viewer/2022103010/5a64cf087f8b9a6d5d8b49e3/html5/thumbnails/70.jpg)
QUESTIONS ?