devops oxford- devops + bigdata @ realtime

Download DevOps Oxford- DevOps + BigData @ RealTime

Post on 14-Jan-2017

66 views

Category:

Documents

5 download

Embed Size (px)

TRANSCRIPT

  • DevOps + BigData @ RealTime

  • DevOps+(DevOps in a startup)

  • DevOps+ Fixing the electrical outage

  • DevOps+ Fixing the electrical outage Cycling to Maplin for an extension lead Dealing with the sewage from the blocked toilet

  • BigData Also known as - Data BigData is writing Doom or Elite for a 32k machine:

  • BigData Or just because you cant quite fit it on a 39 rotating disk:

  • @RealTime Can you survive this ?

  • The Challenge - The 2014 (Football) World Cup

    Analyse every Tweet issued during World Cup games in the 2014 competition

  • The Challenge - The 2014 (Football) World Cup

  • The Challenge - The 2014 (Football) World Cup

    How busy was it expected to be ?

  • The Challenge - The 2014 (Football) World Cup

    How busy was it expected to be ? Predicted 1.2M Tweets per hour Average time for our analysis is 100ms - lengthy texts up to 150ms Our analysis time increases logarithmically with sentence count and sentence length so needed to avoid pathological cases 150ms x 1.2M = 180k sec = 3000 minutes = between 33 and 50 hours

  • The Solution - AWS

    Lots of EC2 instances 3 Availability Zones ELB

  • The Solution - Unattended Build

    Cloud Formation Puppet Route53 ELB 2 min

  • The Challenge - The 2014 (Football) World Cup

    How busy was it ? Predicted peak 1.2M Tweets per hour Actual peak 2.5M Tweets per hour Whole tournament avg 17M per day for 30 days Peak 33M in one day

  • The Challenge - The 2014 (Football) World Cup

    How busy was it ? Predicted peak 1.2M Tweets per hour Actual peak 2.5M Tweets per hour Whole tournament avg 17M per day for 30 days Peak 33M in one day

  • The Challenge - The 2014 (Football) World Cup

    How busy was it ? Predicted peak 1.2M Tweets per hour Actual peak 2.5M Tweets per hour Whole tournament avg 17M per day for 30 days Peak 33M in one day

  • A Long Time Ago In A Galaxy Far Far Away Carhire3000 began - we had 6 servers Building a new server took a week 8 years later:-

    2 acquisitions Booking.com $996 million 600+ servers DevOps team of 5

  • Booking.com methodology - Unattended build in 5 minutes Control Room - Monitoring health of website

  • Control Room - Columnar Databases

  • Columnar Databases Fire & Forget UDP events What do you monitor ? Experiments & Z-score DB Queries < 2 seconds Hot event data in MySQL - Hands Off ! 5 minutes later moved to columnar storage database