devops and cloud native
TRANSCRIPT
[email protected]• In 1999, built a 20M page view per month Web site
• On two (2) HP servers (active-passive, manual failover)
• Dual Pentium 3 CPUs at 500+ MHz, 256MB RAM, 200GB HD
• ASP and C++ on Microsoft IIS
I.T. IS FACING GROWING DEMAND AND COMPLEXITY
2009
790 EB
2010
1.2 ZB
2020
40 ZB
2015
7.9 ZB
http://www.csc.com
Global Data
“EXPONENTIAL GROWTH WILL CONTINUE INDEFINITELY: YOU WILL NEED TO HANDLE TWICE AS MUCH CRAP TODAY AS YOU DID
18 MONTHS AGO.”
“EXPONENTIAL GROWTH WILL CONTINUE INDEFINITELY: YOU WILL NEED TO HANDLE TWICE AS MUCH CRAP TODAY AS YOU DID
18 MONTHS AGO.”
20 © 2014 CA. ALL RIGHTS RESERVED.
Technology opens the door to many more competitors
…and perhaps takes banks out of the equation
WINNERS VS LOSERS
70 © 2014 CA. ALL RIGHTS RESERVED.
How long does it take you to deploy your application?
INCLUDING ROLE BASED ACCESS
INCLUDING MONITORING
INCLUDING CONNECTIVITY
DEVS VS. OPS
• “Move fast and break things.”
• Iterations: from 2 weeks down to continuous
• “It works on my machine!”
• Security? Privacy? Backups?
• “Don’t fix it if it ain’t broke.
• Procurement: from 2 weeks up to 4 months
• Dev ≠ Test ≠ Staging ≠ Production
• Security! Privacy! Backups!
WHAT DEVOPS IS NOT
• Tool or Technology
• Methodology
• Role or a Team
• Silver bullet
VIRTUALISATION
CPU RAM HD NIC
Operating System
Applications
CPU RAM HD NIC
Virtualisation
Operating System
Application
Operating System
Application
Operating System
Application
Operating System
Application
Operating System
Application
Virtualisation Platform
Operating System
Application Application
CLOUD CLOUD
INFRASTRUCTURE AS CODE
• Can be checked into source control
• Can be versioned
• Can be audited
• Can be automatically executed
HOW WE USED TO DO SOURCE CONTROL:
“Don’t touch files A and B I’m not done with them yet.”
GITPlus branching strategy (e.g. git flow)
"ABOUT 15 - 50 ERRORS PER 1000 LINES OF DELIVERED CODE."
Steve McConnell, Code Complete
BUGS PER LINES OF CODE
AUTOMATED TESTING
MICROSERVICES
http://martinfowler.com/articles/microservices.html
WE CAN REWRITE OUR SERVICES AT WILL - WHICH WE DO, RATHER THAN CONTINUE TO REFACTOR THEM OR TO ADD MORE AND MORE TECHNICAL DATA OVER TIME. WE JUST REWRITE THEM WHEN WE GET TO A SCALING INFLECTION POINT.
Kevin GoldsmithVP of Engineering at Spotify
MICROSERVICES
SHIPPING?
SHIPPING!
WHAT DOCKER IS NOT
• Not just lightweight VMs
• Not another configuration management tool
• Not a silver bullet
APPLICATION WITH DEPENDENCIES
http://www.theregister.co.uk/2015/11/06/containers_thrash_vms_in_the_power_consumption_stakes/
“The biggest single benefit of Docker is the extent that it’s empowered the team to build services from scratch. We no longer have a complex set of provisioning scripts or AMIs—we just hand the production cluster an image, and it runs. There’s no more stateful instances, and we’re guaranteed to run the same exact code on both staging and prod.”
http://highscalability.com/blog/2015/10/19/segment-rebuilding-our-infrastructure-with-docker-ecs-and-te.html
DEVCON 2015 TECH RADAR
KUBERNETES• A platform for container management and
orchestration
• Started by Google (based on their internal Borg platform) and released Open Source
• Donated to the Linux Foundation in 2015
• 4000+ contributors and 40,000+ code commits (Dec 2016)
• Being adopted quickly by large Enterprises, Banks, Retail, etc (Walmart, SAP, Goldman Sachs, Box, etc)
• Sometimes referred to as GIFEE ("Google’s Infrastructure for Everyone Else")
KUBERNETES• All the benefits of Docker, but at
enterprise or "Web scale"
• Turn-key support from major cloud providers (Google, Microsoft)
• Can run on-premise
• Bare metal
• OpenStack
• Kubernetes Federated Clusters!
• Hybrid cloud
• Multi-cloud
CLOUD NATIVECLOUD NATIVE
SHIPPING CADENCE DEFINES YOUR COMPANY
CD SECOND ORDER EFFECTS
• Overall development costs reduced by ~40%
• Programs under development increased by ~140%
• Development costs per program reduced by 78%
• Resources driving innovation increased by 5x
CANARY RELASES
PUSHING IN MULTIPLE PHASES• latest - a version of the site running the latest code at all time. Employees
would use this site and find any major bugs almost instantly.
• p1 - a handful of servers that would be the first to run the new code in production. The goal of this was to catch any obvious fatals/warnings in the logs before a new release would gain wide distribution.
• p2 - a larger set of servers on the web tier. The number of servers in p2 increased over time, but I believe hovered around 5% or so. This offered several opportunities, including catching long tail fatals and monitoring resource use along with key user metrics on the servers for any anomalies.
• p3 - the entire web tier.
3 © 2014 CA. ALL RIGHTS RESERVED.
ING: A global Financial Services Group
Over 33 million clients
40 different countries
64,000 employees of which 15,000 IT
€15 bn annual revenue
54 © 2014 CA. ALL RIGHTS RESERVED.
Mobile Banking App
0
2000
4000
6000
8000
10000
12000
14000
16000
18000
Total number of deployments per month
56 © 2014 CA. ALL RIGHTS RESERVED.
Full Continuous Delivery Process with Nightly Builds
60 © 2014 CA. ALL RIGHTS RESERVED.
From 1 Star to 4 Stars in the iStore
63 © 2014 CA. ALL RIGHTS RESERVED.
ING Core Bank
Savings, Loans, Current Accounts
12 million accounts
10 DevOps teams
64 © 2014 CA. ALL RIGHTS RESERVED.
Continuous Delivery is not just for Web Applications
One Branch
One automated test set
One CD pipeline
65 © 2014 CA. ALL RIGHTS RESERVED.
Time to market from >20 weeks
To 4 days
67 © 2014 CA. ALL RIGHTS RESERVED.
From 30+ outages in 2013 to 0 in 2014
While increasing releases from 4 per year to every 3 weeks
• Culture
• Automation
• Lean
• Measurement
• Sharing
WHAT DEVOPS IS
maraming salamatありがとうございます
terimah kasihdaghang salamat
thank you