devopsdays state of the union amsterdam 2014
TRANSCRIPT
Devopsdays State of the Union
John Willis VP of Customer Enablement
Statelessnetworks @botchagalupe
Devops State of the Union
• The 2014 Devops Survey
• Software Defined Everything
• Consumable Composable Infrastructure
#Sometimes presentations write themselves #I was trying to think of what are the latest and greatest things going on that have an impact on us #devops folk… ## last week was a good week for some cool disruptions…
Devops State of the Union
• The 2014 Devops Survey
• Software Defined Everything
• Consumable Composable Infrastructure
The Big Fat Rethink
#Sometimes presentations write themselves #I was trying to think of what are the latest and greatest things going on that have an impact on us #devops folk… ## last week was a good week for some cool disruptions…
Devops Survey
#2013 was a good first start… It proved out the actions of high performance orgs #2014 might not the perfect but…. it’s pretty awesome improvement in that the data is starting correlate high perf w/competitive advantage and in some cases financial improvements #Meta Points: (this is really good data) ##9200 vs 4000 ##Empl Size = 41% in the 500 to 10k+ range ##Dept = Almost 60% are in the right place ##Servers = Roughly 60% >100 - were significant orgs ##20% are rather large orbs (>2k which would be a good indicator of enterprise)
Devops Survey
• Organizational Performance
• IT Performance
• Organizational Culture
• Job Satisfaction
#In summary the report kind of describes this idea of “Org Perf” #Also points out IT Performance = Competitive Advantage (up to 2x) ##A smaller subset of the respondents volunteered co / pub traded name/ of that subset the findings were that there was a 2.5x of ones that fell into the high IT perf categories #Organizational Culture = IT Performance ( One of the strongest predictors of IT Perf ) ##High trust, ##Cross Functional Collaboration, ##Shared Responsibilities, ##Learning Organization #Job Satisfaction is a key indicator for Organizational Performance #We’ve know this all along .. this is devops.. # We already know this as #devops
Devops Survey
• Organizational Performance
• IT Performance
• Organizational Culture
• Job Satisfaction= Devops
#In summary the report kind of describes this idea of “Org Perf” #Also points out IT Performance = Competitive Advantage (up to 2x) ##A smaller subset of the respondents volunteered co / pub traded name/ of that subset the findings were that there was a 2.5x of ones that fell into the high IT perf categories #Organizational Culture = IT Performance ( One of the strongest predictors of IT Perf ) ##High trust, ##Cross Functional Collaboration, ##Shared Responsibilities, ##Learning Organization #Job Satisfaction is a key indicator for Organizational Performance #We’ve know this all along .. this is devops.. # We already know this as #devops
Devops Survey
Devops Competitive Advantage
=
#However, this survey is the first form of proof that we are right… the data… # We can now use this data to start the “proof” that we are on to something. #We need a lot more data….
#
The Big Fat Rethink
#
Software Defined Everything
• Software Defined Data Center
• Software Defined Computing
• Software Defined Storage
• Software Defined Networking
#SDDC has also been referred to as Converged Infrastructure !# SDC - ## First order abstractions: Hypervisors on physical, then/now LXC, libContainer (containers) - Docker! - Micro-task virtualization! ## Second order abstractions are distributed clusters (Cloud, Big Data ) .. well defined API’s !#SDS ## First order abstractions virtualized storage hardware ## Second order abstractions ( Ceph, Swift, Swiftstack, S3).. well defined API’s (can you download the software) !#SDN ##First order abstractions VLAN .. again virtualizing a single box.. switch L2) ##Second order abstractions Tunnels … VXLAN… .. well defined API’s ## SDN offers further service abstractions … ### .. adding functionality into the network routing fabric for example LB (SDAS), FW (SDSec) becomes smart routing !# Major points… ## all three abstractions move one step away from the hardware. Second order abstractions are not directly coupled hardware. ## All three have high order programming interfaces (API’s). #These hight order highly programable abstractions changing the landscape of IT
SDN is the Decoupling of Control From the Data Plan
http://networkstatic.net
SDN is the decoupling of the control plane from the data plane... physically removing… !Network gear has traditionally been a black box w/coupled arch D,C,M !Data Plane.. packet-in packet-out .. nano second .. wire speed Control Plane.. Local distributed brains of the network (BGP, OSPF, MPLS), How the forwarding state gets to the data place .. milliseconds/days !Stanford Seminar - Software-Defined Networking at the Crossroads http://www.youtube.com/watch?v=WabdXYzCAOU !Nicira Notes: Nicira Networks was acquired by VMware for $1.26B. (7/23/12) Posted 7/23/12 at 6:14pm via techcrunch.com !Nicira Networks added Alan Cohen as VP, Marketing. (10/1/11) Posted 10/21/11 at 4:42am !Nicira Networks received $26M in Series C funding. (2/3/11) Posted 2/4/11 at 6:31pm via sec.gov !Nicira Networks added John Vrionis as Investor. (2/1/11) Posted 9/4/12 at 9:08am !
Openflow is to SDN as HTTP is to Web
http://networkstatic.net
OpenFlow is not magic, just a low level primitive with momentum for FIB programming (forwarding plane programming). Explain the flow table... Interesting thing this can be a device as ( a router, switch, firewall, Nat and even a Load Balancer). ... now it starts getting very interesting... The OpenFlow Switch and Controller can communicate via the OpenFlow protocol, !Service chains (firewalls, Load balancers, IDS, L3 Switching) !
Software Defined Everything
• Disaggregation of Hardware/Software
• X86 Compute Servers
• IP Based Storage
• Bare Metal Switches
# We saw disaggregation of compute about 15 years ago (Sun/IBM). # ISCSI over Fiber Channel .. # The latest and greatest is in networking… Cumulus Networks # OCP
!
!
• Software Defined Data Center
• Rethink Declarative
• Rethink Source of Truth
• Rethink Layer 7 Semantics
Software Defined Everything
# We need to start to rethink declarative and desired state infrastructure. # We are somewhat Myopic on compute? ## It’s not just server config, packaging and vm provisioning anymore. ### How do you create declarative desired state converged infrastructure (SDE) ### Storage config, network configurations? ### Declarative Network topologies ### How are SDN flow tables declared (SDN gui’s/API abstractions). Frenetic/Pynetic ### When do we get/have converged Cloud API abstractions (e.g., a jClouds that abstracts SDC,SDS and SDN) ### What does provisioning look like for containers.. #### With things like Mesos ad Fleet the new Kubernetes) ## What is the original desired state and what is the ongoing operational state of truth .. who wins? !#Do our new declarations start looking more like policies. ## I need service A ### it needs n amount of nodes with x amount of cpu/mem per node. ### How is storage allocated for this service (object storage/block storage) ### Does it need virtual perimeters for Networking (VXLAN) and Security (FW) and ### How is this application service request queuing managed (LB SDAS) and also QOS !# infrastructure - is there a common definition of a node? Is there a single source of truth for all “nodes” ## DNS is not enough… ## Service Discovery/Mapping … go based derivatives of zookepper. Etcd/confd .. ### Consul .. someones needs to give Mitchell some Demerol and slow this kid down..
Composable Infrastructure
Composability is a system design principle that deals with the inter-relationships of components. A highly composable
system provides recombinant components that can be selected and assembled in various combinations to satisfy
specific user requirements. !
Wikipedia
# Derek Collison (creator of CloudFoundry) says we are on the dawn of Composeable Infrastructure. Actually Google has been doing this very well of a number of years now. # The big difference in with things like docker containers are consumable…
• Bare Metal (8 weeks)
• Virtualization ( 2 weeks)
• IaaS ( 2 minutes)
• PaaS ( 1 minutes)
• Containers ( 500 ms)
Consumable Composable Infrastructure
#
• Docker
!
• Commoditized Containers
• Portable Images
• Using a Git Like Workflow
Composeable Infrastructure
# How to describe docker to a friend… !## Very easy to install and run LXC containers… ## Docker binaries are portable across multiple visualization infrastructures. ### Truly a create once run anywhere… (BM, VBOX, Vsphere, AWS, GCE, CF) ## My personal favorite is the Git like workflow… (docker pull,push, diff commit) ### The idea is you get the image from a repo, you change it, commit and push it). ### Docker uses “copy on write file system so you can do interesting workflows… !#commit, diff, pull, push
Virtualization Docker
Composeable Infrastructure
# From the #docker site…
• Orchestration
• Mesos
• Kubernetes
• CloudOS
• CoreOS
• Atomic
Composeable Infrastructure
# Openstack is to KVM as Mesos/Kubernetes is to Docker !## Mesos mostly developed by ex google guys who went to Twitter ### Google had the borg—>omega — (OSS) ->Kubernetes !## Distributes container workloads across multiple physical boxes with no/OSS overhead !# Will the real JEOS please stand up…
Racks of the Future
Cumulus Networks !
Opendaylight !
Mesos Server !
Docker Registry !
ETC/D
Cumulus Networks !
OVSDB !
Openvswitch !
Mesos Client !
Docker Engine !
CoreOS
# Left hand side is the management rack… (think Openstack Controller Node) # Right is the compute node .. (see what’s missing… i.e, hypervisor)
John Willis @botchagalupe
Some Images licensed under Bigstock.com