enabling enterprise devops at scale
TRANSCRIPT
2© Copyright 2016 EMC Corporation. All rights reserved.
DevOps is …
…optimizing the flow of work to create products that create value faster and more reliably.
OPS
DEV
3© Copyright 2016 EMC Corporation. All rights reserved.
Success is a company’s ability to see the window of opportunity and capitalize on it.
https://www.ted.com/talks/bill_gross_the_single_biggest_reason_why_startups_succeed?language=en
3
4© Copyright 2016 EMC Corporation. All rights reserved.
Rigid structures and knowledge silos
5© Copyright 2016 EMC Corporation. All rights reserved.
Managed with Manual Processes
6© Copyright 2016 EMC Corporation. All rights reserved.
Heavily Laden with Bureaucracy
6
7© Copyright 2016 EMC Corporation. All rights reserved.
Complexity
Portfolio ProcessesOrganization
7
8© Copyright 2016 EMC Corporation. All rights reserved.
PEOPLE(Skills & Culture)
TOOLS(Platforms & Ecosystems)
DEVOPSTRANSFORMATION
PROCESS(& Practices)
9© Copyright 2016 EMC Corporation. All rights reserved.
Gene Kim: What is Different about High-Performing Companies?
10© Copyright 2016 EMC Corporation. All rights reserved.
ComplexityPortfolio ProcessesOrganization
10
Organizations are Complex Systems
We can fix this…• PMO: “Tighter processes and approvals!”• Accounting: “Outsource more!”• Execs: “Need results, reorg, reorg!”• Ops: “Need new tools, servers, portals!”• Dev: “Put it in the cloud!”• EA: “SOA and micro-services!”
It’s not hard to see…• People are working out of context• The work isn’t visible• Inertia pulls the organization
out of alignment
11© Copyright 2016 EMC Corporation. All rights reserved.
Complexity
Portfolio ProcessesOrganization
11
Solving for Complexity
The only way to fix a complex system like an enterprise is to: create conditions for the system to fix itself
12© Copyright 2016 EMC Corporation. All rights reserved.
Big Bang Transformation “Wishing”
Start
Finish
13© Copyright 2016 EMC Corporation. All rights reserved.
Big Bang Transformation “Reality”
Start
Finish
Fear
Panic
Abort
Maybe
Focus
People revert back toprevious behaviors
14© Copyright 2016 EMC Corporation. All rights reserved.
Improvement in Complexity is well know known: Scientific MethodVariants: PDSA, PDCA, OODA
Plan – Do – Study – Act(PDSA)
The defining trait of high-performingcompanies is that they are good at learning fast
The ability to improve…
15© Copyright 2016 EMC Corporation. All rights reserved.
Teach an Organization to Fix ItselfLittle Js instead of Big J
Start
Finish
FearPanic
Abort
MaybeFocus
FearPanicAbort
MaybeFocus
FearPanic
Abort
MaybeFocus
FearPanicAbort
MaybeFocus
FearPanic
Abort
MaybeFocus
FearPanicAbort
MaybeFocus
FearPanicAbort
MaybeFocus
FearPanicAbort
MaybeFocus
FearPanicAbort
MaybeFocus
“Baby Step” Continuous Improvement
16© Copyright 2016 EMC Corporation. All rights reserved.
ComplexityPortfolio
Processes
Organization
16
Solving for Complexity
Turn Continuous Improvement into an enterprise program by:1. Keep improvement efforts aligned2. Scale quickly3. Span multiple organizational
boundaries4. Work with substantial number of
legacy technologies5. Develop existing staff in mass6. Self fund after initial investment
How to Solve?
17© Copyright 2016 EMC Corporation. All rights reserved.
PROCESS(& Practices)
18© Copyright 2016 EMC Corporation. All rights reserved.
Why Do Continuous Delivery?
Let’s look at a classic release
Design Development Release
19© Copyright 2016 EMC Corporation. All rights reserved.
Why Do Continuous Delivery?
The reality
Deployment Issues
Defects
Product changes
20© Copyright 2016 EMC Corporation. All rights reserved.
Why Do Continuous Delivery?
The reality
Defect introduced
Feature
incorrectly
implemented
Stuff built on
top
Deployment Issues
Defects
Product changes
21© Copyright 2016 EMC Corporation. All rights reserved.
HOW DO I FIT CD INTO MY AGILE PROJECT?
Each release is thus a series of
multiple commits
An Agile release
22© Copyright 2016 EMC Corporation. All rights reserved.
HOW DO I FIT CD INTO MY AGILE PROJECT?
An Agile release…with CD applied
With CD, that is reversed.
Each commit is now a potential release
candidate
23© Copyright 2016 EMC Corporation. All rights reserved.
How does CD Help?
With CD, every build is a release candidate
24© Copyright 2016 EMC Corporation. All rights reserved.
What is Über dependency hell?
Enterprise Portfolios Realities
• Low Automated Change Management
• Slow Feedback & Notifications
• “Ripple Effect”
Portfolio Impacts
• Low Quality/Velocity
• Low IT/Business Performance
• Technical Debt
Weeks Later…Change in App AApp B Breaks when
A is Deployed.
25© Copyright 2016 EMC Corporation. All rights reserved.
Start small and scale with confidence
Incremental Methodology
APP
APP
APP
APPAPPAPP
APP
APP
APP
APPAPP
APPAPPAPP
APP
APPAPP
APP APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APP
APP
APP
APPAPP
APP
APP
APP
APPAPP
APP
APPAPP
APP
APP
Single Application(Project Team)
Dependent Applications(Program Teams)
10-100s Applications(Line-of-Business Portfolio)
100-1000s Applications(Enterprise)
PH
AS
E
1
PH
AS
E
2P
HA
SE
4
PH
AS
E
3
GOAL
GOAL
GOAL
GOAL
Goal+ +
+ +
+ CD PIPELINECD PIPELINES
+
+CD PIPELINES CD PIPELINES
+
26© Copyright 2016 EMC Corporation. All rights reserved.
Beyond Agile: DevOps/CD
Core Value: Our highest priority is to satisfy the customer through early and continuous
delivery of valuable software
Leading to Principles:
1. The process for releasing/deploying software MUST be repeatable and reliable
2. Automate Everything down to bare metal
3. If something’s difficult or painful, do it more often
4. Keep EVERYTHING in source control
5. Done means “released”
6. Build quality in
7. Everyone has responsibility for the release process
8. Improve Continuously
Leads to Practices:
1. Build binaries only once
2. Use precisely the same mechanism to deploy to every environment
3. Smoke test your deployment
4. If anything fails, stop the line
27© Copyright 2016 EMC Corporation. All rights reserved.
*ilities that matter in DevOps/CD
• Automatability
• Configurability
• Deployability
• Evolvability
• Monitorability
• Testability
28© Copyright 2016 EMC Corporation. All rights reserved.
Release Candidate Pipelines
Delivery Pipelines
VERSIONCONTROL
COMMITCREATECHANGE
• New software• Enhancements/fixes• Upgrades• Patches• Configurations
TEST UAT STAGE
PROD
Development & Build Process
1.1
1.2
1.3
1.4
1.5
1.2
Automated Tool Chain
29© Copyright 2016 EMC Corporation. All rights reserved.
Policy Wrappers = Workflows = Software Factories
• Quality Gates• Performance• Architecture• Compliance• Security• Operations
Deployment Environments Management & Monitoring
Version Control Build Tools Continuous Integration Tools Test Frameworks
Test
Manager
Commit & Build Application
Validate Code & Build
DeployApplication
AcceptanceTest
CapacityTest
DeployInfrastructure
Commit & Build Infrastructure
Validate Automation Package
Production Deploy
Version Control
Configuration Managment Artifact Repositories
API Library
API Library
Ideal DevOps Workflow
IDEAL DEVOPS WORKFLOW
POLICY
PO
LIC
Y
PO
LIC
Y
POLICY
30© Copyright 2016 EMC Corporation. All rights reserved.
Release Pipeline Stage Status
31© Copyright 2016 EMC Corporation. All rights reserved.
PEOPLE(Skills & Culture)
32© Copyright 2016 EMC Corporation. All rights reserved.
Introduce Strong Matrix Organization to Reduce Bottlenecks
Cross-Functional, Collaborative Teaming
GROUPING DYNAMIC TEAMING MODEL
GOAL
Operational SilosCentralized Management
Cross-functional TeamSelf-managed; Product Aligned
33© Copyright 2016 EMC Corporation. All rights reserved.
Create Space
• Separate from normal working environment
• Open, collaborative design
• Innovation “vibe”
• “In orbit” of Enterprise
“The Dojo”
34© Copyright 2016 EMC Corporation. All rights reserved.
Model Core Values and “Best Practices”
RECOMMEND PRACTICES:
• Cross-functional Teams
• Retrospective/Improve
• Peer Review/Code Analysis
• Test-Driven Development
• Version control
• Visible Dashboard
• Standup Meetings
VALUES
35© Copyright 2016 EMC Corporation. All rights reserved.
Support Systems and Training
COACHING & CHAMPIONS
TRAINING & DEVELOPMENT
ACTIVE PROJECT INTEGRATION
TOOLING SOLUTIONS
Access to physical, online, and
instructor-led content and labs to
build mastery
Thought-leadership and coaching for individuals and teams. Drives standards and
practices.
Linked to an active development
project to learn and practice on-the-job
(OTJ)
Tools chains integrated into day-to-day practice that support enterprise
objectives and standards
36© Copyright 2016 EMC Corporation. All rights reserved.
Transparent Operations
• Dashboards
• Drill-thru reporting
• Performance Metrics
• Value-driven portfolio management
• Capacity-based planning models
37© Copyright 2016 EMC Corporation. All rights reserved.
TOOLS(Platforms & Ecosystems)
38© Copyright 2016 EMC Corporation. All rights reserved.
Automated Tooling Mapped to Flow
Deployment Environments Management & Monitoring
Version Control Build Tools Continuous Integration Tools Test Frameworks
Test
Manager
Commit & Build Application
Validate Code & Build
DeployApplication
AcceptanceTest
CapacityTest
DeployInfrastructure
Commit & Build Infrastructure
Validate Automation Package
Production Deploy
Version Control
Configuration Managment Artifact Repositories
API Library
API Library
Ideal DevOps WorkflowAPI/clouds enables common
workflow to support multiple
tool and technology stacks
39© Copyright 2016 EMC Corporation. All rights reserved.
Comprehensive Hybrid Cloud Strategy
Self-Service Portal
Develo
pm
ent S
erv
ices
Management Services
EnterpriseHybridCloud
Apps Data Labs
SD
KCLI
API
Operate Subscribe Control
DEV TEST STAGE PROD
• Middleware• Operating System• Compute• Storage• Network IN
FRA
STR
UC
TUR
E
• Data• Application• Runtime• Middleware A
PP
LIC
ATIO
N
DEVOPS PLATFORM
40© Copyright 2016 EMC Corporation. All rights reserved.
Global Design, Local ImplementationsBuild continuous delivery capabilities based on SW Engineering Best Practices
Version Control
Assemble Dependencies
Unit Test BuildCompile BuildAnalyze Code
IaaS Provisioning
Configuration Automation
Middleware & Runtime
DeploymentPortal
Command Line
Code Commit
Manifest/Profile Creation
Analyze Code Unit Test Code
Workflow/Deployment
Engine
Service Catalogue
Environment Profiles/CMDB
Application Deployment & Configuration
Application Bootstrap
CONTINUOUS BUILD/INTEGRATION
PLATFORM AUTOMATION SCRIPTING
Application Environment-as-a-Service (EaaS)
Dashboard/Mgmt Tools
Smoke Test
Functional
Regression
TEST AUTOMATION SUITE
Performance
Penetration
Integration
System
Compliance
UI
Manage & Monitor, Log
Infrastructure Automation Tooling (FEHC)Configuration Tools (Puppet)
Application Development Tools
41© Copyright 2016 EMC Corporation. All rights reserved.
DevOps Application Rationalization
Application Rationalization Consumable by Lean-DevOps
Application COTS(Package)
OR
Middleware
Runtime
Internal Dependencies(libraries, frameworksversions, packages)
Configuration(s)
Container
OS
OS
VM
OS
Infrastructure
External Dependencies(DBs, tiers, SOA, RPC)
Network Storage Compute
Security Manage
ALM Artifacts
Source code, Schema
Configuration(s)
Configuration(s)
Configuration(s)
Services
App/Disposition(Modernize ALM)
42© Copyright 2016 EMC Corporation. All rights reserved.
Everything-as-code (XaaS)Repeatable, consistent, automated configuration
DECLARE DESIRED STATEIN READABLE CODE
VERSION CONTROL
ENVIRONMENT PROFILES
Application Environment-as-a-Service (EaaS)
Orchestration & Deployment Engine
• Storage• Compute• Network• Middleware• Runtime• Application
CONFIG
MANAGE TOKNOWN GOOD STATE
WORK FROMKNOWN GOOD STATE
1
2
3
BINARY
43© Copyright 2016 EMC Corporation. All rights reserved.
Pathway to “Cloud Native”
DE
VO
PS
LEGACY APPPORTFOLIO
REWRITE
REPLATFORM
RETIRERETAIN
MODERNDEVELOPMENT
Architecture , Standards
MigrationExecution
APP MIGRATION FACTORY
APPLICATION PORTFOLIO ANALYSIS
ApplicationAlignment
ApplicationInventory &Classification
NEWAPP DEV
PaaS
IaaS
COST
AVOID REDUCE
INNOVATE/MODERNIZE
DEV TEST STAGE PRODPIPELINES
UPGRADES
EnterpriseHybridCloud
PUBLIC
NATIVE HYBRID CLOUD
44© Copyright 2016 EMC Corporation. All rights reserved.
Success Patterns
45© Copyright 2016 EMC Corporation. All rights reserved.
Assess The Value Stream (First Steps)
46© Copyright 2016 EMC Corporation. All rights reserved.
Build Foundation
GOAL
xFUNCTIONAL TEAM COLLABORATIVE SPACE
DevOpsDoJo
TOOLS
PROCESS & BEST PRACTICE
STANDARDS & GUIDELINES
COACHING & CHAMPIONS
47© Copyright 2016 EMC Corporation. All rights reserved.
Prove DevOps
APPSingle Application
(Project Team)VIA
BIL
IT
Y
GOAL+ +CD PIPELINE
“MVP DevOps Project”
48© Copyright 2016 EMC Corporation. All rights reserved.
Measure Results
“If someone where to accuse you of being successful, could you prove it?”
-Dean Webster, VP Product Development
49© Copyright 2016 EMC Corporation. All rights reserved.
then Scale
APP
APP
APP
APP
CD PIPELINECD PIPELINES
GOAL
TEAM
GOAL
GOAL
GOAL
Goal
TEAMS
TOOLS
PROCESS & BEST PRACTICE
STANDARDS & GUIDELINES
COACHING & CHAMPIONS O
NB
OA
RD
IN
G
Legacy Enterprise
50© Copyright 2016 EMC Corporation. All rights reserved.
John Shook, 2010 MIT Sloan Management Review
Modern Change model
CULTURE
VALUES AND ATTITUDES
WHAT WE DO
TRADITIONAL MODEL:Change thinking to change behavior
Distributethen
Inspect
Provethen
Radiate
DEVOPS MODEL:Change behavior to change thinking
51© Copyright 2016 EMC Corporation. All rights reserved.
Set & Manage Expectations
ARCHITECTURE & STANDARDSENTERPRISE & IT GOALS
MANAGE TO OUTCOMESNOT TO TASKS & ACTIVITIES
• Performance goals
• Business objectives
• Budgetary goals
• Innovation targets
• Etc.
DefineProgram & Team Level Metrics
Development
Architecture
Compliance
Quality
Etc.
•
•
•
•
•
Define DONEFor Enterprise,
Programs, and Teams
54© Copyright 2016 EMC Corporation. All rights reserved.
Maturity AssessmentDevOps/Agility CMM Scores
Survey Results
Level 1 – Initial Level 2 – Managed Level 3 – Defined Level 4 – Quantitatively Measured Level 5 - Optimizing
1.01.52.02.53.03.54.04.55.0
Culture
Organization
People (skills)Process
ToolsDimension Average of Score
Culture 3.7
Organization 3.2
People (skills) 3.1
Process 3.3
Tools 3.2
Grand Total 3.3
55© Copyright 2016 EMC Corporation. All rights reserved.
Antipattern: Have you read these?
56© Copyright 2016 EMC Corporation. All rights reserved.
How can the Federation help us with…
DevOps Transformation?
57© Copyright 2016 EMC Corporation. All rights reserved.
Why the Federation for DevOps?
Single source for DevOps best-of-breed solutions
• Puppet Labs
• Chef
• Microsoft
• CloudBees
Focus on People & Process; Not just technology
Strategic Partnerships w/Leading Vendors
• Business alignment
• Organizational structure
• Application lifecycle policies and procedures
• Development languages (Java, .Net, Ruby, etc.)
• Virtualization platforms (vSphere, HyperV, OpenStack)
• DevOps tools (Puppet, Chef, TFS, etc.)
Evolutionary approach to the cloud
Agnostic approach
• DevOps enabled Cloud Migrations
• Maximize value of brownfield applications
• Empower greenfield application development
58© Copyright 2016 EMC Corporation. All rights reserved.
Service Models
Strategic and planning services that design and define a roadmap and phase 1 (MVP) devops/continuous delivery plan tailored to an enterprise’s unique maturity and readiness.
ADVISOR
Joint partnership to implement continuous delivery tool chains and seed a devops
cultures within an enterprise
Federation-led project to implement prescribed continuous delivery tool chains
that compliment and influence devops initiatives within an enterprise
NAVIGATOR DRIVER
59© Copyright 2016 EMC Corporation. All rights reserved.
Full Spectrum Portfolio Support
Bare Metal VirtualizedContainers &
Services
Enterprise Application Portfolio
DEVOPS ENABLED CONTINUOUS DELIVERY
TOOL CHAIN
• Agile Infrastructure• Multi-Modal Capability• Software Defined• Fully Automated
RESILIENCY AGILITY SPEED
VMware OpenStack Other Provider
60© Copyright 2016 EMC Corporation. All rights reserved.
RE-ENGINEER
Proven Approach
CROSS FUNCTIONAL
DEVOPS TEAM
OPTIMIZED PROCESSES
CONTINUOUS DELIVERY TOOL CHAINS
INDUSTRY LEADING TOOOLS
OPEN PARTNER COMMUNITYAPPLICATION DELIVERY
PIPELINE (SDLC)
62© Copyright 2016 EMC Corporation. All rights reserved.
Improvement Storyboards – A3 Management