2014 state of devops findings! velocity conference

DevOps Means Business

Presented by

Nicole Forsgren Velasquez | Jez Humble | Nigel Kersten | Gene Kim

Findings From Puppet Labs State Of DevOps 2014

Velocity Conference, Santa Clara

June 26, 2014

Puppet Labs Webinar 2014 State of DevOps


Agenda

● Demographics● Key Findings● Survey Methodology and Analysis● IT Performance● Organizational Culture● Job Satisfaction● Big Surprises● Future Plans● Q&A

Gene Kim

Multiple award-winning CTO, researcher and co-author of The Phoenix Project: A Novel About IT, DevOps, and Helping Your Business Win. First time Google Slides user.

Presenters

Nicole Forsgren Velasquez

Professor at the Huntsman School of Business at Utah State University. IT impacts, knowledge management, and analytics consultant and researcher. Stats ninja.

Jez Humble

Principal at Thoughtworks, Lecturer at UC Berkeley, co-author of Continuous Delivery, and accomplished procrastinator.

Nigel Kersten

CIO at Puppet Labs and responsible for IT/Operations there. Almost fully recovered sysadmin.



9,200+ Respondents from 110 countries, across all industries

Demographics

@botchagalupe#devops survey early observations #4 Retail, Banking,

Finance and Healthcare respondents LT 15% total (typically the risk averse

sectors)

https://twitter.com/botchagalupe

https://twitter.com/hashtag/devops?src=hash


27% of

responses from companies with 500 to 9,999 employees

Size of Organization



Size of Infrastructure

51% of

respondents said their infrastructure included <500 Servers

@botchagalupe#devops survey early observations #1

Only 40% of respondents have more than 500 servers

https://twitter.com/botchagalupe


Sze Wa Cheung

[email protected] I added a click fade in effect. That should help to break it down.

Alanna Brown

Thanks!

Alanna Brown

[email protected] is this too cluttered? Any way to make it look better?


Departments

16% identified asDevOps Department

Alanna Brown

[email protected] I think this is an interesting stat to call out. Anything else you want to call out here?

Michelle Carroll

If you're looking for demographic points of interest, allow me to show those john willis tweets I mentioned. He did a nice job of pulling out some interesting factoids: https://twitter.com/botchagalupe/status/474198278427643904https://twitter.com/botchagalupe/status/474198909116760064https://twitter.com/botchagalupe/status/474199279444455424https://twitter.com/botchagalupe/status/474200937750626305https://twitter.com/botchagalupe/status/474201143095361536

Alanna Brown

These are great, Michelle! Thanks so much for pulling these.

2012

2013

2014

DevOps Grows Up

What is devops?

DevOps adoption is accelerating.

Holy cow! DevOps works!


Why I Care: IT Performance Matters


Cross-population Studies

Tradeshows

Industry Press

Blogosphere

Vendors

Consultants

Industry Analyst

Anecdote

Peer Recommendation

Prior Experience

ROI Story Best Practice

Benchmark

Population Study

Academic Study

Intuitive Analytical

We Are Here

High-performing IT Orgs are More Agile


30x 8,000xmore frequent deployments

faster lead times than their peers

High-performing IT Orgs are More Reliable


2x 12xthe change success rate

faster mean time to recover (MTTR)

Firms with high-performing IT orgs are winning

more likely to exceed profitability, market share & productivity goals


2xhigher market capitalization growth over 3 years*

50%

How We Collected Responses


Snowball Sampling

Questions We Asked


Likert-type questions give us a more nuanced measurement.

Latent Constructs


Some things can’t be measured directly, like happiness.

“Happiness Statistics” by Andrew Tarvin is licensed under CC BY 2.0

https://www.flickr.com/photos/andrewtarvin/10845547513/in/photolist-6o69P5-hwogqv-4EL2us-kCzuq-4EL2ru-8groME-ir7oZV-4EFKZx-9dYQva-diKMwy-i8qGT3-61WLzv-nNutS-6BGRY-9xYZkM-4EFKYF-esixgq-auRNAo-7xxdeW-4EL2sC-9y5MZC-esizZ5-brLxbf-nnSgNv-Lsd1m-atEnp2-kg6g3-4hF9py-hGGvQ-4LZfR9-CepHT-4EFKZe-8yL3CN-4EL2v5-5fiUgr-5Ao7bD-9dkjCb-rabrT-dh3UaM-b7uPBR-5VuX7M-4Kj3Bg-6v6UH9-mCmc9F-9FXhd1-6iAwTk-bRHoFz-jtL5H-9pkFfT-9bjjGg

https://www.flickr.com/photos/andrewtarvin/

http://creativecommons.org/licenses/by/2.0/

Quantifying IT Performance


Deploy

30X more frequently with

50% fewer failures

● Deploy Frequency

● Mean Time to Recover (MTTR)

● Lead Time for Changes

Categorizing Companies: High, Medium, Low IT Performance


Deploy Frequency

Mean Time to Recover (MTTR)

Lead Time for Changes

MTTR

Lead Time

Deploy Frequency

High-performing IT orgs

Medium-performing IT orgs

Low-performing IT orgs


Practices Correlated with IT Performance Metrics

Throughput Metrics Stability Metrics

Deployment Frequency● Continuous Delivery● Version Control

Mean Time to Recover (MTTR)● Version Control● Monitoring System and

Application Health

Lead Time for Changes● Version Control● Automated Testing

Change Fail Rate ● Not strongly correlated with

specific practices.● Significant differences between

groups with high, medium and low change fail rates. High performing IT orgs have 50% lower change fail rates than their peers.(mention Roy Rapoport!)


Top Predictors of IT Performance• Peer-review of production changes (vs.

external change approval)• Version control of all production artifacts• Proactive monitoring of the production

environment• High trust culture• Win-win relationship between Dev, Ops and

Infosec• High job satisfaction


Feedback Loops

Source: Elisabeth Hendrickson: Care And Feeding Of Feedback Loops

Alanna Brown

[email protected] I dropped some of your notes from the report outline into the notes section.


Organizational Culture

15% 52% 33%


Organizational Culture Resources● http://www.thisamericanlife.org/radio-archives/episode/403/nummi

● http://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/

● Read Toyota Kata by Mike Rother

http://www.thisamericanlife.org/radio-archives/episode/403/nummi

http://sloanreview.mit.edu/article/how-to-change-a-culture-lessons-from-nummi/


Job Satisfaction

Job satisfaction is the # 1 predictor of organizational performance!

Top Predictors of Job Satisfaction

★ High-trust organizational culture

★ Climate of learning★ Win-win relationships

between ops, dev and infosec teams

★ Proactive monitoring and autoscaling

★ Use of version control for all production artifacts

★ Automated testing

Gene• Version control of the environment is more

important than version control of the code!• There’s never been a better time to know

statistics

Jez• Whether or not you have an integration or

stabilization phase has zero impact on IT perf• Peer review is more effective than CAB

Surprises


Can Large Orgs Be High Performers?


Yes.

But orgs with 10,000+ employees 40% less likely to be high performing vs.

500 employee orgs…

Can Large Orgs AdoptThese Practices?


Yes!(Automated testing,

Continuous integration, proactive monitoring…)

The only practice not being adopted is Peer Review vs.

Change Approval!

Nicole• DevOps practices and IT performance impact

organizational performance• Change fail rate wasn’t part of IT performance

Nigel• Forming new DevOps teams and giving people

DevOps titles is successful in practice

Surprises


• Collect more stock tickers!

• 2014-15 research goals

• What do you want to know?• Put your ideas and problems you want to

explore in this Google+ Community:http://bit.ly/FutureStateOfDevOps


Future Plans

http://bit.ly/FutureStateOfDevOps

Alanna Brown

Audience poll?

nicole forsgren velasquez

[email protected] [email protected] [email protected] i threw in some research goals for next year -- check them out and delete the ones that are stupid and/or top secret and add any of your own.

Nigel Kersten

added one about teams/roles with "DevOps" in their title.

O&Abit.ly/puppetquestions

Sept 3-4 | FlowConFlowCon brings together technologists and industry leaders passionate about innovation through continuous delivery, continuous design, and lean product development. Register at http://flowcon.org/

Sept 23-24 | PuppetConf Join us for the IT automation event of the year. Register now to save $249. http://puppetconf.com

Oct 21-23 | DevOps Enterprise SummitA conference for horses, by horses, featuring Macy’s Disney, GE Capital, and more. Register at http://devopsenterprise.io/

Nov 9-14 | LISA ‘14Systems engineering and ops professionals share real-world knowledge about designing, building & maintaining critical systems. https://www.usenix.org/conference/lisa14


Upcoming Events

http://flowcon.org/

http://puppetconf.com/

http://devopsenterprise.io/

https://www.usenix.org/conference/lisa14

Resources• Download the 2014 State of DevOps Report:

http://puppetlabs.com/2014-devops-report

• Pre-order Jez’s forthcoming book Lean Enterprise: Adopting Continuous Delivery, DevOps, and Lean Startup at Scale: http://amzn.com/1449368425

•Sign up for Gene’s newsletter to receive a sample chapter of his forthcoming book DevOps Cookbook: http://itrevolution.com/blog

http://puppetlabs.com/2014-devops-report

http://amzn.com/1449368425

http://itrevolution.com/blog

2014 state of devops findings! velocity conference

Engineering

researcher

author