Cloud & DevOps

the new normal for digital & ecommerce businesses

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 2

WHO’S THIS EBOOK AIMED AT?

Cloud and DevOps - the new normal for digital and ecommerce business is aimed at dev agencies, app developers, digital/creative agencies, online retailers, basically any organisation where writing code is central or very important to what they do.

If you’re curious about how the cloud or DevOps can benefit your business, this book will give you all the information you need to move forward.

IT’S NOT YOUR FIRST DAY ON EARTH

You’ve heard about DevOps and the cloud. Some of it sounds good. Some of it sounds like hype. You’re willing to learn more though as we’re all hearing more and more about it.

We’ve not written this ebook to dive into the argument about what DevOps should or shouldn’t be. We’ve written it to help you understand exactly what DevOps really is, going into what works and what doesn’t.

Similarly, we’ve not written this to push people to use the cloud - we want to get rid of the hype and bullshit surrounding it so you can see how beneficial for your business the cloud can be.

Cloud & DevOps the new normal for digital & ecommerce businesses

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 4

WHAT IS DEVOPS, AND WHY SHOULD I CARE?

Is it a culture? A job title? A piece of software? A way of thinking? Or an entire new philosophy?

There are loads of definitions for DevOps and maybe it’s important not to get bogged down with trying to define it, but let’s try anyway…

DevOps means:

• new tools • new ways of working • embracing cloudDevOps means development, operations and QA. The ideology behind it is getting developers and operations teams to work closely together to benefit the business. The goal is to reduce friction and increase velocity.

The key goal of DevOps is to improve collaboration. From planning through to delivery and automation of the delivery process. This leads to:

• Improved deployment frequency • New features introduced faster• Quicker turnaround from idea to delivery• Faster time to market• Smaller, more isolated bugs that are easier to fix • Shortened lead time between fixes• Improved time to recovery

The key concept is that the end result is a streamlined development and operations team which leads to faster, and more accurate delivery of code.

It should be a journey to making your application better and deployed faster.

It doesn’t have to be complicated. It requires work upfront, but the long-term benefits and competitive advantage it gives you makes it worth it.

Put simply, the end goal of DevOps is to speed up the coding process.

OPERATIONS

(Quality Assurance)QADEVELOPMENT

(Software Engineering)

DevOPS

CODE

BUIL

D

MONITOR

DEPLOY

REL

EASE O

PERATE

PLAN

TEST

“HIGH-PERFORMING IT ORGANIZATIONS DEPLOY 200 TIMES MORE FREQUENTLY THAN LOW PERFORMERS, WITH 2,555 TIMES FASTER LEAD TIMES” PUPPET’S 2016 STATE OF DEVOPS REPORT

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 6

PEOPLE GO DOWN THE DEVOPS ROUTE FOR VARIOUS REASONS. DEVELOPERS, OPERATIONS AND LEADERS WILL ALL HAVE DIFFERENT INTERESTS IN DEVOPS; YOU’RE PROBABLY ONE OF THESE PEOPLE.

Developers:

A lot of developers have mixed feelings about DevOps. In some cases, their involvement can appear to be extra work for them, even though the end result will save them time.

An automated deployment toolset provides a seamless workflow for developers, encouraging more frequent deployments. Instead of 1 big monthly deployment with 100 new features, you deploy new features individually, hugely reducing the scope for bugs.

Operations:

Without DevOps, the standard approach is for developers to throw a pig over the wall and immediately expect bacon. With DevOps, the shared responsibility model ensures that everybody is on the same page at all times.

Leaders:

It’s no secret that there’s usually a disconnect between the leadership and coders. As DevOps is a fairly new way of doing things, it can sometimes be tough for the leadership to move.

Typically speaking, leaders want a solution which is quick to implement, and has immediate results.

It does however have a huge amount of long term benefits. Cultural changes can be a shock for leadership teams, but it also yields the biggest dividends.

Typically speaking, leaders want a solution to be fast. DevOps isn’t always the quickest solution.

Whilst DevOps isn’t quick to implement, and can require a significant culture change, it does however have a huge amount of long term benefits.

WHAT IS THE CLOUD, AND WHY SHOULD I CARE?

You know what the cloud is. As much as anyone else does, given how woolly the term is.

In this setting, the cloud refers to large-scale public cloud hosting, such as Amazon Web Services (AWS), Google Cloud Platform (GCP) or Microsoft Azure.

These environments are rapidly becoming the new normal for hosted applications and sites, because of their flexibility, scalability and reliability.

The concept of ‘this is my collection of physical servers’ is dated. When used correctly, large-scale public cloud will save you money, take away headaches associated with physical kit, and allow you to focus more on developing, rather than operations.

IS DEVOPS FOR EVERYONE?

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 8

DEVOPS SUCCESS STORY - PWCIncumbent firms are always under pressure from the disruptive nature of new entrants

to their markets, such as fintech startups within the financial services sector.

LEAN and DevOps approaches can help the incumbent organisation to improve themselves and, in a manner of speaking, disrupt their existing processes and

attitudes to “what is good” before someone else does.

The key is to take a holistic view of the whole problem - not to limit your focus to a single technology product, but to look at how overall cost, quality and time

metrics are impacted.

When areas of improvement are identified, spend a short time understanding what changes can be made, along with agreeing on key performance indicators that will

measure improvement and how the overall process benefits.

Always seek continual improvement and not unobtainable perfectionEach possible change should be time boxed, and ideally within short cycles of sprints to provide feedback both externally and internally. Generally the 80:20

rule applies and you’re looking for improvement, not perfection.

This may be by taking an existing set of quality gates, such a regulatory or non-functional tests, and making them part of the delivery process, so that any formal

regulatory validation becomes a true validation of work that has already been tested informally within the process.

Each project may display differing issues. Organisations and teams comprise of individuals, and it’s these individuals that are ultimately the intrinsic part of

any improvement process.

WITHOUT DEVOPS, THE STANDARD APPROACH IS FOR DEVELOPERS TO THROW A PIG OVER THE WALL AND IMMEDIATELY EXPECT BACON.WITH DEVOPS, THE SHARED RESPONSIBILITY MODEL ENSURES THAT EVERYBODY IS ON THE SAME PAGE AT ALL TIMES.

WHY MERGE DEVELOPMENT AND OPERATIONS?

“The notion of DevOps is the realisation that the development process doesn’t end in development. We can’t disconnect the operations piece of software (development) from those other phases of development”. – Ryan Berg (The Case for DevOps)

Before DevOps started growing in popularity, there was no clear link between development and operations.

Devs were responsible for the building of application and operations were responsible for implementing and supporting the solutions.

In a lot of organisations there were metaphorical silos separating these two functions, meaning communication was infrequent and not necessarily relevant/up to date.

There was no formal method of communication between the two teams other than when problems arose during deployment.

But with DevOps, instead of throwing the pig over the fence and hoping it turns into bacon by the time it touches the ground in operations, the relationship between the two teams is changing.

application

development

DEVOPS

operations

PLATFORM

OPERATING SYSTEM

INFRASTRUCTURE

“PEOPLE TURN TO DEVOPS NOT JUST TO IMPROVE DAILY WORKING LIFE AND GET TIME BACK FOR FAMILY, FRIENDS AND BEER, BUT TO IMPROVE THEIR ORGANIZATION’S PERFORMANCE, REVENUES, PROFITABILITY AND OTHER MEASURABLE OUTCOMES.”PUPPET’S 2016 STATE OF DEVOPS REPORT

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 10

WHAT SHOULD YOU EXPECT TO GET OUT OF DEVOPS?

SO FAR IT SOUNDS LIKE THERE’S A FAIR BIT INVOLVED WITH DEVOPS, RIGHT? LUCKILY, THE OUTCOMES ARE PRETTY AWESOME.

Faster, more frequent deployments

You don’t have to choose between speed and accuracy. Your dev, staging and production environments will be identical, because the infrastructure has been written as code. This means there’s no chance of differences between the environments.

An increased number of smaller deployments means that if there are any problems with the code it’s a much easier job to fix.

Because of this you can be confident that code releases tested in staging can be pushed to your production environment and just work.

Your deployments will be faster and more frequent too. This is because the whole ‘making stuff live’ process is no longer a big deal that’s scary and fraught with worries.

All this leads to a massive speed advantage over your competitors.

BUSINESSES THAT ADOPT THE DEVOPS PHILOSOPHY HAVE “24 TIMES FASTER RECOVERY TIMES AND THREE TIMES LOWER CHANGE FAILURE RATES.”PUPPET 2016 STATE OF DEVOPS REPORT

CONFIGURATION MANAGEMENT

Configuration management is the process of systematically handling changes to your infrastructure, in a specific way where it maintains integrity over time. Automation is the heart of configuration management for servers, so there’s no wasting time on repeating processes.

Once you’ve defined the desired state of your infrastructure and applications within your chosen configuration management tool, you can pretty much sit back and relax. Your environment is continuously and automatically enforced in that state, meaning there are not as many little niggles for you to deal with.

Configuration management is all about consistent infrastructures and auditable, automated changes.

30X# of Deploys

non-DEVOPS DEVOPS

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 12

ORCHESTRATION

Orchestration is where you arrange or coordinate multiple environments.

You can use orchestration to do a variety of things, including running a migration on all of your database servers, checking the current Ruby patchlevel of each Rails server and finding any errant Node.js processes and terminating them.

Orchestration is especially important in the delivery of cloud services because they are intended to scale-up both dynamically, without needed human intervention.

BREAK DOWN BARRIERS BETWEEN DEV AND OPS

As we’ve said above, your dev and ops teams effectively merge which means greater cohesion and leads to much more efficient use of team members.

Knowledge sharing is a key component of the DevOps way of working, and for good reason. It’s no secret that bad things happen when the communication between the traditional dev and ops teams breaks down.

This also gives developers the skills to become a full-stack engineer.

More efficient use of team members means that you can save time where it really counts, ultimately saving money too. It’s a win-win!

WHAT DOES IT LOOK LIKE IN PRACTICE?

Our friends at New Relic have a great explanation of the DevOps deployment lifecycle. Here’s how it looks:

1. Check in code2. Pull code changes for build3. Run tests (continuous integration server to generate builds and arrange releases): Test individual models, run integration tests, and run user acceptance tests4. Store artifacts and build repository (repository for storing artifacts, results, and releases)5. Deploy and release (release automation product to deploy apps)6. Configure environment7. Update databases8. Update apps9. Push to users – who receive tested app updates frequently and without interruption10. Application and Network Performance Monitoring (preventive safeguard)11. Rinse and repeat

Develop & TEST

FEEDBACK

DEVELOP

TEST

DEPLOY

MONITOR

ReLEASE & MONITOR

DEVOPS SUCCESS STORY - AUTO TRADER

I’ve been at Auto Trader since 2007, and have seen a big change in how development, product and operations work together since that time. When I started, product and technology were completely separate teams. Technology was spread across seven floors in three buildings (on our main site) with an operations department organised into

a hierarchy above single discipline teams.

Software deployment involved sending emails with Word documents attached. These would contain configuration and details of the source version to build manually. Human beings would copy and paste the configuration entries from the Word document (along with extraneous whitespace that Word liked to randomly insert) which often resulted in failed deployments. Incident review processes were painful or non-existent, and major incidents were always followed by masses of blame. There was almost zero collaboration between development and operations and, as a result,

very little trust.

These days, things are very different. We are now years into a process of change and improvement. A heavy focus has been placed on automation.

Where we previously had very manual deployment processes we now have developers and testers initiating an automated process to deploy their product. As a result

teams feel more responsible for the stability of their product.

Our support team in operations has several layers of visualisations across our production environment and a much clearer handle on where problems lie as a result.

In terms of the benefits we’ve seen as part of this journey, we see this as a core part of our long term strategy - applying continuous learning to the delivery of our products and facilitating a better experience for our customers. That might sound like business speak, but it is important to note that we have buy-in at all levels.

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 14

THE NITTY GRITTY OF DEVOPSDevOps isn’t about specific tools, it’s about the philosophy. That said, there are various factors that can make your life a little easier as you begin adopting it. This section might be teaching you to suck eggs if you’re experienced, but we can pretend you’re starting at the bottom.

Cultural shift

DevOps may require a collective culture change in your team. Everyone needs to be on the same page, and committed to making DevOps work. Although it might initially be difficult, it’s hugely beneficial in the long run.

Configuration management/infrastructure as code

We outlined configuration management above, and we’d say that Puppet or Ansible are the best tools to aid you here.

Loads of businesses currently rely on Puppet, including Salesforce, Verizon and Nasa. If the guys that send people to the moon use it, we’d say it’s pretty good.

Application deployment tools for continuous delivery and testing

Continuous delivery is a major part of DevOps. It ensures that code can be safely deployed at any time, so choosing the right software is vital.

What you choose is up to you and your team. There’s Terraform, CloudFormation, Ansible, and Jenkins.

Code Repository for version control

There’s only one repository you’ll need: Git. You’ll no doubt already know and love Git like we do.

The cloud

You need somewhere to run your stuff, don’t you! This place is the cloud. Cloud computing services like Amazon Web Services, Google Cloud, or Microsoft Azure are good places to start. Even better, get in touch with us and we can talk you through it.

As well as running your dev, the cloud is where you’ll test everything and ultimately go live too.

Monitoring and metrics

It goes without saying that you need to be able to keep track of exactly how your software environment is doing. New Relic allows you to view and analyse a massive amount of data, receiving actionable insights in real-time.

TAKE FULL ADVANTAGE OF THE CLOUD

With your infrastructure written as code and standardised environments you’ll be able to take advantage of those super-sexy features of large-scale public clouds like AWS, Google or Azure. That’s stuff like:

Autoscaling - where your site or app consumes only the resources it needs at that time. Saving you money as you don’t need those expensive.

Higher availability (HA) - because of the standardised images that’ve been built, it’s simple to build out an identical infrastructure across availability zones, or even in a different region, meaning building HA into your application or site is simple.

Geographical diversity - if you have a presence in multiple regions, or worldwide, it’s simple to build out an identical infrastructure in a datacentre, or datacentres, across the other side of the world, meaning fast loading times for all of your users.

You don’t need to specifically use the cloud to take advantage of a DevOps methodology, but without utilising a large-scale public cloud provider you’ll be missing out on some of the best benefits of DevOps..

WHAT TOOLS ARE REQUIRED?

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 16

The blue-green deployment approach is also pivotal in the continuous delivery stage. This is where you have one production environment running 2 versions of code. 1 version blue, 1 version green. At any time one of them is live. For example, let’s say in this instance that blue is live.

Blue would be the place where 100% of your visitors/users would be, and green the place where you implement changes. Over time you’d slowly move visitors/users over to the new green version. You might start slowly with a 90% blue, 10% green split, but that’s up to you.

Once everything is hunky-dory in the green environment, you switch the router so that all incoming requests go to the green environment...the blue one is now idle.

Lean Development

Lean development requires the developer/teams to be as lean as possible, identifying and eliminating waste to hasten the development process.

Maximising the production levels of everyone in the development team is fundamental in the journey to DevOps.

Utilising lean development also means that changing requirements, even at a late stage, can be dealt with quickly. A focus on failing fast means that your team can quickly adapt to any situation.

Continuous Integration

Continuous integration is the practice of speedily integrating new code with the main body of code that’s set to be released. This saves a lot of time when the team is ready to release the code.

From a DevOps perspective, continuous integration provides a barrier prior to deployment where the application code is fully tested.

These tests can cover anything from basic unit tests to test individual functions, to acceptance and business case testing.

You can also do security testing, for things like PCI or ISO27001 compliance.

Finally, it's good practice to add tests for any bugs that have been introduced and resolved. This ensures that any code regressions would ensure that bug is never reintroduced.

Continuous Delivery

Continuous delivery is an extension of continuous integration.

It’s a series of practises designed to ensure that code can be rapidly and safely deployed, at any time.

Since every change to code is delivered in a staging environment using automation, you’ll know for sure that the application can be deployed with a push of the button whenever the business is ready.

You’ll get the code nearly ready to deploy with almost no human interaction.

PHASES OF DEVOPS MATURITY (YOUR ROUTE TO DEVOPS)

LEAN: Frequent testing cycles - basically the application test suite being run whenever new code is written, automatically.

CULTURE: The entire company adopting the DevOps philosophy, from the top down.

METRICS: These can be any metric relevant to the business really. From application performance, user experience, etc. Basically metrics that MD types can understand.

AUTOMATION: Automation of the development cycle.

SHARING: Collaboration between dev and ops.

web SERVER

ROUTER

APP SERVER

shared database

APPLICATION V2

APPLICATION V1

CLOUD & DEVOPS

WWW.STEAMHAUS.CO.UKSTEAMHAUS

PAGE 18

HOW CAN OUTSIDE HELP, HELP?Even though we’ve made this eBook to try and make DevOps and the cloud as easy to understand as possible, it can still be very complicated.

Outside help from a consultancy can save you a lot of time and effort.

ABOUT STEAMHAUS AND OUR MANAGED DEVOPS PACKAGE Managed DevOps from Steamhaus means that we’ll act as an extension of your team.

You’ll rely on us to look after every aspect of the infrastructure that underpins your site or application. You can trust us to make sure everything is running perfectly.

We’ll use some of the tools described earlier in this eBook to automate every aspect of your environment, and we’ll use these tools to make sure that you get all the benefits of true cloud hosting.

We’ll then monitor and maintain your hosting on an ongoing basis, leaving you to get on with what you’re good at: writing code.

DEVOPS SUCCESS STORY - RESPONSETAPBefore our move towards the DevOps ‘nirvana’, symptoms we experienced were Ops individuals only becoming involved at times of crisis or incident, and

being hugely protective of any system.

Our developers had very limited awareness of the production context and environment, leading to siloed decision making. Minimal automation for deployments and manual build processes meaning reliance on both Dev and

Ops to get a release out; Limited visibility of production logs/monitoring due to access restrictions, and 'only if essential' communication between

development and infrastructure.

Fast forward 12 months; where are ResponseTap now? We have transformed our team structure to foster collaboration and taken up a ‘Spotify squads’ inspired model, allowing teams to function autonomously. A single squad

involves dev, test, product and ops individuals all planning work together and visibility into decision making.

Our monitoring and logging has become publicly available to the squads. Production environments are now co-owned by the squad - so much so that we often have testers alerting or checking items in production. We have automated builds, meaning we deploy multiple times a day rather than once

a week.

All that said our biggest indication of being “DevOps” - a recent work football game Ops and Dev on the same side, two goal scorers in that match - one from Dev and one from Ops, so are we “DevOps"? We’re getting there….

G U I

DING

YOUR W

AY

THRO

U

GH

THE

CL

OU D

GET IN TOUCHHave a new project in mind? Want to enhance your current solution? Let’s talk.

UNIFORM RESOURCE LOCATORwww.steamhaus.co.uk

STATIC TELEPHONE0161 820 2020

ELECTRONIC MAILhello@steamhaus.co.uk

LOCATIONFourways House, 57 Hilton Street Manchester, M1 2EJ