implementing blue-green deployment with atlassian bamboo

Implementing blue-green deployments with Bamboo Overview of blue-green deployment followed by group discussion

Twitter: @dkcwd

First a little background….

Purpose of the session During the July 2016 Sydney Atlassian DevOps meetup there was a request from group members to discuss blue-green deployments using Bamboo and Bitbucket.

We’re here to discuss blue-green deployment, sharing knowledge on what can work well and discuss strategies for automating blue-green deployments.

This session will focus on a subset of the Atlassian tools however, we will discuss other tools too.

The session should: provide an overview of how to manage blue-green deployment with Bamboo encourage you to get started with your own automated blue-green deployments

Reminder about this session We may have people of all levels of experience in this session.

The initial slides should align us with a common understanding of blue-green deployment so we can explore the topic as a group.

When we begin the wider discussion please feel free to share your knowledge and experience with the group and ask questions too.

What is blue-green deployment?

?

Blue-green deployment overview

Blue-green deployment is a strategy for reducing risk when making changes which are due to be promoted into the production environment.

Successful implementation should result in a reduction of risk and friction for all involved in the deployment process.

So….let’s talk about what that looks like.

Overview Prior to promoting changes to a production environment, a separate “parallel” stack of resources is created.

This can allow near seamless switching between the stacks if behavior is in line with expectations.

Overview The ”current” production stack is referred to as blue and the “new” stack which has been created is referred to as green.

Ok, that sounds expensive….is it?

Potentially! We’re talking about creating a clone of production environment resources.

It makes sense to assess whether it is the right approach for your SDLC.

Understand the cost of downtime as well as the cost of duplicating your stack.

Frictionless deployments sound great however “bill shock” is quite the opposite.

Let’s continue…

Overview If behavior of the green stack is in line with expectations then the green stack should be promoted as “blue” with close to zero downtime.

If behavior is not in line with expectations then the switch should not take place.

If a “rollback” is required, it should be timely and low risk.

Overview If a green stack is promoted to blue in a cloud environment, the previous blue stack should probably be decommissioned.

Otherwise there will be some very unhappy CFOs….

Importance of environment boundaries

Persistent data stores and message queues are likely to be considered non-disposable and may require more complex strategy….

Importance of environment boundaries

Centralised data writing strategies may be required to reduce risk of corruption. Let’s look at some common blue-green deployment patterns.

Blue-green deploy patterns There is no one size fits all approach however it’s good to be aware of 2 patterns:

1. DNS cutover 2. Direct interaction with load balancers

DNS cutover Gradually filter user traffic from blue to green stack using DNS settings configured with a short time to live (TTL).

Monitor and if necessary ”rollback” so all traffic is diverted to the blue stack.

The key questions are, “how long does it take to switch over and how long to rollback?”

Direct interaction with load balancers

Avoid potential issues with DNS TTL and propagation.

Potentially leverage an additional internal load balancer to allow further testing.

Ok so now we are aligned on that, let’s recap what Bamboo is for.

A quick reminder of what Bamboo is for….

Build, Test, Deploy Build

Most front end application clients are composed from a variety of files and rely on assets such as images, JavaScript, CSS.

A build process will typically involve creating an ”artefact”.

An “artefact” could be a .zip file containing an optimized version of the application.

It could also be one of more files which will be useful for a more complex process.

Build, Test, Deploy Test

In this step the ”artefact” may be tested using one or more appropriate test methodologies.

If the tests pass then this should mean the artefact is “deployable”.

Note that there may be a number of additional steps which you may need to go through to verify whether the artefact is actually “deployable” and once you are aware of those steps the next step could be to automate them!

Build, Test, Deploy Deploy

In this step the tested ”artefact” is deployed using an appropriate strategy.

If the deployment process fails for any reason, ideally it will be easy to “roll back” until another attempt can be made.

If you are already familiar with these concepts then you should find it relatively easy to begin working with Bamboo following the documentation.

New build plans

There are options to create a new plan from scratch or clone one.

Making changes to a build plan

To edit a plan you will need to select “Configure plan”….

Configuring tasks related to a build

Each step can be configured using widgets or inline scripts….

Configuring tasks related to a build

*If checking out multiple repositories use a specific “Checkout Directory”

* Why? Because it makes it easier to use inline scripts such as: cd example-application/stack/php-api/src/ && php composer.phar install –vvv

Can we do blue-green deployment with it?

Almost anything is possible with time…

Consider leveraging build scripts create stacks plus artefacts for deployment then use a mixture of additional deployment and build tasks.

Consider leveraging existing add-ons in the Bamboo ecosystem or making calls from Bamboo scripts to external APIs.

There are plenty of different tools out there to choose from including some which have specific blue-green workflows.

An interesting platform to investigate…

https://fabric8.io/

Let’s discuss blue-green deployment strategy….

(group discussion)

So where to from here? Great question!

Research whether blue-green is right for your SDLC and if so, start experimenting…

Some useful links for additional reading, in particular, investigate how to use a “serverless” approach to trigger changes. http://martinfowler.com/bliki/BlueGreenDeployment.html

https://www.thoughtworks.com/insights/blog/implementing-blue-green-deployments-aws

https://botleg.com/stories/blue-green-deployment-with-docker/

http://dan.bravender.net/2014/8/24/Simple_0-Downtime_Blue_Green_Deployments.html

https://marketplace.atlassian.com/plugins/org.gaptap.bamboo.cloudfoundry.cloudfoundry-plugin/server/overview

https://www.youtube.com/watch?v=aX54mhZbN58

http://fbrnc.net/blog/2016/05/green-blue-deployments-with-aws-lambda-and-cloudformation

Enjoy and share what you do….

I’m on Twitter: @dkcwdLinkedIn: https://au.linkedin.com/in/daveclarkprofile