Puppet and AWS is easy …… ?

Cara Crawford and Scott Coulton

Who are we? DevOps/Infrastructure Manager

2  

Cara Crawford •  8 + years development experience •  6 + years operational sysadmin experience •  Extensive experience across website solutions in both datacentre

and cloud solutions across Linux and Windows environments

   

@cara_p

Who are we? DevOps Solution Architect

3  

Scott Coulton •  10 + years hosting solutions experience •  Extensive experience in architecture across AWS, Puppet, Linux,

Docker and all things open source Sc

   

   

h#ps://github.com/sco#y-­‐c  h#ps://registry.hub.docker.com/repos/sco#yc/  

h#ps://forge.puppetlabs.com/sco#yc  

@sco#coulton  

Who are we?

4  

Healthdirect Australia manages the following healthcare service: •  Healthdirect nurse helpline and health information •  After hours GP helpline •  Pregnancy, Birth and Baby •  Mindhealthconnect •  National Health Service Directory •  My Aged Care •  Video Call Consulting

   

What we were trying to solve August 2013

5  

•  Majority of servers located in AWS •  Manual deployments •  Cloning for provisioning •  Inconsistent environments •  No CD •  No CI •  No Infrastructure as code •  No Automated testing •  Very limited automation

   

Deployments would take weeks or months with a high level of developer input onto cloned servers

How did we solve it

6  

First Iteration – Nov 2013 •  Basic Combination approach of Boto and Python for AWS scripting •  Basic Opensource Puppet for configuration, server management

Third Iteration – Jan 2015 •  AWS Puppet module with cloud formation •  Advanced puppet coding standards with testing •  Container services and schedulers

Second Iteration – Feb 2014 •  Improvements to scripts •  Additional Puppet coding standards including upgrading to Enterprise •  Heading towards 100% Puppet coverage

Our Environment

7  

5 development environments 3 Puppet masters Dynamically controlling the VPC environments they are responsible for Each environment the accesslevel reducing towards Stage and Prod being automated environments

So it all looks easy, Right !!!!!

8  

What have we learnt ?

9  

A solid foundation -  Development … Where to start ??? -  Unit and acceptance testing frameworks -  CD with Puppet

Deploying to AWS with Puppet -  Tips and tricks

What we are working on now to take Puppet to the next level

Development, Where to start ?

10  

If you have never written a module start here !!!

11  

h#ps://forge.puppetlabs.com/  

Start with a solid development foundation !!!

12  

h#ps://github.com/mitchellh/vagrant  

A  must  have  for  any  Puppet  development    

• Quick  deployments  • A  massive  community    

•  Supports  any  OS  you  do    •  You  can  deploy  a  single  machine  or  100    

•  It  is  as  complex  or  simple  as  you  need  it  to  be    

•  Its  all  Ruby      

If you have not developed Vagrant framework , Here are is a good one

13  

h#ps://github.com/adrienthebo/oscar  

What we have done with Vagrant

14  

• Re  useable  VagrantFile  with  variables  •  Yaml  box  configuraMon    

• MulM-­‐node  test  environments  with  both  open  source  and  Puppet  Enterprise    

•  Full  Hiera  integraMon      

• We  have  a  full  replicaMon  of  producMon  on  our  laptops      

VagrantFile

15  

Yaml backend

16  

Unit testing is a must !!!

17  

• Picks  up  human  error  

•  Tests  module  dependancies  

•  EssenMal  for  the  CD/CI  process      

Tools for the job h#p://rspec-­‐puppet.com/  

h#ps://github.com/rodjek/rspec-­‐puppet  

What do tests look like ?

18  

A  simple  example  

A  complex  example  

execute  the  test  •  rake  spec_prep  &&  rspec  

But wait what don't test cover ?

19  

• Package  versions,  If  a  file  exists  etc  

So we need acceptance testing •  This  where  Vagrant  comes  back  in    

Run the module !!!! Puppetlabs  have  a  very  good  framework  for  this  h#ps://github.com/puppetlabs/beaker  

So now we have our working modules … How do we integrate them with CD ?

20  

•  r10k  h#ps://github.com/puppetlabs/r10k  is  a  good  place  to  start.  

This  will  give  you  a  great  transport  mechanism  from  source  control  to  the  Puppet  master.  

But what if we are using a CD tool (ie Jenkins etc) ?

Puppet code deployment flow

21  

So some CD tricks and tips we use

22  

Every  module  has  its  own  repo  

So some CD tricks and tips we use

23  

Each  pipeline  runs  different  test  before  it  can  move  to  the  next  

AWS Deployment Tools

24  

…  And  because  we  are  not  big  fans  of  JSON  h#ps://github.com/bazaarvoice/cloudformaMon-­‐ruby-­‐dsl  

Example of the AWS Puppet module

25  

Benefits of the AWS Puppet module

26  

• Anyone  on  the  team  can  develop  a  new  AWS  instance,  elb  etc  • Out  of  the  box  we  have  great  unit  tesMng  coverage  • On  every  Puppet  run  reverts  the  environment  to  the  desired  state  

Number  1  2p  for  AWS  

h#ps://docs.puppetlabs.com/guides/rest_api.html  

The  HTTP  API  

AWS  SQS  +  Puppetlabs  HTTP  API  =  Happy  engineers      

27  

What we are working on to take Puppet and AWS to the next level

28  

29  

require => Class['docker']

docker-compose.yml.erb

What we are working on now to take Puppet to the next level

30  

Docker  scheduling  with  Puppet  

Dynamic  Hiera  backend  and  service  discovery  

So that was easy right ????

31  

Questions http://www.healthdirect.gov.au