cloud meets fog & puppet a story of version controlled infrastructure
DESCRIPTION
Talk at rootconf - A conference at Bangalore for sysadmins. Gist of the talk:- Puppet is a great configuration management tool and git is great at version controlling.AWS lets you create instances in few clicks. But when it comes to large deployments only automation(where tools come together) can make you productive and happy. I will take you through following.. Fog - The Ruby cloud services library and how it helps you to create vendor neutral cloud deployments, Puppet- Multi region puppet masters, Ruby- How Ruby pulls the strings together in EC2/ELB/RDS creation, Security group creation, IP authorization, Route53 DNS etc, Git- how we use git to version control deployment configs/configurations.TRANSCRIPT
Cloud meets Fog & PuppetA Story of Version Controlled Infrastructure
Habeeb Rahman | apigee | @habi_pk | [email protected]
About:apigee
We love APIs Developers => API <= Enterprises
Usergrid
Take 1Third-party web
tool and ...
….Following Manual TasksShareImages CreateInstancesOnebyOne
InstallPackages InstallLicense
CreateEBSVolumes OpenPorts
AttachEIP GenerateServerConfigApply Patches GenerateNagiosConfig
+ a Few Misc
Take 2Cloud meets puppet and ...
(aws console, git, puppet)
….Following Manual TasksShareImages CreateInstancesOnebyOne
InstallPackages InstallLicense
CreateEBSVolumes OpenPorts
AttachEIP GenerateServerConfigApplyPatches GenerateNagiosConfig
+ a Few Misc
Take 3Cloud meets Fog and …
(puppet cloud provisioner, fog, ruby)
….Following Manual TasksShareImages CreateInstancesOnebyOne
InstallPackages InstallLicense
CreateEBSVolumes OpenPorts
AttachEIP GenerateServerConfigApplyPatches GenerateNagiosConfig
+ a Few Misc
Customer in a 'yaml' file...
Git version controls Puppet Configurations & Cloud Deployments
(Yes! Version Controlled Infrastructure)
Git hooks
Pre-update => puppet syntax checks(puppet parser, puppet lint)
Post-update => sync with regional puppet masters
Puppet
Git branches => Puppet environments
Single puppet master => Test Dev Production
manifests and files/data => Separate git repo
Puppet masters per AWS regions
Hiera: a pluggable hierarchical data store
Why Fog?
Cross service compatibility(multitude of services=> Storage, Compute, DNS…)
Avoid vendor lock-in(Vendor independent=>EC2, Rackspace, Slicehost, Linode…)
Power of Ruby
gem install fog
require ‘fog’
~/.fog
Fog quick starter kit
Look at the test suites (thanks to TDD) https://github.com/fog/fog/blob/master/tests/aws/requests/compute/instance_tests.rb
Fog command line
[root@zebra ~]$ less ~/.fog[root@zebra ~]$ fog testaws Welcome to fog interactive! :testaws provides AWS>>
ec2 = Fog::Compute.new( provider: "AWS", aws_access_key_id: "KEY", aws_secret_access_key: "SECRET")
rsc = Fog::Compute.new( provider: "Rackspace", rackspace_api_key: "KEY", rackspace_username: "SECRET")
ec2.servers.create( flavor_id: "c1.medium", image_id: "ami-6936fb00", # Ubuntu 10.04 LTS key_name: "mykeyname", groups: ["default"],)
rsc.servers.create( flavor_id: 5, # 4GB Memory image_id: 49, # Ubuntu 10.04 LTS name: 'my_server_name')
Open Source Fog alternatives
libcloud (Python)jclouds (Java)
deltacloud (Ruby)
Why Ruby?
“Ruby is designed for programmer productivity and fun”
Matsumoto(creator of the ruby language)
'everything-ruby' by Ajey Gorehttp://confreaks.com/videos/921-rubyconfindia2012-everything-ruby
irb is your friend (ruby interactive shell)
root@zebra:~/$ irb1.9.3-p125 :001 >
irb alternative = > pry (gem install pry)
The Layers of Abstraction...
Fog => Cloud abstraction
Puppet => OS abstraction
Notes to self >>
Start small...
...and iterate reiterate & re reiterate
Select right tools(Plenty of tools => Puppet/Chef/Cfengine/Ansible
but what is good for you/team/culture?)
Design your SaaS but make sure it's 'ops' ready
(from the day you start the design)
http://puppetlabs.com/http://fog.iohttp://git-scm.com/http://ansible.github.com/