packing it in: images, containers, and config management
DESCRIPTION
Learn about the many different tools gaining momentum to manage system configurations. Hear about when you should think about configuration management tools and when it might be OK to just pack it into a re-usable image. We’ll cover the basics around Docker and Packer before diving into a full-stack example leveraging all three topics in harmony together.TRANSCRIPT
Tuesday, July 15, 14
Packing It In: Images, Containers, and Config ManagementMichael GoetzSr. Consulting Engineer @ [email protected]
Tuesday, July 15, 14
Who am I?• Sr. Consulting Engineer @ Chef
• 8+ years of experience planning, managing and operating web scale and enterprise applications
• Avid woodworker
Tuesday, July 15, 14
This talk isn’t about joining a cult...• Lots of opinions exist that claim to be the “only right
way” to manage your systems
• The true path is the best combination that makes you go faster, in a safe and secure manner
• Use a toolbox, not one tool
http://leavingthecult.com/
Tuesday, July 15, 14
So what are my options?• Artisanal machines made of metal and sweat• Pristine virtual machines• Isolated containers• Just-in-time automatic configuration management• All (or some) of the above?
Tuesday, July 15, 14
Artisanal machines made of metal and sweat • Do we really need to talk about why this sucks?
• If you want to work on artisan crafts, take up woodworking
http://www.juggernautwoodworking.com/images/carve.jpgTuesday, July 15, 14
Containers vs. Virtual Machines
• Containers consist of an application and its dependencies, running in isolation in userland outside the kernel.
• Virtual Machines create an entire machine, including a fully functional operating system.
https://www.docker.io/static/img/about/docker_vm.jpg
Tuesday, July 15, 14
Hurray! We can go back to golden images, right?• The “golden image” problem still exists with containers, but on a much smaller
scale• A dozen “server” images become dozens of “container” images• AUFS layering mitigates some sprawl, but has a limit• Modularity of applications without convergence of the entire system just kicks the
can down the road
http://images.smh.com.au/2011/10/28/2737998/ipad-art-wide-shipping-420x0.jpgTuesday, July 15, 14
What about configuration management?• Convergence - coming to a desired end state• Congruence - building a result from a blank state
• Always building from scratch can be time consuming
• Specification of application versions becomes extremely important
• Changes can happen unexpectedly if you don’t plan ahead
Convergence is like fixing the outcome and compute the route (like a GPS finder), and congruence is about repeating a recipe in a sequence of known steps to massage a system into shape”
– Mark Burgess
Tuesday, July 15, 14
Tuesday, July 15, 14
Let’s talk real world here...• My application system has:
• An OS layer that rarely changes• A few supporting applications that change semi-
frequently• My application code that changes rapidly
• This can translate to:• VM image to act as a base OS + some deltas• Container images for supporting applications• Configuration management to maintain overall state
Tuesday, July 15, 14
So wait... that still seems like a lot of work• With 3 layers of your application stack to maintain, it feels like the maintenance
demand will only go up
• We’ll use three tools to manage each layer:• Packer - building and maintaining images (virtual machine host)• Chef - building Docker images, provisioning the VM and managing the
configuration of running containers• Docker - running the containers
Tuesday, July 15, 14
What is Packer?• Half the battle is keeping VM images up-
to-date
• The more time spent refreshing VM images, the more table flipping that will ensue
• Packer is tool for creating identical machine images for multiple platforms from a single source configuration
• Makes programmatically building VM images super easy!
{ "builders": [{ "type": "amazon-‐ebs", "region": "us-‐east-‐1", "source_ami": "ami-‐8ade42ba", "instance_type": "m3.medium", "ssh_username": "ubuntu", "ami_name": "my ami {{timestamp}}" }], "provisioners": [{ "type": "chef-‐solo", "cookbook_paths": ["cookbooks"], "json": { "name": "my_node", "run_list": [ "recipe[docker]", "recipe[my_application]" ] } }]}
Tuesday, July 15, 14
What is Docker?• Docker combines Linux containers (LXC) with AUFS to
create portable, lightweight application containers
• Docker containers are running instances of Docker images
• Docker images can be shared via a public or private registry
• Containers can be single application processes or lightweight virtual machines if a supervisor is provided.
Tuesday, July 15, 14
What is Chef?• Chef is an automation platform that manages
infrastructure as code
• Configuration of systems is performed by reusable recipes that are shared across your entire infrastructure
• Information about the various infrastructure components is cataloged and made available to to inform the rest of the topology configuration
• Chef can run on demand or as a managed service to keep infrastructure convergent
Tuesday, July 15, 14
Chef-Container• A version of chef-client that includes
components to support running the chef-client from within a Linux container• Packaged with chef-client, runit and
chef-init• Allows you to bootstrap the container
without an SSH connection• Use chef-client resources the same way
in a container as on any UNIX- or Linux-based platform
• Can manage multiple services within a single container using chef-init & runit
Tuesday, July 15, 14
The knife-container plugin• Used to initialize and build containers•knife container docker init•knife container docker build
• Docker support today, other containers planned
• Berkshelf integration
• Supports Chef-Zero or Chef-Client modes
Tuesday, July 15, 14
Let’s get to building!• Starting with a solid foundation is key to success
• Identify the core components that are unlikely to change, but are different from default settings• Security policies/applications• Image hardening• Core component packages• Docker tooling
• The goal is to create a minimal base VM, combined with the components that are consistently configured across your entire application infrastructure
Tuesday, July 15, 14
Demo: Building the VM
Tuesday, July 15, 14
Building the Docker factory• We need a repeatable factory for building Docker
images for the supporting applications
• Chef-container lets us use our existing Chef cookbooks to create reusable Docker images
• The key to success is isolation - create the smallest Docker images that will work
• Hook up your continuous integration system to crank out new images as cookbooks are updated
Tuesday, July 15, 14
Demo: Building the Docker Factory
Tuesday, July 15, 14
Bringing it all together• Now that we have our base VM and Docker factory
running, let’s manage an active application stack
• Chef will provision servers with the base VM, build and run the Docker containers
• Ongoing convergence of the overall desired state of the system will be managed by chef-clients running inside each container.
Tuesday, July 15, 14
Demo: Using Chef to manage the entire system
Tuesday, July 15, 14
Wrapping Up• Don’t join a cult
• Use what works to make things faster, more secure and more stable
• Keep the base VM small, but not too small
• Use containers to manage isolated, reusable applications
• Maintain a convergent infrastructure with automated configuration management
Tuesday, July 15, 14
Want to know more?• Release: Chef Container 0.2.0 (beta) - http://www.getchef.com/blog/2014/07/15/
release-chef-container-0-2-0-beta/
• Chef Containers Documentation - http://docs.opscode.com/containers.html
• Video demo - https://www.youtube.com/watch?v=nSB9rHG1_FQ&feature=youtu.be
• Packer - http://www.packer.io/
• Docker - http://www.docker.com/
Tuesday, July 15, 14
