a metadata ocean in chef and puppet
DESCRIPTION
What is Metadata and how it is represented in Chef and Puppet? What are the challenges for metadata sharing across providers and possible solutions.TRANSCRIPT
![Page 1: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/1.jpg)
A metadata ocean in Puppet and Chef FOSDEM’14
Marc Cluet DevOps & Automation Advisory Rackspace
http://www.flickr.com/photos/elias_daniel/7470265444/
![Page 2: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/2.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Who am I?!Marc Cluet (@lynxman)
More than 16 years of experience as a SysAdmin Founding member of Juju and MAAS while at Canonical Leading a DevOps Engineering Team at Rackspace Likes DevOps, programming and long walks on the beach
![Page 3: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/3.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
What is Rackspace?!Founded in 1998 in San Antonio, TX by three guys that wanted to create a hosting company Home of Fanatical Support /o/ Second biggest Public Cloud in the world OpenStack Project co-founder
![Page 4: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/4.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
What is Metadata?!
/http://www.flickr.com/photos/boojee/3743753784/
![Page 5: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/5.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
What is Metadata?!
http://www.flickr.com/photos/ggvaidya/8256283184/
Metadata is “data about data”!Structural Metadata!Descriptive Metadata!
![Page 6: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/6.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Chef!Metadata is compiled on the client at execution time!Metadata is always stored in its entirety in the node[] dictionary!$node[var] = val
![Page 7: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/7.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Chef!You can add metadata in your!
Attribute File!Node/Recipe!Environment!Role!
![Page 8: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/8.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Organisation in Chef!
http://docs.opscode.com/essentials_environments.html
![Page 9: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/9.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Puppet!Metadata is always compiled on execution timeMetadata vars can’t be reassigned!$::var = val
![Page 10: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/10.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata in Puppet!You can add metadata in your!!Class vars!!Nodes!!Top Scope!
![Page 11: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/11.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Scoping in Puppet!
http://docs.puppetlabs.com/puppet/latest/reference/lang_scope.html
![Page 12: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/12.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hiera in Puppet!Solves metadata fragmentation issues!Defines hierarchy for easy organisation!All in clean and properly styled in either YAML or JSON!
![Page 13: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/13.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hiera in Puppet!:backends: - yaml - json:yaml: :datadir: /etc/puppet/hieradata:json: :datadir: /etc/puppet/hieradata:hierarchy: - "%{::clientcert}" - "%{::custom_location}" - common
![Page 14: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/14.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/musebrarian/3289649684/
![Page 15: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/15.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Bad metadata practices!In Chef!
Not keeping consistency between cookbook scope and environment scope!Not removing metadata from nodes when deprecated!Directly assigning metadata in a cookbook without a default attribute!
![Page 16: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/16.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Bad metadata practices!In Puppet!
Directly assigning metadata in a class without a default value!Assigning vars in nodes.pp!Not assigning global vars in site.pp!
![Page 17: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/17.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/deepstereo/9616389940/
![Page 18: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/18.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Good behaviours!Separate vars from execution!
Do not store vars on the execution area (classes/cookbooks)!
High abstraction, good hierarchy!Metadata is all about context!
!
![Page 19: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/19.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Good behaviours!
Global
Environment
Node
![Page 20: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/20.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK http://www.flickr.com/photos/monica_andre/3119991125/
![Page 21: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/21.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata Across Providers!
Provisioner cloud-init
Config Mgmt Puppet
$dog = cute $::dog = big
WHO IS RIGHT?!?!
![Page 22: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/22.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Metadata Across Providers!
http://www.flickr.com/photos/pss/233841567/
![Page 23: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/23.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hard Problem to Solve!
Metadata brain split problem!Gets worse by adding more players!
System view varies!Not same info from bios than from a user level program as facter or ohai!
!
![Page 24: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/24.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Possible Solutions!Central DB using something fast!!Redis, mongoDB, mySQL!
Metadata translators might fix it?!!Write for everything!
Creates new SPoF!!Hard to solve without throwing $$!
![Page 25: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/25.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Hard Problem to Solve!
http://xkcd.com/927/
![Page 26: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/26.jpg)
RACKSPACE® HOSTING | WWW.RACKSPACE.CO.UK
Possible Solutions!
Scope vars from top of chain!Pass them over as ext facts!$dog in cloud-init becomes $::dog in puppet!
![Page 27: A Metadata Ocean in Chef and Puppet](https://reader034.vdocuments.site/reader034/viewer/2022052618/554bc044b4c90530298b5021/html5/thumbnails/27.jpg)
27 RACKSPACE® HOSTING | © RACKSPACE US, INC. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.CO.UK
RACKSPACE® HOSTING | 5 MILLINGTON ROAD | HAYES, UNITED KINGDOM UB3 4AZ UK SALES: +44 (0)20 8712 6507 | UK SUPPORT: 0800 988 0300 | WWW.RACKSPACE.CO.UK
@lynxman!
http://slideshare.net/lynxmanuk/!
http://www.rackspace.co.uk/technicalservices/ !
!