docman : ultimate solution to manage site factories using drupal
DESCRIPTION
Introducing Docman (available on github, alpha state, but used already in production environment): the Swiss Army Knife for Drupal multisite docroot management and deployment. Docman acts as a layer between your docroot – usually a git repository somewhere, but not limited to it– and multiple vendors working on different websites using your standards and predefined sets of modules. Remember how hosting sales teams always tried to push you to buy more docroots "because otherwise it would be hard to manage websites in a Drupal multisite environment"? Docman can simplify your life so you will be able to stick to one docroot and multiple independent websites in it using the same Drupal core. Have you ever tried to oblige different vendors to work with one Drupal core for completely different websites, inside a multisite environment, without making them break everything and with clear deployment schema? Find out how docman can make the once daunting task of multisite deployment more efficient. Does the governance of multiple projects in one multisite environment scare you? Docman has hooks, like Drupal, to launch your tests whenever needed and for each website in multisite environment independently. During this presentation you will peek inside the inner workings of the docman tool and see: All the advantages it will bring to your multisite /single doc root deployments A real-world case study of setting up a stable platform using this concept and the tools needed A development process using docman ('docman build local development' command and others) Real-world example: You have a docroot at Acquia Cloud You have a small governance team setting up standards (Drupal core, list of contrib modules) You have 3 vendors working on three different websites in multisite environmentTRANSCRIPT
http://corp.adyax.com/themes/adyax/logo.png
D E V O P S H T T P : / / B I T . L Y / D O C M A N 2 0 1 4
DOCMAN
A L E K S E Y T K A C H E N K O A N D A L E K S A N D R T O L S T I K O V
The swiss army knife for Drupal multisite docroot management and deployment
http://corp.adyax.com/themes/adyax/logo.png
WHO AM IOleksiy Tkachenko
With Drupal from 2007With Adyax from 2009
PM, Drupal architect
France / Ukraine
LARGEST DRUPAL SHOP IN EUROPE
350 PROJECTS 150 DRUPALERS
7 YEARS OF GROWTH 10K HOURS PROJECTS
http://corp.adyax.com/themes/adyax/logo.pngBEFORE WE START
/ VAR / WWW
Acquia: your docroot folder is the root-level of your Drupal installation and its websites and contains its files and directories,
including Drupal's index.php, includes directory, and modules directory.
DOCROOT?
PROBLEM 1:Multiple suppliers need to work with one multisite installation
PROBLEM 1:
Deployment is painful - fatal errors, incompatibility
Production environment - never stable
How to merge and integrate the work of multiple people?
How to merge and integrate the work of multiple teams?
PROBLEM 2:
Standard modules, best practices and team discipline
PROBLEM 2:
No development standards. Panels? Contexts?
Standards exists, but not respected
Drupal best-practices are not respected
Modules in /all or in /default?
No team discipline (features are overridden, environments omitted, etc)
PROBLEM 3:Governance
PROBLEM 3:
Who should update Drupal?
What modules needs to be updated?
What is happening in my docroot?
What are the latest changes and stable versions?
PROBLEM 4:How and when?
PROBLEM 4:
When to run tests?
How to run tests?
In which environment?
PROBLEM 5:
Cloud hosting usually is very limited in multisite management
INTRODUCING DOCMAN
# gem install docman
http://adyax.github.io/docman/
SCHEMA
GIT: PROJECT A
GIT: CORE GIT:
SITESGIT:
CLOUD HOOKS
DOCMAN
DOCROOT
GIT: PROJECT B
GIT: COMMON
GIT: PROFILES
!!!!
REPOSITORIES: CONFIG
/master config.yaml
|-‐-‐ config.yaml (environments config)|-‐-‐ master |-‐-‐ info.yaml (Main docroot build config) |-‐-‐ docroot | |-‐-‐ info.yaml (Main docroot build config) | |-‐-‐ after_build.sh (executed after build) | |-‐-‐ profiles | |-‐-‐ info.yaml (Projects dir build config) | |-‐-‐ project_profile (if exists) | |-‐-‐ info.yaml (Profile build config) | |-‐-‐ after_build.sh (executed after build) | |-‐-‐ projects | |-‐-‐ info.yaml (Projects dir build config) | |-‐-‐ project_code | |-‐-‐ info.yaml (Code build config) | |-‐-‐ after_build.sh (executed after build) | |-‐-‐ sites | |-‐-‐ info.yaml (Docroot sites dir build config) | |-‐-‐ after_build.sh (executed after build) | |-‐-‐ common | |-‐-‐ info.yaml (SG core common files dir build config) | |-‐-‐ after_build.sh (executed after build)
https://github.com/Adyax/docman-configTemplate:
config.yaml
REPOSITORIES: CONFIG--- environments: dev: deploy_target: git_target state: development target_checker: handler : :ssh file_path: /mnt/www/html/subscription_name_dev # Edit this! ssh_host: staging-xxxx.prod.hosting.acquia.com # Edit this! ssh_user : subscription_name # Edit this! test: deploy_target: git_target state: staging target_checker: handler : :ssh file_path: none file_path: /mnt/www/html/subscription_name_dev # Edit this! ssh_host: staging-xxxx.prod.hosting.acquia.com # Edit this! ssh_user : subscription_name # Edit this! prod: deploy_target: git_target state: stable tagger : enabled: true handler : :option
master/common/info.yaml
REPOSITORIES: CONFIGstatus: enabled type: repo repo: git@this-is-your-git-host:common/common.git # Edit this. This is your repository with common modules for everyone. order: 30 states: # Git flow! development: type: branch version: develop staging: type: branch version: master stable: source: type: :retrieve_from_repo repo: :project_repo branch: state_stable file: info.yaml hooks: builder : after_execute: - type: :script location: $INFO$/after_build.sh execution_dir : $PROJECT$ params: - environment
Common code repo described
master/docroot/info.yaml
REPOSITORIES: CONFIG
type: repo repo: git@this-is-your-git-host:common/drupal-core.git # This is your Drupal repo. order: 1 states: development: type: branch version: master staging: type: branch version: master stable: type: branch version: master hooks: builder : after_execute: - type: :script location: $INFO$/after_build.sh execution_dir : $PROJECT$ params: - environment Drupal core repo
described
master/projects/project1/info.yaml
REPOSITORIES: CONFIGtype: repo repo: git@this-is-your-git-host:project1.git # Edit this. This is your project repository with custom code. states: development: type: branch version: develop staging: type: branch version: master stable: source: type: :retrieve_from_repo repo: :project_repo branch: state_stable file: info.yaml location: $PROJECT$/tools/deploy/$ENVIRONMENT$/after/after.sh execution_dir : $ROOT$/docroot params: - environment - type: :script location: $PROJECT$/tools/deploy/common/after/after.sh execution_dir : $ROOT$/docroot params: - environment
Project code repo described (not fully)
Drupal 7 !!!
REPOSITORIES: CORE
Branches: develop master
state_stable
REPOSITORIES: SITES
SITES
/all /default /site_a /site_b sites.php
Branches: develop master
state_stable
https://github.com/Adyax/docman-sites
/modules /themes /libraries
REPOSITORIES: PROJECT
PROJECT A
Branches: develop master
state_stable
https://github.com/Adyax/docman-project
!!!!
REPOSITORIES: CLOUD HOOKS
/common /dev /prod /samples /scripts /test README.md
FEATURES
Focused on cloud hosting: Acquia Cloud, Pantheon and others, but not limited to.
Clear separation of the websites code in multisite environments
Incremental pushes of the finished docroot to the cloud hosting
Code separation for easy deployment
FEATURES
Drupal version agnostic Drupal 8 ready!
Force people to keep features by default, updb, features revert, registry rebuild on each push to environments
IF SOMETHING HURTS, DO IT MORE OFTEN.
http://evan.bottch.com/2010/05/26/continuous-integration-if-something-hurts-do-it-more-often
FEATURES
Multiple docroots to support? Not a problem!
Stable & versioned production environment
Jenkins friendly workflow
Deployment pipelines organisation (with Jenkins integration)http://www.infoq.com/minibooks/continuous-delivery-overview
Different deployment scenarios achievable through config: Continuous integration
Continuous delivery Continuous deployment
LOCAL ENVIRONMENT
# docman build local development
DEV + STAGE ENVIRONMENT
# docman build git_target staging
# docman build git_target development
LIVE ENVIRONMENT
# docman build git_target stable
AVAILABLE COMMANDS
$ docman init <dir> <config-repo> !$ docman build <target> <env> !$ docman bump stable
!
AVAILABLE HELPERS
CHANGELOG file in each repo automatically generated using commit comments !VERSION file in each repo automatically generated with the latest version (all branches) !info.yaml files everywhere helps you with what is what
CASE STUDY 1: BEFORE
INITIAL CODEBASE
CLONE
SITE 2 CODEBASE SITE 3 CODEBASE
CASE STUDY 1: PROBLEMS
Conflict with Features (modules) on the cloned website
Manual deployment for each website
Pain to maintain
Standards? Which one?
Code duplication
CASE STUDY 1: AFTER
GIT: SITE A
GIT: SITE B
!GIT:
COREСOMMON
DOCROOT docman
gitGIT: SITES
CASE STUDY 2: SERIOUS
Drupal is a company level standard for websites (yay!)
Global company
At least 3 different Drupal shops independently delivering websites constantly
CASE STUDY 2: PROBLEMS
At least 3 different Drupal shops independently delivering websites.
Maintenance? Each agency defines.
Standards? Each agency defines. (Panels? Context?)
Deployment? Approach is different per agency (capistrano, manual, ftp, you name it).
Hosting is in the same place but each time configured differently
CASE STUDY 2: SOLUTION
GIT: SITE A
GIT: SITE B
!GIT:
CORECOMMON GIT:
SITESGIT:
CLOUD HOOKS
INTEGRATION PLATFORM
DOCROOT Acquia Cloud
git hooks > Jenkins > docman
Gitlab
http://corp.adyax.com/themes/adyax/logo.pngDEMO TIME
ROADMAP
@Todo: Vagrant image generation per website for easy local development
@Todo: wizard to generate repository with configs
@Todo: Documentation
NEED HELP
@Todo: Config templates for various cloud hosting systems
@Todo: More deployment targets
@Todo: More docroot templates (Pantheon, Aberdeen Cloud, etc)
@Todo: Better config error handling
http://corp.adyax.com/themes/adyax/logo.pngWHAT D ID YOU TH INK?E V A L U A T E T H I S S E S S I O N - http://bit.ly/docman2014
FOLLOW US @ADYAXFOLLOW ME @SHUMUSHIN