modern continuous delivery with docker and liferay
TRANSCRIPT
@mdelapenya #LRDEVCON
PRODUCTION SYSTEMSTHIS TALK IS NOT ABOUT…
http://cdn.wallpapersafari.com/41/27/KhsIdT.jpg
@mdelapenya #LRDEVCON
THIS TALK IS ABOUT…
DEVELOPMENT PROCESSEShttp://www.itechscope.gr/img/banner-image-process.jpg
@mdelapenya #LRDEVCON
CONTINUOUS DELIVERY @ LIFERAY
▸Continuous Delivery
▸Challenges
▸Approach to our Development Process
@mdelapenya #LRDEVCON
CONTINUOUS DELIVERY
http://www.jsg.utexas.edu/lacp/files/PGC_Pipeline_Data.jpg
@mdelapenya #LRDEVCON
CONTINUOUS DELIVERY▸Helps in building the product with the most possible
quality
▸Repeat hard tasks (release, deploy) more times
▸Find bugs the soonest, when it’s still cheap
▸Reduce feedback loop from customers
@mdelapenya #LRDEVCON
INTERNAL CUSTOMERS▸Engineering teams:
▸Developers, Support engineers, Testers, Consultants, Designers, Product Managers
▸Other nontechnical teams:
▸Sales, Marketing, …
@mdelapenya #LRDEVCON
CD FOR OUR TEAM MEANS…▸Very briefly…
▸Automate Build + Dependency Management
▸Automate Testing (Pull Request Tester)
▸Automate internal deployments
@mdelapenya #LRDEVCON
PULL-REQUEST TESTER▸JSP Compile, Semantic Versioning, Portlet
Spec TCK, Plugins compile…
▸JBoss 7, Tomcat 8, Weblogic 12, Wildfly 10…
▸Hypersonic, MySQL 5.6, MariaDB10, Oracle12, PostgreSQL 9.4, Sybase16, DB2 10.5…
@mdelapenya #LRDEVCON
CHALLENGES
http://www.greenbookblog.org/wp-content/uploads/2015/06/challenges.jpg
@mdelapenya #LRDEVCON
CHALLENGES▸Old-School IT procedures
▸Divergent environments
▸Bug reproducibility
@mdelapenya #LRDEVCON
OLD-SCHOOL IT PROCEDURES▸New environments request
▸Physic Servers && Virtual Machines
▸Virtualization (Proxmox)
▸Operate the environment
@mdelapenya #LRDEVCON
OLD-SCHOOL IT PROCEDURES▸Manual or semi-automated deployments:
▸Jenkins jobs
▸Customised Shell scripts
▸rsync
▸Specific application configuration
@mdelapenya #LRDEVCON
OLD-SCHOOL IT PROCEDURES▸Documentation
▸Wikis
▸ Intranets
▸Google Docs
▸SysAdmin’s head?
@mdelapenya #LRDEVCON
DIVERGENT ENVIRONMENTS▸Manual configuration produces divergency
▸DEV servers are not a copy of PROD servers
▸O.S., CPUs, RAM, disk, libraries, configuration
▸Same problem with DEV laptops
@mdelapenya #LRDEVCON
INCREASED COMPLEXITY
▸Different source code versions
▸Different O.S.
▸Different application servers
▸Different databases
@mdelapenya #LRDEVCON
DB2 10.1
MariaDB 10
MySQL 5.6
Oracle 12cR1
PostgreSQL 9.3
SQL Server 2012
Sybase ASE 16
JBoss EAP 6.4 X X X X X X X
Tomcat 8.0 X X X X X X X
Weblogic 12c X X X X X X X
Websphere 8.5 X X X X X X X
Wildfly 10.0 X X X X X X X
@mdelapenya #LRDEVCON
💔❤▸Not changing context,
stay in the GIT workspace
▸Using an already-built environment, many times
▸Using automated procedures
▸Changing context: stash or WIP commit
▸Building in a specific commit, many times
▸Using manual procedures (DRY)
DEVELOPERS: LOVE IT OR LIST IT
@mdelapenya #LRDEVCON
THE SYSTEM WE LOVE… ❤▸Modern and easy to use
▸Builds environments in an automated way, with everything (application plus dependencies)
▸Stores those environments for reusability
▸ Increases teams’ autonomy
▸ Increases collaboration with I.T.▸Does it sound as DEVOPS culture?
@mdelapenya #LRDEVCON
JENKINS▸Automation server
▸Jobs that trigger builds
▸Builds compound by steps
▸Plugins ecosystem
@mdelapenya #LRDEVCON
DOCKER CONTAINERS▸Docker Inc. is a company supporting
containers (Linux and Windows!)
▸Container = lighter virtualisation than VM
@mdelapenya #LRDEVCON
DOCKER CONTAINERS▸Standar De-Facto
▸Other container technologies: OpenVZ, LXC, Rocket, etc.
▸Image model based in Layers
▸CLI and REST API
▸BIG ecosystem around
@mdelapenya #LRDEVCON
ROLES▸Docker Registry
▸Docker Engine
▸Docker Builder
▸WOMC! (Works On My Computer)
@mdelapenya #LRDEVCON
ROLE: DOCKER PRIVATE REGISTRY▸Docker Hub: public IT infrastructure hub
▸Private Registry: on-premise
▸VM with massive storage for images
▸docker-compose
▸Containers: Apache + Registry
▸Security: OpenSSL
▸ https://docs.docker.com/registry/recipes/apache
@mdelapenya #LRDEVCON
ROLE: DOCKER ENGINE▸Docker daemon > O.S. package
▸Application infrastructure and runtime dependencies standardised
▸Same runtime environment from Docker Engine to Docker Engine
▸Docker Engine = Docker Host
@mdelapenya #LRDEVCON
ROLE: DOCKER ENGINE▸VM per team as team’s lab
▸VM provisioned Ansible role
▸Runs Liferay images from private registry
▸docker run or docker-compose
▸Liferay + Database
▸Containers and images clean-up
@mdelapenya #LRDEVCON
WHY VMS?▸They are truly helpful for us
▸Moving a whole docker host between machines
▸Backup of a whole docker host, including its containers
▸Security: isolated teams (potential shutdowns, resources-draining by other teams)
@mdelapenya #LRDEVCON
ROLE: DOCKER BUILDER▸Jenkins slaves
▸Fetches portal source code from master branch, reading last commit ——> HASH commit
▸Generates a Liferay Docker image every night
▸Uploads those images to the private docker registry
▸Deploys that MASTER to a controlled environment
@mdelapenya #LRDEVCON
NAME CONVENTION FOR IMAGES
PRIVATE_REGISTRY/GIT_REPO :
GIT_BRANCH-HASHTIMESTAMP
@mdelapenya #LRDEVCON
NAME CONVENTION FOR IMAGES (2)
LOCALHOST:5000/LIFERAY-PORTAL :
20161115123000MASTER-ABCDEF1234567890
@mdelapenya #LRDEVCON
NAME CONVENTION FOR IMAGES (3)
PRIVATE_REGISTRY/GIT_USER/GIT_REPO :
GIT_BRANCH-HASH
@mdelapenya #LRDEVCON
NAME CONVENTION FOR IMAGES (4)
LOCALHOST:5000/MDELAPENYA/LIFERAY-PORTAL :
FEATURE-1-ABCDEF1234567890
@mdelapenya #LRDEVCON
DOCKER ENGINE
CT.1 CT.2 CT.N
docker-engine-team3
DOCKER ENGINE
CT.1 CT.2 CT.N
docker-engine-team1 DOCKER ENGINE
CT.1 CT.2 CT.N
docker-engine-team2
CT.1 CT.2 CT.N
DOCKER ENGINE
PRIVATE REGISTRY
docker-engine-core
Virtual Machine
Container
Physical Machine
docker-builder-i
push
run
run
pull
run
@mdelapenya #LRDEVCON
CT.1 CT.2 CT.N
DOCKER ENGINE
PRIVATE REGISTRY
docker-engine-core
pull & push
DOCKER ENGINE
CT.1 CT.2 CT.N
docker-engine-team-i
run
run
run
DOCKER ENGINE
CT.1 CT.2 CT.N
local-machine-j
run
Virtual Machine
Container
Physical Machine
@mdelapenya #LRDEVCON
DIRECT BENEFITS▸Less dependency on IT
▸More autonomous teams
▸Both Developers and nontechnical Users can build environments more quickly (demos and showcases!!)
▸With training, Developers can reproduce new environments starting from already created ones
@mdelapenya #LRDEVCON
SUMMARISING…▸Define your CD pipeline: min(Build + Test + Deploy)
▸Build Docker image within the pipeline
▸Publish images for teams
▸More autonomous teams
▸Increase of development speed