solum - openstack paas / alm
TRANSCRIPT
Solum Applicat ion Li fecycle Management
for OpenStack
Devdat ta Kulkarn i , devdat ta .ku lkarn [email protected] ( i rc : devku lkarn i )
What is Solum?
OpenStack project that provides easy mechanism for application developers to deploy and run their applications on OpenStack starting from application’s source code Why should you care? (the value proposition)
– For operators, Solum provides ability to make your OpenStack cloud more useful for your clouds’ application developers by simplifying the process of deploying applications to it
– For developers, Solum provides an easy-to-use platform for building, testing, and deploying applications on OpenStack clouds
Outline
• Team introduction • Project timeline • Community involvement • Project details • Solum concepts and features • Rackspace experience • Upcoming features • Call to action • Question and answers
Team introduction
• Devdatta Kulkarni – PTL for Mitaka – First member on the Solum development team – With Rackspace for close to six years – PhD in CS from University of Minnesota Minneapolis
• Adrian Otto – PTL till Liberty
• Key team members: – Roshan Agrawal, Murali Allada, Ed Cranford, James Li, Melissa Kam, Dimitry
Ushakov, Angus Salkeld, Paul Czarkowski, Arati Mahimane, Paul Montgomery, Noorul Islam, Julien Vey, Pierre Padrixe, Georgy Okrokvertskhov, Ravi Sankar Penta, Nick Silkey, Randall Burt, Vijendar Komalla, Keith Bray, Gil Pilz
Timeline
October 2013: Project announced on OS-dev mailing list as a OpenStack-related project (i.e. stackforge)
December 2013: SFO mid-cycle meet-up at Rackspace office
March 2014: Raleigh mid-cycle meet-up at RedHat office
May 2014: Initial demo of Solum at Atlanta summit
June 2015: Solum accepted into OpenStack big tent
Community involvement
Companies that have contributed patches: Rackspace, Independent, Numergy, Redhat, Mirantis, HP, AT&T, Oracle, IBM, DreamHost
http://bit.ly/1PPhZ7I
http://bit.ly/1P3fBbW
Project details
• Wiki: https://wiki.openstack.org/wiki/Solum
• Meetings: Tuesdays 1700 UTC in #openstack-meeting-3
• IRC: Channel: #solum on freenode
• Code:
https://github.com/openstack/solum https://github.com/openstack/python-solumclient
Solum – What, why, how
What is Solum?
System which provides a declarative model for application developers to deploy and run their application’s on OpenStack starting from application’s source code Why should you care?
– For operators, Solum provides ability to make your OpenStack cloud more useful for your clouds’ application developers by simplifying the process of deploying applications to it
– For developers, Solum provides an easy-to-use platform for building, testing, and deploying applications on OpenStack clouds
Project goals
Application Stack Flexibility Ability to support applications written in different languages and using different application frameworks
Developer Productivity Ability to perform CI/CD, integration with github
Add-On Services Extensibility Ability to support different add-on services
Application Portability Ability to deploy applications across different OpenStack clouds (use native OpenStack services)
Solum abstractions
• App
• Languagepack (LP)
• Deployment Unit (DU)
• Workflow
• Add-on
Solum abstractions - App
Declarative application definition
• source code repository • run command • port • languagepack
version: 1 name: cherrypy description: python web app languagepack: python source: repository: https://github.com/rackspace-solum-samples/solum-python-sample-app.git revision: master workflow_config: run_cmd: python app.py ports: - 80
Solum abstractions - Languagepack (LP)
• Docker image which contains application specific build and runtime libraries
• A LP needs to implement Solum’s languagepack contract A well-known file available at a known location on the languagepack image
/solum/bin/build.sh
• How to build a LP? Dockerfile
• Operator-defined or user-defined
Solum abstractions - Languagepack Example : Py thon
FROM ubuntu:precise MAINTAINER Murali Allada <[email protected]> RUN apt-get -yqq update RUN apt-get -yqq install python-pip RUN apt-get -yqq install python-dev COPY build.sh /solum/bin/
https://github.com/rackspace-solum-samples/solum-languagepack-python
#!/bin/bash # Check if pip is installed pip help [[ $? != 0 ]] && echo python-pip is not installed. && exit 1 # Install app dependencies cd /app pip install -r requirements.txt
build.shDockerfile
Solum abstractions - Deployment Unit (DU) (1/2)
• Docker image that is formed from the languagepack image with application’s source code added to it
DU = LP + application source code
• Solum guarantees the DU contract Application source code available at a known location on the DU image /app
• How to build a DU? – Construct Dockerfile with languagepack as the base image, application
source code injected, run command as the entry point – Build the DU image from this Dockerfile
Solum abstractions - Deployment Unit (DU) (2/2)
• Building a DU – Start from the specified languagepack LP – Specify execution of languagepack’s ‘build.sh’ – Inject application source code – Use the run command specified in app definition as the default
Entrypoint to run the DU
• LP and DU storage Glance, Swift, Docker registry
16
Solum abstractions - Workf low
• Abstraction to represent execution of application deployment consisting of one or more workflow stages
• Supported workflow stages
– Build DU (and store it for future use) – Run unit tests, build DU – Run unit tests, build DU, deploy DU (if unit tests pass) – Build DU, deploy DU – Deploy a previously built DU (not yet available)
• A workflow can be triggered from github webhooks
Solum abstractions - Add-ons
• Services needed by an application E.g.: relational database such as Trove
• DU parameters Solum supports ability to pass service’s connection parameters to application DU
Architecture and system details
High-level Architecture
Solum API
Solum Worker
Solum Deployer Queue Queue
Queue
Solum Conductor
Heat
Solum Database
Glance, Swift, Docker Registry
Builds LP and DU Deploys DU
- Decoupled architecture
- Asynchronous communication
Demo
https://vimeo.com/143425822
Experience
Usage experience
• Languagepacks - Internal test users mostly tended to use operator defined languagepacks - Languagepacks for Wordpress and NodeJS+MongoDB application also
created by some
• Apps Need for ease of application registration led to supporting interactive prompts for getting app information
• Github integration – Started with support for public repositories – Internal use-cases required adding support for • Private repositories • CLI support for registering deploy keys • CLI support for two-factor authentication
Challenges in building LPs and DUs
How to provide isolated environment for building LP and DU Docker images and running untrusted unit tests on solum-worker?
• Timeout mechanism to constrain and limit the running time of unit testing scripts
• Isolated ‘git clone’ with resource constraints on CPU, memory, disk on
containers running unit testing scripts
• Running unit testing scripts as unprivileged user inside a container • Easy-to-use CLI for operator to kill long running (malicious) containers
Reliability improvements
• Success rate of building and saving DU and LP images improved from initial 80% to 98.8%
• DU and LP images saved in Glance and Swift using ‘docker save’
– Retry mechanism for performing git actions (git clone) and Docker actions (build, save, load)
– Better use of Swift client to upload/download
– Race condition handling in ‘docker load’ and ‘docker rmi’ – Perform ‘docker rmi’ only for DU images on the worker node and not for LP images
DU Deployment - Suppor ted options
• nova-docker driver Works with DU images stored in Glance
• Heat + CoreOS VM + DU location provided through user_data section – Works with DU images stored in Swift and Docker registry – Currently 1-1 mapping of DU to VM
Related Projects
• Deis • CloudFoundry • OpenShift
• Main value proposition of Solum is it is natively designed and built for OpenStack – Multi-tenancy through Keystone – Docker image storage through Glance and Swift (also Docker registry) – Deployment through Heat and Nova
Upcoming features
Key Features for Mitaka
• Application scaling DU scheduling is the key
Leverage Heat + Magnum
• Application monitoring DU monitoring is the key
Leverage Heat + Ceilometer
• Application Environments (dev/test/staging) Ability to deploy pre-built DUs
• Multi-tier applications
Conclusions
• Solum is ready for early adopters Appealing operators to consider deploying Solum in their OpenStack clouds and provide feedback
• Solum is looking for application developers to try it – Deploy applications starting from source code – Build custom languagepacks – Customize application workflows for continuous integration testing – Integrate with Github (public and private) – Easy-to-use CLI
• Exciting features have been planned for Mitaka Come and help build J
Contact Information
Devdatta Kulkarni [email protected] (irc: devkulkarni) Solum team members available on: #solum
O N E FA N AT I C A L P L A C E | S A N A N T O N I O , T X 7 8 2 1 8
U S S A L E S : 1 - 8 0 0 - 9 6 1 - 2 8 8 8 | U S S U P P O R T: 1 - 8 0 0 - 9 6 1 - 4 4 5 4 | W W W . R A C K S PA C E . C O M
© RAC K S PAC E LT D . | RAC K S PAC E® AN D FAN AT I C AL S U P P ORT ® ARE S ERV I C E M ARK S OF RAC K S PAC E U S , I N C . REG I ST ERED I N T HE U N I T ED STAT ES AN D OT HER C OU N T RI ES . | W W W.RAC KS PAC E .C OM
US
Thank you
FROM ubuntu:14.04 RUN apt-get -yqq update && \ apt-get -yqq install openjdk-7-jdk RUN apt-get -yqq install maven ENV JAVA_HOME /usr/lib/jvm/java-7-openjdk-amd64 COPY bin/build.sh /solum/bin
https://github.com/rackspace-solum-samples/solum-languagepack-java
mvn -v [[ $? != 0 ]] && echo "maven is not installed" && exit 1 # build cd /app mvn -q package
FROM build.sh
Solum abstractions - Languagepack Example : Java
https://github.com/rackspace-solum-samples/solum-languagepack-wordpress
Dockerfile -------------- Install PHP, required Apache modules
build.sh ------------ Empty
App run command -------------------------- Setting the environment variables and starting the app
Solum abstractions - Lang uag ep ack Examp le : Wordp res s
Basic operational flow
• API receives a request to build and deploy an app
• API sends the request to the Worker
• Worker downloads the specified LP from configured storage
• Worker builds the DU and stores it
• Worker informs the Deployer to deploy the DU
• Deployer deploys the DU by calling Heat