© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Extending Cloud FoundryWith Custom Integration
Cornelia DavisScott Frederick
Who and What...
2
Extending the set of services
Customizing application containers
© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Services
Demo
4
Push app with trace
When you cf push
5R
oute
r
Cloud Controller
Service Broker
(i.e. mysql)
VM for Service Instances
DEA
CLI
MySQL(process)
provision (HTTP) provision (HTTP)up to you
bind (HTTP) bind (HTTP)up to you
...go ahead and deploy (HTTP)
credentials
DropletcredentialsBrowse Note:
• Don’t call ‘em “Gateways” anymore• Broker only used during
•(un)provision•(un)bind
Demo
6
• Go to app code to look at how credentials are accessed
Agenda
§Extending services offerings within run.pivotal.io–Partnering
§Building services–Into your own Cloud Foundry
§User-provided service instances–For access to pre&externally-provisioned services
run.pivotal.io Marketplace Services
8
Contact Nima Badiey ([email protected])
to play
run.pivotal.io
run.pivotal.io Marketplace Services
9
Cloud Controller
AppDirectService Broker
DEA
Dropletcredentials
Browser
credentials
Demo
10
• Create service instance from console
• push an app and bind to that service
Cloud Foundry Services• There is only one built in - mysql
– or many more depending on how you count• How to build them - the evolution:
– CF v1, subclass Ruby base classes– CF v2, a RESTful protocol (which is itself evolving)
Polyglot!
• There are three parts to the protocol...
11
1Broker Registers Offerings
12
Rou
ter
Cloud Controller
VM for Service Instances
DEA
CLI
My Service(process)
Authenticate
Services Catalog
Bro
wse
r
UA
A
token
Get catalog (HTTP)
Update catalog (HTTP)
On Broker Startup...
Service Broker
(i.e. mysql)
2Broker Services (un)provision and (un)bind
13
Rou
ter
Cloud Controller
Service Broker
(i.e. mysql)VM for Service
Instances
CLI
My Service(process)U
AA
provide shared secret (HTTP)
Prerequisite...
shared secret
2Broker Services (un)provision and (un)bind
14
Rou
ter
Cloud Controller
Service Broker
(i.e. mysql)VM for Service
Instances
DEA
CLI
My Service(process)
up to you
Dropletcredentials
Bro
wse
r
UA
A (un)provision (HTTP)
(un)bind (HTTP)
credentials
3Orphan Management
15
Rou
ter
Cloud Controller
Service Broker
(i.e. mysql)VM for Service
Instances
CLI
My Service(process)
Authenticate
up to you
Services Catalog
UA
A
token
Get catalog (HTTP)
Broker is responsible for eventual consistency...
New (V2) Services API• Simplifies things a bunch!• Unidirectional - everything initiated from Cloud Controller• Asynchronous provisioning• From V1 RESTful Services API:
– Replace offerings registration with endpoint that allows the CC to GET the offerings
– Remove orphan management
16
It’s Up To YOU Patterns
17
VM for Service Broker
Service Broker
VM for Service Instances
My Service(process)
Node
VM for Service Instances
My Service(process)
Node
VM for Service Instances
My Service(process)
Node
NATS
provision
One option (CF v1 services)...
It’s Up To YOU Patterns
18
VM for Service Broker
Service Broker
VM for Service Instances
Node
VM for Service Instances
Node
VM for Service Instances
Node
NATS
provision
Another...
WardenSvc Process
WardenSvc Process
WardenSvc Process
WardenSvc Process
VM for Service Instances
VM for Service Instances
VM for Service Instances
It’s Up To YOU Patterns
19
VM for Service Broker
Service Broker
NATS
provision
And another...
NodeNode
Warden
Warden WardenSvc Process Svc Process
Svc Process
Node
BOSH deployment
packages
- service broker - your impl
- service node - services binaries - services warden - your node impl
20
jobs
- service broker - ref service broker pkg - start web service
- service node - ref service node pkg - start node
Provisions broker VMs
Provisions service VMs
Configure shared secret (for example)
But what about...
21
AppDirect Service Broker
Service Broker
Service Broker
MySQL MySvc
core
ERP Oracle
?
User-provided Service Instances
22
Rou
ter
Cloud Controller
CLI
UA
A
provide service credentials (HTTP)
Prerequisite...
ERP
OracleDEA
Dropletcredentials
Bro
wse
r
credentials(un)bind (HTTP)
Demo
23
register user provided service instance
Tell ‘em what you told ‘em...
§Extending services offerings within run.pivotal.io–Partnering
§Building services–Into your own Cloud Foundry
§User-provided service instances–For access to pre&externally-provisioned services
© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Buildpacks
Deploying to Cloud Foundry
26
Staging and Buildpacks
Buildpacks are responsible for preparing the machine image for an application
27
Application
Container
Runtime
Operating System
Libraries
DEA
Buildpack } Droplet
Libraries = runtime package managers like Ruby gems, Node modules, Python PIP packagesApp bits are downloaded into image by DEA, buildpack may move them as necessary
CompatibilityCloud Foundry buildpacks follow the Heroku buildpack design
Cloud Foundry and Heroku buildpacks are compatible (if you take care to make them compatible)
Other PaaS providers are adopting the buildpack design
28
You can choose to implement platform-specific features in buildpacks and make them incompatibleFor compiled languages (Java) CF expects source to be compiled to machine code before being pushed. Heroku buildpacks will compile. Buildpack is an emerging convention for PaaS
Built-in or BYO
cf push
The application is tested against a set of curated
buildpacks
29
cf push --buildpack <url>
The buildpack is referenced by a Git URL
Git != GitHub - use any public Git provider or local Git repositoriesGit repository must be visible on the network that CF is running on - public CF requires public Git repo, private CF can use private or public Git repo
Built-in Buildpacks
30
→→→
Built into the CF open-source codebase and into Pivotal Hosted CF. Builtin means they are symlinked into the DEA in the buildpacks directory. Symlinking other buildpacks makes them builtin also. NodeJS and Ruby buildpacks are forks of Heroku buildpacks. Java buildpack is written from scratch for CF.
Tested Buildpackshttps://github.com/cloudfoundry-community/cf-docs-contrib/wiki/Buildpacks
31
LanguagesContainers
Haskell
Not tested or endorsed by Pivotal, all community-contributed and maintained.
Existing buildpacks are a great place to start when creating a new buildpack from scratch.
/bin/detect app_directoryInspect app bits to determine buildpack applicability
/bin/compile app_directory cache_directory Download and install runtime, container, packages, libraries; install app bits as necessary
/bin/release app_directoryBuild app start command
32
Buildpack API
Written in bash shell scripts or Ruby
/bin/detect
Inspect the app bits to determine if the buildpack knows how to handle the application
33
Gemfile exists
package.json exists
setup.py exists
On match, return exit code 0 and write to STDOUT a string identifying the buildpack
(often just the name of the language supported)
Java intentionally left off this list because the CF Java buildpack has more complex detection criteria
/bin/detect
$ cf push
DEA iterates over built-in buildpacks calling
/bin/detect scripts until one of them returns
exit code 0
34
$ cf push --buildpack <url>
/bin/detect is not called
/bin/compile
Download and install any necessaryruntime (Java VM, Ruby interpreter, JavaScript interpreter)container (web server)support libraries, packages, modules (Ruby gems, NPM packages)
...and then installing the app bits into the runtime or container
35
This is where most of the buildpack work is done.More package manager examples - Python PIP, Perl CPAN
Installing the app may mean copying or moving the app files from the location DEA put them in, or symlinking them to a different location
/bin/compile CachingRuntime, container, and support packages are downloaded from sources external to Cloud Foundry
DEA provides a location for storing downloaded artifacts to speed subsequent staging operations
36
External source can include custom packages in S3 buckets or public mirrors for public CF, or internal download locations for private CF.
/bin/release
Build a YAML-formatted hash with three possible keys
37
addons: []config_vars: {}default_process_types: web: <start command>
On Cloud Foundry, currently only the web: value is used to get the start command for the app
If creating a buildpack for compatibility, consult documentation for other platforms to see how addons and config_vars might be used.
Java BuildpackSupports a variety of JVM languages, containers, and frameworks with a modular, configurable, and extensible design
38
Java Buildpack Concepts
39
Containers
How an application is run
Frameworks
Additional application transformations
JREsJava Runtimes
Frameworks - loose definition, basically anything that is orthogonal to the container
Java Buildpack Concepts
40
Containers Frameworks
JREs
OpenJDK
Java main()TomcatGroovy
Spring Boot CLIPlay
Spring configPlay config
Play JPA configNew Relic
Zero Effort Spring - Tue 12:45-2:15 - Dave Syer and Phil Webb
Container Detection Criteria
41
Java main() META-INF/MANIFEST.MF exists with Main-class attribute set
Tomcat WEB-INF directory exists
Groovy.groovy file with a main() method, or.groovy file with no classes, or.groovy file with a shebang (#!) declaration
Spring Boot CLIone or more POGO .groovy files with no main() method, andno WEB-INF directory
Play start and lib/play.play_*.jar exist
Choose zero or one
Detection is typically based on file existence and content of files.
Zero Effort Spring - Tue 12:45-2:15 - Dave Syer and Phil Webb
This list is likely to grow to support other containers like Spring XD.
Framework Detection Criteria
42
Spring spring-core*.jar exists
Play config Play application detected
Play JPA config play-java-jpa plugin exists in app
New Relic New Relic service bound to app
Choose all that apply
Detection is typically based on file existence and content of files. Can also be done based on services bound to an app.
/bin/compile Output Example-----> Downloaded app package (18M)
-----> Downloading OpenJDK 1.7.0_21 JRE (17.5s) Expanding JRE to .java (1.4s)
-----> Downloading Auto Reconfiguration 0.7.1 (1.4s) Modifying /WEB-INF/web.xml for Auto Reconfig-----> Downloading Tomcat 7.0.42 (3.5s) Expanding Tomcat to .tomcat (0.2s)
Downloading Buildpack Tomcat Support 1.1.1 (0.0s)-----> Uploading droplet (55M)
43
DEA
Buildpack
DEA
See What’s Going On $ cf files <app-name> app
.buildpack-diagnostics/
.java/
.lib/
.tomcat/
META-INF/
WEB-INF/
assets/
44
Buildpack-installed runtime Buildpack-installed support libraries Buildpack-installed container
DEA-downloaded application files
The buildpack-installed locations are decisions made by the Java buildpack - other buildpacks will put things in other places.
See What’s Going On $ cf files <app-name> staging_info.yml
detected_buildpack: openjdk-1.7.0_21, tomcat-7.0.42,
spring-auto-reconfiguration-0.7.1
start_command: JAVA_HOME=.java
JAVA_OPTS="-Dhttp.port=$PORT
-Djava.io.tmpdir=$TMPDIR -XX:MaxPermSize=52428K
-XX:OnOutOfMemoryError=./.buildpack-diagnostics/killjava
-Xmx384M -Xss1M" .tomcat/bin/catalina.sh run
45
CustomizationTwo ways to customize the Java buildpack
Configure artifacts used by standard JREs, Containers, and Frameworks
Extend the buildpack with your own JREs, Containers, and Frameworks
Customization is done by forking the buildpack
46
Awesome documentation is in the github repo
Customization by ConfigurationConfiguration files in java-buildpack/config determine the behavior of a JRE, Container, or Framework
47
# cloudfoundry/java-buildpack/config/openjdk.yml---version: 1.7.0_+
repository_root: "http://download.pivotal.io.s3.amazonaws.com/openjdk/{platform}/{architecture}"
memory_sizes:
memory_heuristics:
heap: 0.75
permgen: 0.1
stack: 0.05
native: 0.1
# http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/index.yml---1.6.0_27: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.6.0_27.tar.gz1.7.0_21: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz1.7.0_25: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_25.tar.gz1.8.0_M6: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M6.tar.gz1.8.0_M7: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M7.tar.gz
version and repository_root are common to all components that download artifactsother options are specific to the component
1. consult <component-name>.yml2. read repository_root/index.yml3. look up the specified version in the index4. download the matching artifact
Customization by ConfigurationExample: customizing the Tomcat artifact for download
48
# cloudfoundry/java-buildpack/config/tomcat.yml---version: 7.0.42repository_root: "http://files.example.com/tomcat-custom"support: version: 1.1.+ repository_root: "http://files.example.com/tomcat-buildpack-support"
# http://files.example.com/tomcat-custom/index.yml---7.0.40: http://files.example.com/tomcat-custom/tomcat-7.0.40.tar.gz7.0.41: http://files.example.com/tomcat-custom/tomcat-7.0.41.tar.gz7.0.42: http://files.example.com/tomcat-custom/tomcat-7.0.42.tar.gz
Customization by ConfigurationTomcat container supports simple customization of context.xml and server.xml.
49
cloudfoundry/java-buildpack/resources/tomcat/confcontext.xmlserver.xml
fork, modify, push
Customization by ExtensionImplement a JRE, Container, or Framework support class as one Ruby file in the appropriate directory
(with additional support classes as necessary)
50
cloudfoundry/java-buildpack/lib/java_buildpack
jrecontainerframework
One class is sufficient for most extensionsExisting support classes serve as good examples
Customization by ExtensionSupport class types have similar interfaces, following the buildpack scripts naming conventions
51
# initialize the support class with platform information provided in context# context includes app_dir, lib_dir, environment, java_home, java_opts, vcap_application, vcap_servicesdef initialize(context)
# return a String or an Array<String> that uniquely identifies the container/framework/jre, or nildef detect
# download and unpack the container/framework/jre, and transform the application as necessarydef compile
# create and return the command to run the application with (containers) or add options to context[:java_opts] (frameworks)def release
if detect returns nil, compile and release will not be calledonly one container support class can return non-nil from detect
Customization by ExtensionAdd new support class to config/components.yml
52
---containers: - "JavaBuildpack::Container::Groovy" - "JavaBuildpack::Container::Main" - "JavaBuildpack::Container::SpringBootCli" - "JavaBuildpack::Container::Tomcat" - "JavaBuildpack::Container::Play"jres: - "JavaBuildpack::Jre::OpenJdk"frameworks: - "JavaBuildpack::Framework::JavaOpts" - "JavaBuildpack::Framework::NewRelic" - "JavaBuildpack::Framework::PlayAutoReconfiguration" - "JavaBuildpack::Framework::PlayJpaPlugin" - "JavaBuildpack::Framework::SpringAutoReconfiguration"
The buildpack will iterate over all components, applying them as necessary
CustomizationMuch more information and documentation included in the GitHub repository
https://github.com/cloudfoundry/java-buildpack
53
The buildpack will iterate over all components, applying them as necessary
© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.
Cornelia [email protected]@cdavisafc
Scott [email protected]
@scottyfred
Community Engineer, Cloud Foundry at Pivotal