extending cloud foundry with custom integration

© 2013 SpringOne 2GX. All rights reserved. Do not distribute without permission.

Extending Cloud FoundryWith Custom Integration

Cornelia DavisScott Frederick

Who and What...

2

Extending the set of services

Customizing application containers


Services

Demo

4

Push app with trace

When you cf push

5R

oute

r

Cloud Controller

Service Broker

(i.e. mysql)

VM for Service Instances

DEA

CLI

MySQL(process)

provision (HTTP) provision (HTTP)up to you

bind (HTTP) bind (HTTP)up to you

...go ahead and deploy (HTTP)

credentials

DropletcredentialsBrowse Note:

• Don’t call ‘em “Gateways” anymore• Broker only used during

•(un)provision•(un)bind

Demo

6

• Go to app code to look at how credentials are accessed

Agenda

§Extending services offerings within run.pivotal.io–Partnering

§Building services–Into your own Cloud Foundry

§User-provided service instances–For access to pre&externally-provisioned services

run.pivotal.io Marketplace Services

8

Contact Nima Badiey ([email protected])

to play

run.pivotal.io

run.pivotal.io Marketplace Services

9

Cloud Controller

AppDirectService Broker

DEA

Dropletcredentials

Browser

credentials

Demo

10

• Create service instance from console

• push an app and bind to that service

Cloud Foundry Services• There is only one built in - mysql

– or many more depending on how you count• How to build them - the evolution:

– CF v1, subclass Ruby base classes– CF v2, a RESTful protocol (which is itself evolving)

Polyglot!

• There are three parts to the protocol...

11

1Broker Registers Offerings

12

Rou

ter

Cloud Controller


DEA

CLI

My Service(process)

Authenticate

Services Catalog

Bro

wse

r

UA

A

token

Get catalog (HTTP)

Update catalog (HTTP)

On Broker Startup...

Service Broker

(i.e. mysql)

2Broker Services (un)provision and (un)bind

13

Rou

ter

Cloud Controller

Service Broker

(i.e. mysql)VM for Service

Instances

CLI

My Service(process)U

AA

provide shared secret (HTTP)

Prerequisite...

shared secret

2Broker Services (un)provision and (un)bind

14

Rou

ter

Cloud Controller

Service Broker


Instances

DEA

CLI

My Service(process)

up to you

Dropletcredentials

Bro

wse

r

UA

A (un)provision (HTTP)

(un)bind (HTTP)

credentials

3Orphan Management

15

Rou

ter

Cloud Controller

Service Broker


Instances

CLI

My Service(process)

Authenticate

up to you

Services Catalog

UA

A

token

Get catalog (HTTP)

Broker is responsible for eventual consistency...

New (V2) Services API• Simplifies things a bunch!• Unidirectional - everything initiated from Cloud Controller• Asynchronous provisioning• From V1 RESTful Services API:

– Replace offerings registration with endpoint that allows the CC to GET the offerings

– Remove orphan management

16

It’s Up To YOU Patterns

17

VM for Service Broker

Service Broker


My Service(process)

Node


My Service(process)

Node


My Service(process)

Node

NATS

provision

One option (CF v1 services)...


18


Service Broker


Node


Node


Node

NATS

provision

Another...

WardenSvc Process

WardenSvc Process

WardenSvc Process

WardenSvc Process





19


Service Broker

NATS

provision

And another...

NodeNode

Warden

Warden WardenSvc Process Svc Process

Svc Process

Node

BOSH deployment

packages

- service broker - your impl

- service node - services binaries - services warden - your node impl

20

jobs

- service broker - ref service broker pkg - start web service

- service node - ref service node pkg - start node

Provisions broker VMs

Provisions service VMs

Configure shared secret (for example)

But what about...

21

AppDirect Service Broker

Service Broker

Service Broker

MySQL MySvc

core

ERP Oracle

?

User-provided Service Instances

22

Rou

ter

Cloud Controller

CLI

UA

A

provide service credentials (HTTP)

Prerequisite...

ERP

OracleDEA

Dropletcredentials

Bro

wse

r

credentials(un)bind (HTTP)

Demo

23

register user provided service instance

Tell ‘em what you told ‘em...

§Extending services offerings within run.pivotal.io–Partnering

§Building services–Into your own Cloud Foundry

§User-provided service instances–For access to pre&externally-provisioned services


Buildpacks

Deploying to Cloud Foundry

26

Staging and Buildpacks

Buildpacks are responsible for preparing the machine image for an application

27

Application

Container

Runtime

Operating System

Libraries

DEA

Buildpack } Droplet

Libraries = runtime package managers like Ruby gems, Node modules, Python PIP packagesApp bits are downloaded into image by DEA, buildpack may move them as necessary

CompatibilityCloud Foundry buildpacks follow the Heroku buildpack design

Cloud Foundry and Heroku buildpacks are compatible (if you take care to make them compatible)

Other PaaS providers are adopting the buildpack design

28

You can choose to implement platform-specific features in buildpacks and make them incompatibleFor compiled languages (Java) CF expects source to be compiled to machine code before being pushed. Heroku buildpacks will compile. Buildpack is an emerging convention for PaaS

Built-in or BYO

cf push

The application is tested against a set of curated

buildpacks

29

cf push --buildpack <url>

The buildpack is referenced by a Git URL

Git != GitHub - use any public Git provider or local Git repositoriesGit repository must be visible on the network that CF is running on - public CF requires public Git repo, private CF can use private or public Git repo

Built-in Buildpacks

30

→→→

Built into the CF open-source codebase and into Pivotal Hosted CF. Builtin means they are symlinked into the DEA in the buildpacks directory. Symlinking other buildpacks makes them builtin also. NodeJS and Ruby buildpacks are forks of Heroku buildpacks. Java buildpack is written from scratch for CF.

Tested Buildpackshttps://github.com/cloudfoundry-community/cf-docs-contrib/wiki/Buildpacks

31

LanguagesContainers

Haskell

Not tested or endorsed by Pivotal, all community-contributed and maintained.

Existing buildpacks are a great place to start when creating a new buildpack from scratch.

/bin/detect app_directoryInspect app bits to determine buildpack applicability

/bin/compile app_directory cache_directory Download and install runtime, container, packages, libraries; install app bits as necessary

/bin/release app_directoryBuild app start command

32

Buildpack API

Written in bash shell scripts or Ruby

/bin/detect

Inspect the app bits to determine if the buildpack knows how to handle the application

33

Gemfile exists

package.json exists

setup.py exists

On match, return exit code 0 and write to STDOUT a string identifying the buildpack

(often just the name of the language supported)

Java intentionally left off this list because the CF Java buildpack has more complex detection criteria

/bin/detect

$ cf push

DEA iterates over built-in buildpacks calling

/bin/detect scripts until one of them returns

exit code 0

34

$ cf push --buildpack <url>

/bin/detect is not called

/bin/compile

Download and install any necessaryruntime (Java VM, Ruby interpreter, JavaScript interpreter)container (web server)support libraries, packages, modules (Ruby gems, NPM packages)

...and then installing the app bits into the runtime or container

35

This is where most of the buildpack work is done.More package manager examples - Python PIP, Perl CPAN

Installing the app may mean copying or moving the app files from the location DEA put them in, or symlinking them to a different location

/bin/compile CachingRuntime, container, and support packages are downloaded from sources external to Cloud Foundry

DEA provides a location for storing downloaded artifacts to speed subsequent staging operations

36

External source can include custom packages in S3 buckets or public mirrors for public CF, or internal download locations for private CF.

/bin/release

Build a YAML-formatted hash with three possible keys

37

addons: []config_vars: {}default_process_types: web: <start command>

On Cloud Foundry, currently only the web: value is used to get the start command for the app

If creating a buildpack for compatibility, consult documentation for other platforms to see how addons and config_vars might be used.

Java BuildpackSupports a variety of JVM languages, containers, and frameworks with a modular, configurable, and extensible design

38

Java Buildpack Concepts

39

Containers

How an application is run

Frameworks

Additional application transformations

JREsJava Runtimes

Frameworks - loose definition, basically anything that is orthogonal to the container

Java Buildpack Concepts

40

Containers Frameworks

JREs

OpenJDK

Java main()TomcatGroovy

Spring Boot CLIPlay

Spring configPlay config

Play JPA configNew Relic

Zero Effort Spring - Tue 12:45-2:15 - Dave Syer and Phil Webb

Container Detection Criteria

41

Java main() META-INF/MANIFEST.MF exists with Main-class attribute set

Tomcat WEB-INF directory exists

Groovy.groovy file with a main() method, or.groovy file with no classes, or.groovy file with a shebang (#!) declaration

Spring Boot CLIone or more POGO .groovy files with no main() method, andno WEB-INF directory

Play start and lib/play.play_*.jar exist

Choose zero or one

Detection is typically based on file existence and content of files.

Zero Effort Spring - Tue 12:45-2:15 - Dave Syer and Phil Webb

This list is likely to grow to support other containers like Spring XD.

Framework Detection Criteria

42

Spring spring-core*.jar exists

Play config Play application detected

Play JPA config play-java-jpa plugin exists in app

New Relic New Relic service bound to app

Choose all that apply

Detection is typically based on file existence and content of files. Can also be done based on services bound to an app.

/bin/compile Output Example-----> Downloaded app package (18M)

-----> Downloading OpenJDK 1.7.0_21 JRE (17.5s) Expanding JRE to .java (1.4s)

-----> Downloading Auto Reconfiguration 0.7.1 (1.4s) Modifying /WEB-INF/web.xml for Auto Reconfig-----> Downloading Tomcat 7.0.42 (3.5s) Expanding Tomcat to .tomcat (0.2s)

Downloading Buildpack Tomcat Support 1.1.1 (0.0s)-----> Uploading droplet (55M)

43

DEA

Buildpack

DEA

See What’s Going On $ cf files <app-name> app

.buildpack-diagnostics/

.java/

.lib/

.tomcat/

META-INF/

WEB-INF/

assets/

44

Buildpack-installed runtime Buildpack-installed support libraries Buildpack-installed container

DEA-downloaded application files

The buildpack-installed locations are decisions made by the Java buildpack - other buildpacks will put things in other places.

See What’s Going On $ cf files <app-name> staging_info.yml

detected_buildpack: openjdk-1.7.0_21, tomcat-7.0.42,

spring-auto-reconfiguration-0.7.1

start_command: JAVA_HOME=.java

JAVA_OPTS="-Dhttp.port=$PORT

-Djava.io.tmpdir=$TMPDIR -XX:MaxPermSize=52428K

-XX:OnOutOfMemoryError=./.buildpack-diagnostics/killjava

-Xmx384M -Xss1M" .tomcat/bin/catalina.sh run

45

CustomizationTwo ways to customize the Java buildpack

Configure artifacts used by standard JREs, Containers, and Frameworks

Extend the buildpack with your own JREs, Containers, and Frameworks

Customization is done by forking the buildpack

46

Awesome documentation is in the github repo

Customization by ConfigurationConfiguration files in java-buildpack/config determine the behavior of a JRE, Container, or Framework

47

# cloudfoundry/java-buildpack/config/openjdk.yml---version: 1.7.0_+

repository_root: "http://download.pivotal.io.s3.amazonaws.com/openjdk/{platform}/{architecture}"

memory_sizes:

memory_heuristics:

heap: 0.75

permgen: 0.1

stack: 0.05

native: 0.1

# http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/index.yml---1.6.0_27: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.6.0_27.tar.gz1.7.0_21: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_21.tar.gz1.7.0_25: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.7.0_25.tar.gz1.8.0_M6: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M6.tar.gz1.8.0_M7: http://download.pivotal.io.s3.amazonaws.com/openjdk/lucid/x86_64/openjdk-1.8.0_M7.tar.gz

version and repository_root are common to all components that download artifactsother options are specific to the component

1. consult <component-name>.yml2. read repository_root/index.yml3. look up the specified version in the index4. download the matching artifact

Customization by ConfigurationExample: customizing the Tomcat artifact for download

48

# cloudfoundry/java-buildpack/config/tomcat.yml---version: 7.0.42repository_root: "http://files.example.com/tomcat-custom"support: version: 1.1.+ repository_root: "http://files.example.com/tomcat-buildpack-support"

# http://files.example.com/tomcat-custom/index.yml---7.0.40: http://files.example.com/tomcat-custom/tomcat-7.0.40.tar.gz7.0.41: http://files.example.com/tomcat-custom/tomcat-7.0.41.tar.gz7.0.42: http://files.example.com/tomcat-custom/tomcat-7.0.42.tar.gz

Customization by ConfigurationTomcat container supports simple customization of context.xml and server.xml.

49

cloudfoundry/java-buildpack/resources/tomcat/confcontext.xmlserver.xml

fork, modify, push

Customization by ExtensionImplement a JRE, Container, or Framework support class as one Ruby file in the appropriate directory

(with additional support classes as necessary)

50

cloudfoundry/java-buildpack/lib/java_buildpack

jrecontainerframework

One class is sufficient for most extensionsExisting support classes serve as good examples

Customization by ExtensionSupport class types have similar interfaces, following the buildpack scripts naming conventions

51

# initialize the support class with platform information provided in context# context includes app_dir, lib_dir, environment, java_home, java_opts, vcap_application, vcap_servicesdef initialize(context)

# return a String or an Array<String> that uniquely identifies the container/framework/jre, or nildef detect

# download and unpack the container/framework/jre, and transform the application as necessarydef compile

# create and return the command to run the application with (containers) or add options to context[:java_opts] (frameworks)def release

if detect returns nil, compile and release will not be calledonly one container support class can return non-nil from detect

Customization by ExtensionAdd new support class to config/components.yml

52

---containers: - "JavaBuildpack::Container::Groovy" - "JavaBuildpack::Container::Main" - "JavaBuildpack::Container::SpringBootCli" - "JavaBuildpack::Container::Tomcat" - "JavaBuildpack::Container::Play"jres: - "JavaBuildpack::Jre::OpenJdk"frameworks: - "JavaBuildpack::Framework::JavaOpts" - "JavaBuildpack::Framework::NewRelic" - "JavaBuildpack::Framework::PlayAutoReconfiguration" - "JavaBuildpack::Framework::PlayJpaPlugin" - "JavaBuildpack::Framework::SpringAutoReconfiguration"

The buildpack will iterate over all components, applying them as necessary

CustomizationMuch more information and documentation included in the GitHub repository

https://github.com/cloudfoundry/java-buildpack

53

The buildpack will iterate over all components, applying them as necessary


Cornelia [email protected]@cdavisafc

Scott [email protected]

@scottyfred

Community Engineer, Cloud Foundry at Pivotal

extending cloud foundry with custom integration

Technology

service brokerprovisionvm

service instancesnats

service processvm

service process node

service instances wardennats

service instances wardenvm

provisions service vms

demo10create service