© 2014 IBM Corporation

1733 Liberty Buildpack:

Designed for Extension -

Integrating Services in IBM

BlueMixRohit Kelapure

Jim Stopyro

Please Note

IBM’s statements regarding its plans, directions, and intent are subject to change

or withdrawal without notice at IBM’s sole discretion.

Information regarding potential future products is intended to outline our general

product direction and it should not be relied on in making a purchasing decision.

The information mentioned regarding potential future products is not a

commitment, promise, or legal obligation to deliver any material, code or

functionality. Information about potential future products may not be incorporated

into any contract. The development, release, and timing of any future features or

functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM

benchmarks in a controlled environment. The actual throughput or performance

that any user will experience will vary depending upon many factors, including

considerations such as the amount of multiprogramming in the user’s job stream,

the I/O configuration, the storage configuration, and the workload processed.

Therefore, no assurance can be given that an individual user will achieve results

similar to those stated here.

1

BlueMix & Cloud Foundry Concepts

Applications: Artifact that the end developer is building

Services: Code that BlueMix hosts that offers functionality for apps ranging from utility functions to very complex business logic

Buildpack: A collection of code that is responsible for transforming pushed application artifacts into a ready to run droplet, in a process referred to as ‘staging’.

Droplet: A package containing everything that is needed in order to successfully run your application, e.g. JVM, Liberty core libraries, the application itself, short of the operating system.

Warden: The mechanism used to achieve application isolation in the cloud foundry environment

Organizations & spaces: organizational units in the Cloud Foundry infrastructure that can be used to store and track application resources

Users: Developer interacting with BlueMix

Boilerplate: Container of one application with multiple services

BOSH:

2

Cloud Foundry (20,000 ft view)

3

Cloud Foundry Architecture

4

Cloud Foundry Services

Managed Services

User-provided Service Instances

5

Deployment models

1. Entire service packaged and deployed by BOSH alongside

Cloud Foundry

2. Broker packaged and deployed by BOSH alongside Cloud

Foundry, rest of the service deployed and maintained by other

means

3. Broker and optionally service pushed as an application to

Cloud Foundry user space

4. Entire service, including broker, deployed and maintained

outside of Cloud Foundry by other means

6

Services in

BlueMix

BlueMix simplifies the use

of services by managing

the provisioning of new

instances of the service

and the binding of those

service instances to your

application.

7

WebSphere Liberty Buildpack

Buildpack for running applications on IBM WebSphere

Application Server Liberty Profile.

Designed to run "packaged" servers and web applications

Generates the liberty server configuration for a bound services

Simplify developers’ lives by requiring minimal configuration and

making it easy to consume services

Loads into the server only what is needed for a running

application

https://github.com/cloudfoundry/ibm-websphere-liberty-

buildpack

8

Responsibility of the Liberty Buildpack

The buildpack will convert and environment variables provided by CF into configuration variables for the Liberty server.

The variables end up in runtime-vars.xml, and are referenceable from a pushed server.xml

Auto generate configuration for the bound services

Auto wire resources references to the appropriate cloud service resource

Starting and stopping the Liberty server and controlling the process environment

Generate the Liberty server configuration when war/ear files are pushed

Pull in additional Liberty runtime packages when the bound services need them

9

Liberty Buildpack components

10

How applications are staged

11

Buildpack

Server configuration when pushing war/ear apps

12

Service information in runtime-vars.xml

13

Service configuration snippets in server.xml

14

How to write cloud foundry service

Develop a service broker using Spring Service Broker - this

repository provides a template for building v2 service brokers in

Spring, and includes an example implementation for MongoDB

1. Fork the project

2. Implement 3 interfaces in the

package alternatively, you can use the included

and just implement the other 2 interfaces

3. Ensure your service impls are annotated with

4. Build the project and run the tests: gradle build

5. Push the broker to Cloud Foundry as an app:

6. Register your service broker with CF:

15

Developing a service broker

When developing new Service Broker, a few questions need to be answered:

What happens when a new service instance is created?• A new database is created on an existing MongoDB server

What happens when a new bind request is received?• A new user is created for the database instance using the bind

request id sent by the cloud controller.

What credentials does an application need to bind to the service instance?

• The URI of the MongoDB server, the database created as a service instance, and the user account created as part of the binding request

Look at existing examples from

16

Service Broker API

17

Cloud Controller

18

External Service Providers

Third party services are on boarded to BlueMix via the IBM

Cloud Partner Marketplacehttp://www.ibm.com/cloud-

computing/us/en/partner-landing.html

You can send an email to joinnow@us.ibm.com requesting

information on how to become a BlueMix Partner Service.

19

Service Integration with the Liberty buildpack

BlueMix IBM Created services• Configuration service

• SQLDB

• Monitoring service

• Auto Scaling service

• Security Service

• Data Cache

• Session Cache

• Log Analytics

• BLUAcceleration

• Elastic MQ

• Cloudant

BlueMix IBM Community Services• MySQL

• PostgreSQL

• MongDB

• Twillio

20

Information about bound services is

available in the VCAP_SERVICES env var.

Some services are container managed.

(SessionCache) Require xml.

Some services can be either container

managed or application managed. (RDB)

Some services contain multiple features which can be separately enabled. (LogAnalysis)

Some services have local analogs (RDB, mongo) and some do not (LogAnalysis)

Services may require client driver jars, extension features (wxs esa), Liberty features, bootstrapping.properties.

Service Provider Plugins

Service plugins are driven in the compile phase of the liberty Buildpack

Service plugins can be written to handle automatic configuration of a bound service during application staging. This is also known as auto-configuration.

• One Service Plug-in for each service type (mongo, DataCache, SQLDB, etc.,)

Each service plugin implements the plugin API. A service plug-in requires :• Service.yml -Configuration for the service

• Service.rb - Impl. class for the service

ServiceManager.rb in the liberty buildpack orchestrates the creation of service plugin instances

• Parse VCAP_SERVICES to determine bound services

• Map each bound services to a plug-in instance type.

• Create instances and drive life-cycle

The same set of service plug-ins can be used to support service binding either at application deployment time, or at post deployment time (late binding).

• This is due to re-stage being required for any service binding update

A default service plugin generates the runtime variables that are included in the server.xml for every bound service

21

Service Provider API

• Service plugins can assume that the liberty server and the

application bits are extracted and available

• Service Brokers

22

Service Provider API <service_name>.rb

23

Service Provider plugin configuration <service_name>.yml

24

Best Practices for service providers

Users will be provided an option to opt-out of auto-configuration

when the generated configuration is not correct or if the service

plugin cannot discern intent correctly

Services can provision features from liberty runtime extended in

the droplet by returning true in ?

Service Provider framework does NOT deal with partitioned

server.xml configuration

Service plugins should implement extensive logging using

Do not create dependencies between services

Play nice with other service plugins and main compile event

loop of the buildpack

Follow consistent schema naming conventions in

VCAP_SERVICES.

25

Java EE Resource Auto-wiring for IBM created BlueMix

services

26

Development Workflow for a service plugin

Fork open source liberty buildpack

Ensure that your service plugin adheres to the APIs and guidelines laid down in this document

Provide good test coverage via rspec

Issue a github Pull Request against the OS Buildpack

Review of the PR ensues by committers of the liberty buildpack

Submitting the pull request will automatically run all of the rspecand rubocop tests using TravisCI

If everything is green and the review is complete then a committer can accept the pull request.

• This will run the tests for a second time

Code is merged into the buildpack!

Service Provider is responsible for keeping plugin impl up to date with any CF buildpack API changes

27

Constraints on applications using services running on

Liberty in BlueMix

You can make any sort of TCP outbound connection

SSH/SSL/HTTP/HTTPS/<any protocol> from a container inside of CF. This

must be possible for apps running inside of BlueMix to connect to a wide

variety of services located on the internet

Applications cannot involve service resource interaction in a 2 phase commit

If the service is not available the application should

• Fallback and degrade gracefully when possible.

• Fail fast when fallbacks aren’t available and rapidly recover

28

Troubleshooting and Debugging service binding and

providers with the liberty buildpack

Print VCAP_SERVICES environment variable in the application

as a debug string

Use the Service console link from Ace to view the state of the

service

Document procedures for log retrieval and page out when a

critical service goes down in production

When everything else fails delete the service instance, rebind

and push the app

Please note that restart of the app after binding a service

instance will not cause a restage of the app

29

Interesting BlueMix sessions

Creating & Consuming Shared Services for Codename: BlueMix

- CSD-2776

A Deep Dive into the Liberty Buildpack on IBM BlueMix - AAI-

1713

Messaging in the cloud with Elastic MQ, MQ Light and BlueMix -

AMC-1897

Reduce the Complexity of Application Delivery & Focus on What

Matters with DevOps Services & BlueMix - AAD-2132

Questions?

We Value Your Feedback

Don’t forget to submit your Impact session and speaker

feedback! Your feedback is very important to us – we use it to

continually improve the conference.

Use the Conference Mobile App or the online Agenda Builder to

quickly submit your survey

• Navigate to “Surveys” to see a view of surveys for sessions

you’ve attended

32

Thank You

33

Legal Disclaimer

• © IBM Corporation 2014. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained

in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are

subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing

contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and

conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or

capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to

future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by

you will result in any specific sales, revenue growth or other results.

• If the text contains performance statistics or references to benchmarks, insert the following language; otherwise delete:

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will

experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user's job stream, the I/O configuration, the storage

configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

• If the text includes any customer examples, please confirm we have prior written approval from such customer and insert the following language; otherwise delete:

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs

and performance characteristics may vary by customer.

• Please review text for proper trademark attribution of IBM products. At first use, each product name must be the full name and include appropriate trademark symbols (e.g., IBM

Lotus® Sametime® Unyte™). Subsequent references can drop “IBM” but should include the proper branding (e.g., Lotus Sametime Gateway, or WebSphere Application Server).

Please refer to http://www.ibm.com/legal/copytrade.shtml for guidance on which trademarks require the ® or ™ symbol. Do not use abbreviations for IBM product names in your

presentation. All product names must be used as adjectives rather than nouns. Please list all of the trademarks that you use in your presentation as follows; delete any not included in

your presentation. IBM, the IBM logo, Lotus, Lotus Notes, Notes, Domino, Quickr, Sametime, WebSphere, UC2, PartnerWorld and Lotusphere are trademarks of International

Business Machines Corporation in the United States, other countries, or both. Unyte is a trademark of WebDialogs, Inc., in the United States, other countries, or both.

• If you reference Adobe® in the text, please mark the first use and include the following; otherwise delete:

Adobe, the Adobe logo, PostScript, and the PostScript logo are either registered trademarks or trademarks of Adobe Systems Incorporated in the United States, and/or other countries.

• If you reference Java™ in the text, please mark the first use and include the following; otherwise delete:

Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.

• If you reference Microsoft® and/or Windows® in the text, please mark the first use and include the following, as applicable; otherwise delete:

Microsoft and Windows are trademarks of Microsoft Corporation in the United States, other countries, or both.

• If you reference Intel® and/or any of the following Intel products in the text, please mark the first use and include those that you use as follows; otherwise delete:

Intel, Intel Centrino, Celeron, Intel Xeon, Intel SpeedStep, Itanium, and Pentium are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and

other countries.

• If you reference UNIX® in the text, please mark the first use and include the following; otherwise delete:

UNIX is a registered trademark of The Open Group in the United States and other countries.

• If you reference Linux® in your presentation, please mark the first use and include the following; otherwise delete:

Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of

others.

• If the text/graphics include screenshots, no actual IBM employee names may be used (even your own), if your screenshots include fictitious company names (e.g., Renovations, Zeta

Bank, Acme) please update and insert the following; otherwise delete: All references to [insert fictitious company name] refer to a fictitious company and are used for illustration

purposes only.

34