mongola - cloud foundry

© 2012 VMware, Inc. All rights reserved

MongoDB App DuetRuby and Java on Cloud Foundry

By Monica Wilkinson and Josh LongCloud Foundry, Developer Relations

MongoLA 2012Thursday, January 19, 12

CONFIDENTIAL

About Monica Wilkinson

2

Loves the web and data portability.

Developer Advocate @ Cloud Foundry12 years development experience.

Last 5 years in Social WebOpen Web Standards Advocate

@[email protected]

Thursday, January 19, 12

CONFIDENTIAL

About Josh Long

3

Spring Developer [email protected]@springsource.com

this is important!

SpringSource.org/rooFree Book!


mailto:[email protected]


CONFIDENTIAL4

Cloud Foundry


CONFIDENTIAL

About Cloud Foundry

5

The first Open PaaS

Multi(n) Languages, Frameworks, Services & Clouds

Open Source


CONFIDENTIAL

STEP 1 - Get a Cloud Foundry account

https://my.cloudfoundry.com/signup/MongoLA

6


http://cloudfoundry.com/signup

http://cloudfoundry.com/signup

https://my.cloudfoundry.com/signup/HACK

https://my.cloudfoundry.com/signup/HACK

CONFIDENTIAL

STEP 2 - Download VMC

•sudo gem install vmc

•vmc login <username>

7


CONFIDENTIAL8

STEP 3: Push your app to Cloud Foundryvmc push --runtime=ruby19



You are doneService gets created and bound

Code gets pushed

App is Live immediately at the url you requested


CONFIDENTIAL

Every day development Debugging and accessing the data locally

• Caldecott --> Service tunneling. Access your Cloud Foundry service as if it was local.

10


CONFIDENTIAL

Tunneling

11

gem install caldecott

vmc tunnel <mongodb>


CONFIDENTIAL

Using your favorite tools

12


CONFIDENTIAL13


© 2012 VMware, Inc. All rights reserved14

Why MongoDB?


CONFIDENTIAL15

Data Access Challenge #1: Scale Horizontally


CONFIDENTIAL16

Data Access Challenge #2: Heterogeneous


CONFIDENTIAL17

New demands on data access

•... until we needed inexpensive horizontal scaling for some large web based applications ...

•... and we needed to deploy our apps in the cloud ...

* image courtesy of Bitcurrent


CONFIDENTIAL18

NoSQL offers several data store categories

ColumnKey-Value Document Graph

Redis, Riak

Cassandra,HBase

MongoDB Neo4J


CONFIDENTIAL

About Mongo DB

19

1. No-SQL database

2. Stores JSON-style documents

3. Horizontally scalable

4. Full Indexing Support

5. Open Source

6. Great Community(You!)

development : {

tools: many,

language_support: superb,

agility: high

},

production: {

speed: fast

fault_tolerance: true

scalability: high

=


CONFIDENTIAL20

NoSQL offers several data store categories

ColumnKey-Value Document Graph

MongoDB (who cares about the rest?)


CONFIDENTIAL21

Mongo and Ruby Deep Dive


CONFIDENTIAL

Integration with Box(.net)

22


CONFIDENTIAL23

• Requirement: Clone sample Box app for developers into their own Cloud Foundry account so they can learn how to use the BOX Api.

Box Sample App Creation Wizard


CONFIDENTIAL24

• No command to download all the files for an app or copy it to another account.

Challenges

•Solution: The Cloud•Get the code for an app from: GitHub•Use vmc gem to upload the contents to your desired cloud


CONFIDENTIAL

VMC as a gem

Can be used to authenticate users and exchange their credentials for an API token

Edit Applications and Services from your code•Files

•Environment Variables

Start and Stop Apps

25


CONFIDENTIAL

How we used VMC on this project

26

def create() begin @vmcclient.create_app(@manifest["name"], @manifest) rescue RuntimeError => ex if (ex.message =~ /Error 701/) new_candidate = @generator.next unless new_candidate.nil? change_name! new_candidate create else # Format is "Error #{parsed_body[:code]}: #{desc}" raise "App Url: #{@uri} is already taken" end else raise ex end end end

def delete() @vmcclient.delete_app(@manifest["name"]) end

def copy_code() @vmcclient.upload_app(@manifest["name"], @app_meta.build!) end


CONFIDENTIAL

More Challenges

External Models we didn’t control

• Cloud Foundry Apps: Files, Runtime, Services, Env Vars, etc

• GitHub Repositories: Files, Commits, branches, tags

• Solution

• Use an Object Document Mapper. I chose Mongoid

• Easy to add new objects and properties

• No Migrations !

27


CONFIDENTIAL28


CONFIDENTIAL

Things you should know about Mongoid Project Page - http://mongoid.org.

Features:• Uses Active Model (Dirty Tracking, Mass Assignment

Security, Validations, Callbacks, JSON/XML Serialization)• Indexing• Supports identity map, replica sets and sharding• Custom field serialization

• A Range in Ruby 1...3 could be stored in MongoDB as a Hash { :min => 1, :max => 3 }

• Localization Support:(Translated Fields, i18n Fallbacks)• Inheritance

29


http://mongoid.org/

http://mongoid.org/

CONFIDENTIAL

How we used Mongoid

30

module CloudFoundry class AppInfo include TmpZip include Mongoid::Document

embeds_many :app_clone_requests belongs_to :repo, :class_name => "GitHub::RepositorySnapshot"

field :display_name, :type => String field :description, :type => String field :instances, :type => Integer, :default => 1 field :memory, :type => Integer, :default => 128 field :runtime, :type => String field :framework, :type => String field :env_vars, :type => Hash field :thumb_url, :type => String field :browseable, :type => Boolean field :cloneable, :type => Boolean field :starting_url, :type => String

index :display_name, :unique => true

validates_presence_of :display_name, :runtime, :framework validates_presence_of :repo, :if => :cloneable


CONFIDENTIAL

How we used Mongoid

31

module GitHub class RepositorySnapshot include TmpZip include Mongoid::Document

field :url, :type => String field :name, :type => String field :parent, :type => String field :branch, :type => String, :default => 'master' field :tag, :type => String field :commit, :type => String

index :url, :unique => true

validates_presence_of :url, :name, :parent, :commit, :branch

def url=(value) parts = value.gsub(/https\:\/\/github.com\//, '').split('/') if (parts.length == 2) write_attribute(:parent, parts[0]) write_attribute(:name, parts[1]) else write_attribute(:parent, nil) write_attribute(:name, nil) end write_attribute(:url, value) end


CONFIDENTIAL32

Getting your ruby app to use the proper module CloudFoundry

class Mongo

def self.config

Mongoid.configure do |config|

conn_info = nil

dbname = 'db'

if ENV['VCAP_SERVICES']

services = JSON.parse ENV['VCAP_SERVICES']

services.each do |service_version, bindings|

mongo_binding = bindings.find {|binding| binding['label'] =~ /mongo/i}

conn_info = mongo_binding['credentials'] if mongo_binding

end

raise "ERROR - Could not find connection info for mongo" unless conn_info

else

conn_info = {'hostname' => 'localhost', 'port' => 27017}

dbname = "gallery_db"

end

@@cnx = Mongo::Connection.new conn_info['hostname'], conn_info['port'], :pool_size => 5, :timeout => 5

db = @@cnx[dbname]

if conn_info['username'] and conn_info['password']

db.authenticate conn_info['username'], conn_info['password']

end

config.master = db

end

end

end

end


CONFIDENTIAL33

Demo


CONFIDENTIAL

Durran, creator of Mongoid detailed for us:

34

Full atomic update support out of the box for *all* MongoDB atomic operations, either explicitly or handled under the covers by Mongoid itself with support for embedded n levels deep. ie:

model = Model.find(id)

model.field = “value”

model.relations.build(field: “value”)

model.save

#=> Does

{ field: { “$set” : “value” },

relations: { “$push” : { field: “value” }}} for you.

*or things like*

model.add_to_set(field: “value”)


CONFIDENTIAL

Durran, creator of Mongoid detailed for us

35

“Smart memory management during iteration and working with relations: Mongoid never loads everything into memory unless you specifically want it to. It can handle working with millions of documents without putting stress on the server/RAM.”


CONFIDENTIAL

More Challenges Make app fast.

• Downloading and copying files from app to app is not cheap

• Solution• Avoid downloading the build if it has not changed

• Local cache using Mongo GridFS

36


CONFIDENTIAL

Working with GridFS Saving and Reading filesOnce you have a Grid object, you can start saving data to it.

@db = Mongo::Connection.new.db('social_site')@grid = Grid.new(@db)

# On this Cloud Foundry App@grid = Mongo::GridFileSystem.new(Mongoid.database)

# Saving IO data and including the optional filenameimage = File.open("me.jpg")id = @grid.put(image, :filename => "me.jpg")Grid#put returns an object id, which you can use to retrieve the file:

# Get the file we savedimage = @grid.get(id)

Deleting filesDeleting a file is as simple as providing the id:

@grid.delete(id2)

37


CONFIDENTIAL38

Spring Data on Cloud Foundry Deep Dive


CONFIDENTIAL39

Spring Frameworkbuilt-in data access support

•Transaction abstractions

•Common data access exception hierarchy

•JDBC - JdbcTemplate

•ORM - Hibernate, JPA support

•OXM - Object to XML mapping

•Serializer/Deserializer strategies (Spring 3.0)

•Cache support (Spring 3.1)


CONFIDENTIAL40

Spring Data Building Blocks

•Low level data access APIs✓MongoTemplate, RedisTemplate ...

•Object Mapping (Java and GORM)

•Cross Store Persistence Programming model

•Generic Repository support

•Productivity support in Roo and Grails


CONFIDENTIAL41

Spring Data Document Mongo

•MongoTemplate

•MongoConverter interface for mapping Mongo documents

•SimpleMongoConverter for basic POJO mapping support

•Leverage Spring 3.0 TypeConverters and SpEL

•Exception translation

•Advanced Mapping

•Annotation based (@Document, @Id, @DbRef)

•MongoRepository

•Built on Hades support for JPA Repositories


CONFIDENTIAL42

Simple Domain Class


CONFIDENTIAL43

Mongo TemplateDirect Usage of the Mongo Template:


CONFIDENTIAL43


Insert into “Person”

Collection


CONFIDENTIAL43


findOne using query: { "name" : "Joe"}

in db.collection: database.Person


CONFIDENTIAL43


Dropped collection [database.person]


CONFIDENTIAL44

Generic Repository Interface for generic CRUD operations on a repository for a specific type


CONFIDENTIAL45

Paging and Sorting RepositoryExtends “CrudRepository” Paging and Sorting Repository:


CONFIDENTIAL45

Paging and Sorting RepositoryExtends “CrudRepository”

Usage:

Paging and Sorting Repository:


CONFIDENTIAL46

Custom Repository Custom Repository:


CONFIDENTIAL46

Custom Repository Custom Repository:

Keyword Sample Logical result

GreaterThan findByAgeGreaterThan(int age) {"age" : {"$gt" : age}}

LessThan findByAgeLessThan(int age) {"age" : {"$lt" : age}}

Between findByAgeBetween(int from, int to) {"age" : {"$gt" : from, "$lt" : to}}

NotNull findByFirstnameNotNull() {”firstname" : {"$ne" : null}}

Null findByFirstnameNull() {”firstname" : null}

Like findByFirstnameLike(String name) "firstname" : firstname} (regex)

Keywords :


CONFIDENTIAL47

JPA and MongoDB JPA “Customer” with a “SurveyInfo” Document


CONFIDENTIAL48

Using a Cross-StoreSaving a Customer with a SurveryInfo


CONFIDENTIAL48

Using a Cross-StoreSaving a Customer with a SurveryInfoCreate Customer


CONFIDENTIAL48


Create SurveyInfo


CONFIDENTIAL48


Assign Survey to Customer


CONFIDENTIAL48


Save


CONFIDENTIAL48


Save

Mongo Document:


CONFIDENTIAL

Accessing Services Bound to Cloud Foundry

49

private String mongoDatabaseServiceName = "survey-mongo";

@Beanpublic CloudEnvironment cloudEnvironment() { return new CloudEnvironment();}

@Beanpublic MongoServiceInfo mongoServiceInfo() { return cloudEnvironment().getServiceInfo( mongoDatabaseServiceName, MongoServiceInfo.class);}

@Beanpublic MongoDbFactory mongoDbFactory() { MongoServiceCreator msc = new MongoServiceCreator(); MongoDbFactory db = msc.createService(mongoServiceInfo());}


CONFIDENTIAL

Accessing Services Bound to Cloud Foundry

50

@Inject private Mongo mongo ;

@Inject private MongoTemplate mongoTemplate;


CONFIDENTIAL51

Demo 2



Thank YouCloudFoundry Spring Source Samples:

http://bit.ly/cloudfoundry-samples

http://cloudfoundry.com

Questions: @cloudfoundry

•Josh -> @starbuxman or [email protected]

•Monica -> @ciberch or [email protected]


https://github.com/springsource/cloudfoundry-samples

https://github.com/springsource/cloudfoundry-samples





http://twitter.com/cloudfoundry

http://twitter.com/cloudfoundry

http://twitter.com/ciberch






mongola - cloud foundry

Technology