Norman Richardsorb@nostacktrace.comImmutant

Clojure Web Development

$ lein new compojure myapp

$ lein ring server

Congratulations, you now have a Clojure web application listening on port 3000!

Immutant

What about?

? database

? messaging

? scheduling

? clustering

? production

Immutant

+ Based on JBoss AS 7

+ Makes use of Clojure standards (ring)

+ Production-ready stack out of the box

Setup

+ add immutant plugin to ~/.lein/profiles.clj

{:user {:plugins [[lein-immutant "1.0.0"]]}

Setup

$ lein immutant installDownloading http://repository-projectodd.forge.cloudbees.com/release/org/immutant/immutant-dist/1.0.0/immutant-dist-1.0.0-slim.zipdone! Extracting /var/folders/jw/p379vgcn62q51q4_95xm1v6m0000gp/T/lein-immutant-8041099c-b2c5-4342-ac81-9e3986dfcbd2/immutant-dist-1.0.0-slim.zipExtracted /Users/norman/.immutant/releases/immutant-1.0.0-slimLinking /Users/norman/.immutant/current to /Users/norman/.immutant/releases/immutant-1.0.0-slimThe following versions of Immutant are installed to /Users/norman/.immutant(* is currently active via /Users/norman/.immutant/current):

* 1.0.0 (type: slim)

Setup

$ lein immutant deployDeployed myapp to /Users/norman/.immutant/current/jboss/standalone/deployments/myapp.clj

Setup

$ lein immutant runStarting Immutant: /Users/norman/.immutant/current/jboss/bin/standalone.sh=========================================================================

JBoss Bootstrap Environment

JBOSS_HOME: /Users/norman/.immutant/current/jboss

JAVA: java

JAVA_OPTS: -server -XX:+UseCompressedOops -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

=========================================================================

16:13:01,823 INFO [org.jboss.modules] (main) JBoss Modules version 1.2.0.CR116:13:02,294 INFO [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA16:13:02,344 INFO [org.jboss.as] (MSC service thread 1-6) JBAS015899: JBoss AS 7.2.x.slim.incremental.6 "Janus" starting ... ... ...16:13:12,534 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.2.x.slim.incremental.6 "Janus" started in 10969ms - Started 128 of 175 services (46 services are passive or on-demand)

Setup

$ lein immutant run --clustered ... ...16:17:44,573 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss AS 7.2.x.slim.incremental.6 "Janus" started in 10170ms - Started 130 of 223 services (92 services are passive or on-demand)

Housekeeping

$ lein immutant undeployUndeployed /Users/norman/.immutant/releases/immutant-1.0.0-slim/jboss/standalone/deployments/myapp.clj

Housekeeping

$ lein immutant env immutant-home: /Users/norman/.immutant/current jboss-home: /Users/norman/.immutant/current/jboss

Housekeeping

$ lein immutant listThe following applications are deployed to /Users/norman/.immutant/current: blue-app.clj (status: deployed) red-app.clj (status: deployed)

Modules+ immutant.web+ immutant.xa+ immutant.cache+ immutant.messaging+ immutant.jobs+ immutant.daemons

immutant.web

(defproject somewebapp "0.1.0-SNAPSHOT" ;; ... :plugins [[lein-ring "0.8.5"]] :ring {:handler somewebapp.handler/app})

(defproject somewebapp "0.1.0-SNAPSHOT" ;; ... :immutant {:nrepl-port 0 :context-path "/"})

immutant.web

(ns immutant.init  (:require [somewebapp.handler :as handler]            [immutant.web :as web]))

(web/start "/" handler/app)

immutant.web

/context-prefix /handler-path /app-path

From the deployment

From immutant init From your application

immutant.web

(def app (-> app-routes ;; additional handlers handler/site))

(def app (let [store (immutant.web.session/servlet-store)] (-> app-routes ;; additional handlers (handler/site {:session {:store store}}))))

immutant.web

+ works with non-ring apps (eg. Pedestal)+ control over virtual hosts and context paths+ can dynamically start/stop endpoints

+ minimal changes to your code, framework agnostic

immutant.xa

(defonce my-datasource (xa/datasource "mydb" {:adapter "postgres" :host "33.33.33.50" :username "db-user" :password "db-password" :database "my-awesome-db"}))

(def db-spec {:datasource my-datasource})

immutant.xa

(jdbc/with-connection db-spec (jdbc/do-commands "CREATE TABLE widgets ( name VARCHAR NOT NULL PRIMARY KEY, count VARCHAR NOT NULL DEFAULT 0);"))

(jdbc/with-connection db-spec (korma.core/select :widgets))

(def db-spec {:name "java:jboss/datasources/SomeExternalDS"})

immutant.xa

(defn new-widget [name number] (let [widget {:name name :count number}] (xa/transaction (jdbc/with-connection db-spec (korma/insert :widgets (korma/values widget))) (msg/publish "/queue/widgets" widget))))

+ integrates well with Clojure database things+ XA transactions are tricky and hard to test+ Most apps use many non-XA resources

+ database config can be external and shared

immutant.xa

+ All the immutant services are XA

immutant.cache

(def game-cache (cache/create "game" :persist true :sync true)) (def job-cache (cache/create "job-status" :ttl 10 :idle 1 :units :minutes))

immutant.cache

(defn add-points [player points] (let [current-score (get game-cache player 0) new-score (+ current-score points)] (cache/put game-cache player new-score) new-score))

immutant.cache

+ Really easy to use+ Interface isn’t very Clojure-like, needs abstraction+ Should a data grid be deployed along side app?

+ Infinispan does much more than caching

immutant.messaging

(msg/start "/queue/orders")(msg/start "/topic/new-widget") (defn process-order [order] (println "processing" order) (msg/publish "/topic/new-widget" {:widget order})) (defonce widgets (atom []))(defn cache-widget [widget] (swap! widgets #(conj % widget)) (println "I have" (count @widgets) "widgets")) (msg/listen "/queue/orders" process-order)(msg/listen "/topic/new-widget" cache-widget)

immutant.messaging

+ Priority, expiration, persistence, properties+ Easy to toss around Clojure data structures+ Pipelines are very cool

+ Topics and queues

immutant.jobs(defonce heartbeat (atom nil)) (defn send-heartbeat [] (riemann/send-event {:service "heartbeat" :state "ok" :metric 1})) (defn heartbeat-thread [every-n-sec] (while true (send-heartbeat) (Thread/sleep (* 1000 every-n-sec)))) (defn start-heartbeat [] (swap! heartbeat (fn [heartbeat] (or heartbeat (doto (Thread. #(heartbeat-thread 10)) .start)))))

 (defn stop-heartbeat [] (swap! heartbeat (fn [heartbeat] (when heartbeat (.stop heartbeat)))))

immutant.jobs

(defn send-heartbeat [] (riemann/send-event {:service "heartbeat" :state "ok" :metric 1})) (defn start-heartbeat [] (jobs/schedule :heartbeat send-heartbeat :every 10000)) (defn stop-heartbeat [] (jobs/unschedule :heartbeat))

+ Also supports CRON-like scheduling syntax+ Can be used for ad-hoc jobs+ Not easy to see what jobs are scheduled/running

+ Jobs can be on all nodes or just one

immutant.jobs

immutant.daemons

(defonce listener (atom nil))(defn start-listener [] (swap! listener (fn [listener] (msg/listen "/queue/guesses" play/handle-guess)))) (defn stop-listener [] (swap! listener (fn [listener] (when listener (msg/unlisten listener))))) (daemon/daemonize "guess-listener" start-listener stop-listener :singleton true)

immutant.daemons

+ Can be long running+ Singleton processes can be reliably setup

+ Manage application-specific services

The PI Game

πhttps://github.com/orb/pi-game

PI Game Demo

+ EDN api to get state and send actions+ Game state saved in persistent cache

+ Clojurescript front end, Clojure back end

++

+ Player actions are sent to guess queue

Singleton daemon creates one listener per cluster

Demo uses two immutant nodes with nginx in front

But...

• Works out of the box

• It’s good Clojure - not “Clojure 2 Enterprise Edition”

• Best solution for multiple applications

• Best solution when you want to work with Java or Ruby apps

• JBoss is production tested

• Responsive dev team

• Are these the services you would have chosen?

• It’s one big blob, not lots of composable blocks

• Does it fit your application architecture?

• Does it fit your ops strategy?

Yes

• App was ring/jetty - using our own custom startup

• Wanted to split our APIs out and modularize app instead of deploying as a monolithic app

• Wanted to be able to support rolling updates across nodes

• Liked the idea of being able to bring mature clustering, caching and messaging into the app almost for free

Immutant at Threatgrid

• Transition to Immutant was very fast and required only minor changes to our customized startup code

• We ran into a few small issues, but the Immutant dev team had almost immediate fixes/workarounds.

• Transitional codebase still supports running outside Immutant, so we haven’t been able to leverage some features

• Ops team strongly prefers individual services that can be managed independently - Immutant is one big ball

• Our web nodes and data nodes scale separately - Immutant seems to prefer a more homogenous deployment

• Immutant a la carte would really hit a sweet spot

And.... we’re still deciding