custom deployments with sbt-native-packager
TRANSCRIPT
Why Native Packager?
• Java = “run anywhere”
• Native Packager = “run Java anywhere”
• Classpath, JVM parameters, command-line arguments, environment variables, behaviour on quit, …
$ bin/my-‐first-‐app -‐h Usage: [options]
-‐h | -‐help print this message -‐v | -‐verbose this runner is chattier -‐d | -‐debug set sbt log level to debug -‐no-‐version-‐check Don't run the java version check. -‐main <classname> Define a custom main class -‐jvm-‐debug <port> Turn on JVM debugging, open at the given port.
# java version (default: java from PATH, currently java version "1.8.0_45") -‐java-‐home <path> alternate JAVA_HOME
# jvm options and output control JAVA_OPTS environment variable, if unset uses "" -‐Dkey=val pass -‐Dkey=val directly to the java runtime -‐J-‐X pass option -‐X directly to the java runtime (-‐J is stripped)
# special option -‐-‐ To stop parsing built-‐in commands from the rest of the command-‐line. e.g.) enabling debug and sending -‐d as app argument $ ./start-‐script -‐d -‐-‐ -‐d
In the case of duplicated or conflicting options, basically the order above shows precedence: JAVA_OPTS lowest, command line options highest except "-‐-‐".
What is an SBT API?
Setting[T]Computation run once, returning T
> set name := { println("hello world!”); "name" } [info] Defining *:name [info] Reapplying settings... hello world! [info] Set current project to name (in build file:/Users/gcoady/my-‐project/) > name [info] name
Task[T]Computation run every time value is needed, returning T
> set run := { println("hello world"); () } [info] Defining *:run [info] The new value will be used by no settings or tasks. [info] Reapplying settings... blah [info] Set current project to name (in build file:/Users/gcoady/my-‐project/) > run hello world [success] Total time: 0 s, completed 11-‐Feb-‐2016 14:59:18 > run hello world [success] Total time: 0 s, completed 11-‐Feb-‐2016 14:59:19
Dependencies
• Tasks can depend on other tasks and settings
• Settings can depend on other settings
KeysTyped identifier & documentation for settings/tasks
> inspect name [info] Setting: java.lang.String = name [info] Description: [info] Project name. [info] Provided by: [info] {file:/Users/gcoady/my-‐project/}my-‐project/*:name
Native Packager API
AutoPlugin EcosystemSbtNativePackager
UniversalPlugin
DockerPlugin
LinuxPlugin
WindowsPluginDebianPlugin RpmPlugin
JavaAppPackaging
Universal Configuration• Provided by Universal Plugin
• Platform-independent layout
• Staging
• Package zip & tgz files
• Depended on by other deployment formats
Universal Configuration
val mappings = TaskKey[Seq[(File, String)]]( "mappings", "Defines the mappings from a file to a path, used by packaging, for example.")
Starting your plugin
sbtPlugin := true// The Typesafe repositoryresolvers += "Typesafe repository" at "https://repo.typesafe.com/typesafe/releases/"addSbtPlugin("com.typesafe.sbt" % "sbt-native-packager" % "1.0.3" % "provided")
Starting your pluginobject MyPlugin extends AutoPlugin { val MyPluginConfig = config("myplugin") extend Universal object autoImport { val myPluginSetting = settingKey[Seq[String]](“A custom setting") val myPluginTask = taskKey[File](“A custom task") } import autoImport._ override val requires = JavaAppPackaging override val projectSettings = inConfig(MyPluginConfig)(Seq( // Configure settings for project here ))}
Case Study
• YourKit Profiler:An awesome CPU and memory Java Profiler
• Lots of annoying steps to install
Plugin Requirements
• Add platform-specific shared library
• Add flags to use library as Java agent
• Allow changes to configuration at deployment time
Adding resources$ find src/main/resources -‐type f src/main/resources/yjp-‐2015-‐build-‐15068/bin/aix-‐ppc-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/aix-‐ppc-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/freebsd-‐x86-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/freebsd-‐x86-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/hpux-‐ia64-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/hpux-‐ia64-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐aarch64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐armv5-‐sf/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐armv7-‐hf/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐ppc-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐ppc-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐ppc64le/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐x86-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/linux-‐x86-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/mac/libyjpagent.jnilib src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐sparc-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐sparc-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐x86-‐32/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/solaris-‐x86-‐64/libyjpagent.so src/main/resources/yjp-‐2015-‐build-‐15068/bin/win32/yjpagent.dll src/main/resources/yjp-‐2015-‐build-‐15068/bin/win64/yjpagent.dll
Adding resources
val stream = Option(getClass.getResourceAsStream(path)) stream match { case Some(s) => val tempFile = targetDir / "yourkit" / p / s"yourkit.$ext" tempFile.getParentFile.mkdirs() IO.transferAndClose(s, new java.io.FileOutputStream(tempFile))
Adding resources
mappings in Universal ++= yourKitAgents.value.map(agent => agent.sourceFile -> agent.targetPath )
Java entry point generation
• Provided by JavaAppPackaging
• Creates start script for Java applications
• Arbitrary bash code can be injected
• Adding Java agents to Java binaries
• Environment discovery & injection
Injecting arbitrary code
val bashScriptExtraDefines = TaskKey[Seq[String]]( “bashScriptExtraDefines", "A list of extra definitions that should be written to the bash file template.")
Bash script helpers
• ${app_path}/…/ — root directory of distribution
• addJava — adds a Java argument
• addApp — adds an argument to your application
• addResidual — adds an argument to your application, goes after non-residuals
Injecting bash code
def startYourKitScript(defaultStartupOptions: String): String = """ if [[ -‐z "$YOURKIT_AGENT_DISABLED" ]]; then if [[ -‐z "$YOURKIT_AGENT_STARTUP_OPTIONS" ]]; then YOURKIT_AGENT_STARTUP_OPTIONS="""" + defaultStartupOptions + """" export YOURKIT_AGENT_STARTUP_OPTIONS fi"""
bashScriptExtraDefines += """addJava "-‐agentpath:${app_home}/../""" + mapping + """=${YOURKIT_AGENT_STARTUP_OPTIONS}""""
Trying it out
addSbtPlugin("com.gilt.sbt" % "sbt-yourkit" % "0.0.2")
enablePlugins(PlayScala, YourKit)
Debugging
• Universal configuration IS universal
• universal:stage presents application layout
Online Examples• https://github.com/gilt/sbt-yourkit
• https://github.com/gilt/sbt-newrelicDownloads New Relic agent with Ivy Generates NR configuration from SBT settings Adds agent, configuration, and appropriate startup arguments
• https://github.com/gilt/sbt-aspectjweaver Downloads AspectJWeaver agent with Ivy Adds agent and appropriate startup arguments
Native Packager Supported Formats
• Zip/TGZ Archives
• Windows MSI
• OSX DMG
• Debian DEB
• Red Hat / Fedora RPM
• Docker
Why not one more?
Case Study: ACI Images
• Used by RKT (Docker alternative)
• TAR format
• Optional compression, GPG signatures
• /manifest: Image metadata
• /rootfs/: Image contents
Defining required keys
object autoImport { val aciDependencies = settingKey[Seq[String]](“ACI Dependencies") val aciManifest = taskKey[File]("ACI Manifest") }
Reusing existing keys
val Aci = config("aci") extend Universal
Create file/path mappingsmappings := ( renameDests((mappings in Universal).value, "rootfs") ++ Seq(aciManifest.value -‐> “manifest") )
def renameDest(originalPath: String, dest: String) = "%s/%s" format (dest, originalPath)def renameDests(from: Seq[(File, String)], dest: String) = for { (f, path) <-‐ from } yield (f, renameDest(path, dest))
Create target file
packageBin := Archives.makeTarball(Archives.gzip, “.aci")( target.value, normalizedName.value, mappings.value, None)
Summary
• Extending SBT Native Packager is easy
• Alter program environment
• Create alternative formats for distribution
Questions?