the future of jruby - baruco 2013

The FUTURE of

Tuesday, September 17, 13

Me

• Charles Oliver Nutter

• @headius

• Java developer since 1996

• JRuby developer since 2006

• Red Hat / JBoss polyglot group


Have you heard ofJRuby?


Have you triedJRuby?


Are you usingJRuby?


What is JRuby?


Ruby on the JVM


Ruby on the JVM

I don't like Java so I don't like JRuby


Ruby on the JVM


LOL applet

s


Ruby on the JVM

JVM SUCKS R

OFL


LOL applet

s


Ruby on the JVM

JVM SUCKS R

OFLAbstractMetaRubyImplementationFactoryFactoryImpl


LOL applet

s


Welcome to Spain


JRuby is Ruby!!!on the JVM... shhh!


The Basics• Compatible with Ruby 1.8.7, 1.9.3

• Mostly written in (clean) Java

• More and more in Ruby going forward

• Entire world of JVM libraries available

JVM

JDK Classes Other Libraries

JRuby Core Classes JRuby Runtime

More Core Classes Standard Lib Extras

Your Application

FFITuesday, September 17, 13

Roadmap

1.6.0 1.6.1

1.6.2 1.6.3 1.6.7.2

1.7.0.pre1

1.6.4 ... 1.6.8

...1.7.3 1.7.4 1.7.5

1.7.6

...9000!

Next week or two


JRuby 9000 really is the next version...

9k...Coming 2014


One point release later...


9K Questions

• Ruby 2.0 or 2.1-only?

• 1.8 support gone

• 1.9 support gone

• Java 7+ only?

• New compiler will be... ?


Why JRuby?


JRuby Team


JRuby Team

Charlie


JRuby Team

Charlie Tom

Nick Hiro Marcin Nahi Wayne Subbu DouglasDouglasContribsDouglas


JRuby Team

Charlie Tom

Nick Hiro Marcin Nahi Wayne Subbu DouglasDouglasContribs

DouglasDouglasOpenJDKDouglasDouglasAndroid

DouglasDouglasJ9DouglasDouglasOther

JVMs

Douglas


JVM Over Time

0

7.5

15

22.5

30

Java 1.4 Java 5 Java 6 Java 7

JRuby 1.0.3 (bm_red_black_tree.rb)


Versus MRI 1.8

0

7.5

15

22.5

30

Java 1.4 Java 5 Java 6 Java 7

JRuby 1.0.3 (bm_red_black_tree.rb) MRI 1.8


0ms

75ms

150ms

225ms

300ms

188KB/29MB 27MB/127MB 199MB/238MB

Time per GC versus heap usage

Tim

e pe

r G

C

Heap usage (MRI/JRuby)

Ruby 2.0.0 JRuby


Features Over Time

1.6 1.8.4 1.8.6 1.8.7 1.9.2 1.9.3 2.0 2.1

Ruby FeaturesJRuby Support


However...

• JVM development is not fast

• JRuby must move forward

• Constantly improving


What could be better?

Performance

Native Extensions

Concurrency

Startup Time


Performance


Hard to Optimize

• Dynamic calls with lots of overhead

• Dynamic object structure with indirection

• Lots and lots of objects


Solutions


Compile to Bytecode

• JVM likes JVM bytecode (surprise!)

• Simple compilation of Ruby

• Let JVM do the work

• Can we do better?


Invokedynamic

• New JVM feature for languages

• Bytecode + IR to describe calls

• JVM patches straight through

• Optimize any kind of call like Java

• Ruby as fast as Java...in theory


0 1 2 3 4

ruby-1.9.3 + Ruby

ruby-2.0.0 + Ruby

maglev + Ruby

macruby-0.12 + Ruby

rbx-2.0.0rc1 + Ruby

ruby-1.9.3 + C ext

ruby-2.0.0 + C ext

jruby + Ruby

jruby + Java ext

red/black tree, pure Ruby versus native

Runtime per iteration


0 1 2 3 4

ruby-1.9.3 + Ruby

ruby-2.0.0 + Ruby

maglev + Ruby

macruby-0.12 + Ruby

rbx-2.0.0rc1 + Ruby

ruby-1.9.3 + C ext

ruby-2.0.0 + C ext

jruby + Ruby

jruby + Java ext

3.96s

2.48s




0 1 2 3 4

ruby-1.9.3 + Ruby

ruby-2.0.0 + Ruby

maglev + Ruby

macruby-0.12 + Ruby

rbx-2.0.0rc1 + Ruby

ruby-1.9.3 + C ext

ruby-2.0.0 + C ext

jruby + Ruby

jruby + Java ext

3.96s

2.48s

1.39s

1.19s




0 1 2 3 4

ruby-1.9.3 + Ruby

ruby-2.0.0 + Ruby

maglev + Ruby

macruby-0.12 + Ruby

rbx-2.0.0rc1 + Ruby

ruby-1.9.3 + C ext

ruby-2.0.0 + C ext

jruby + Ruby

jruby + Java ext

3.96s

2.48s

1.39s

1.19s

0.51s

0.51s

0.51s




0 1 2 3 4

ruby-1.9.3 + Ruby

ruby-2.0.0 + Ruby

maglev + Ruby

macruby-0.12 + Ruby

rbx-2.0.0rc1 + Ruby

ruby-1.9.3 + C ext

ruby-2.0.0 + C ext

jruby + Ruby

jruby + Java ext

3.96s

2.48s

1.39s

1.19s

0.51s

0.51s

0.51s

0.29s

0.1s




But...

• Indy was really slow in first Java 7 release

• Got fast in 7u2...and turned out broken

• Rewritten for 7u40

• Slow to warm up

• Getting reports that there's still issues

• Java 8 due in March


Other Options

• New IR compiler/runtime in 9k

• Optimize Ruby code before JVM

• Specialize types, elide allocations


LexicalAnalysisParsing

SemanticAnalysis

Optimization

Bytecode Generation

Interpret

AST

IR Instructions

CFG DFG ...

Existing

New!

Dalvik Generation ...


1 check_arity(2, 0, -1)2 a(0:0) = recv_pre_reqd_arg(0)3 thread_poll4 line_num(2)5 %v_2 = call(+, a(0:0), [1:Fixnum])6 return(%v_2)

-Xir.passes=LocalOptimizationPass,DeadCodeElimination

def foo(a, b) c = 1 d = a + cend

0 check_arity(2, 0, -1)1 a(0:0) = recv_pre_reqd_arg(0)2 b(0:1) = recv_pre_reqd_arg(1)3 %block(0:2) = recv_closure4 thread_poll5 line_num(1)6 c(0:3) = 1:fixnum7 line_num(2)8 %v_0 = call(+, a(0:0), [c(0:3)])9 d(0:4) = copy(%v_0)10 return(%v_0)

Optimization

propagation


Other Options

• Truffle/Graal

• New compiler backends from Oracle

• Graal = direct API to native JIT

• Truffle = magic optimizing AST atop Graal

• Ruby on Truffle 5x-6x faster than JRuby

• But...


The Hard Part {


Sooo....

• Keep working with JVM guys on InDy

• Get our own optimizing compiler done

• Explore Graal/Truffle backend

• Compiler geeks wanted! :-)


Concurrency


True Parallellism

Ruby Threads

NativeThreads

Ruby 1.8.7 Ruby 2.0.0

Green Threading

CPU Coresin Use

JRuby

Global LockSingle Thread Real Threading


Multicore in MRI

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

Ten-way concurrency * 200MB = 2GB


Multicore in MRI

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

100-way concurrency * 200MB = 20GB

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance

200MB MRI Instance


Multicore in JRuby

300MB JRuby Instance

One instance across 10 threads = 300MB


Multicore in JRuby

300MB JRuby Instance

One instance across 100 threads = 300MB


But...

• Ruby world is still growing up

• Concurrency tools being created

• Libraries being made threadsafe

• We need to do more to help


Unsafe Operations

• Concurrent read+write on core structures

• Non-atomic updates

•@count +=1

•@cache ||= MyCache.new

• Thread pooling

• Coordinating threads


thread_safe

• Concurrency-safe Hash

• Concurrency-safe Array

require 'thread_safe'

sa = ThreadSafe::Array.newsh = ThreadSafe::Hash.new


hamster• Persistent collections for Ruby

• A la Clojure and others

simon = Hamster.hash(:name => "Simon", :gender => :male) simon[:name] # => "Simon"simon.get(:gender) # => :male james = simon.put(:name, "James") # => {:name => "James", :gender => :male}simon # => {:name => "Simon", :gender => :male}james[:name] # => "James"simon[:name] # => "Simon" male = simon.delete(:name) # => {:gender => :male}simon # => {:name => "Simon", :gender => :male}male.has_key?(:name) # => falsesimon.has_key?(:name) # => true


atomic

• Atomic value holder

• Safely update current value

• Edit value only if unchanged

• Full CPU-level atomicity guarantees


require 'atomic'

my_atomic = Atomic.new(0)my_atomic.value # => 0my_atomic.value = 1my_atomic.swap(2) # => 1my_atomic.compare_and_swap(2, 3) # => true, updated to 3my_atomic.compare_and_swap(2, 3) # => false, current is not 2

my_atomic = Atomic.new(0)my_atomic.update {|v| v + 1}begin my_atomic.try_update {|v| v + 1}rescue Atomic::ConcurrentUpdateError => cue # deal with it (retry, propagate, etc)end


jo

• Threaded implementation of "goroutines"

• "channel" for communication


# pinger ponger printerdef pinger(c) 20.times { c << 'ping' }end

def ponger(c) 20.times { c << 'pong' }end

def printer(c) 40.times do puts c.take sleep 1 endend

c = chanjo {pinger(c)} # all on separate threadsjo {ponger(c)}jo {printer(c)}


Bottom Line

• Concurrency can work in Ruby

• Use the right tools and patterns

• Immutability FTW

• Test your apps and libs on JRuby!


Native Extensions


Why Not Ruby?

• Performance

• Fine grained (lots of calls down to C)

• Coarse grained (toss work over the wall)

• Library access


JRuby 1.6 C Exts

• Limited support (now disabled)

• Will be moved to external gem

• If you want it, support it

• Some stuff worked...most didn’t


Problems

• Performance

• Data copying to emulate raw structs

• Locking to keep C code thread-safe

• Multiple JRuby instances in one JVM

• No way from C to know which one

• Huge API to support


Alternatives


Java Integration

• Call Java (Scala, Clojure, ...) from Ruby

• Smart mapping of method names

• Type conversions as appropriate

• Super easy and fun


import javax.swing.JFrameimport javax.swing.JLable

frame = JFrame.new("Window")label = JLabel.new("Hello")

frame.add(label)frame.default_close_operation = JFrame::EXIT_ON_CLOSEframe.packframe.visible = true


Java Native Extensions

• Similar to C ext for MRI, but with Java

• Fast call protocol...basically free

• Same GC for all objects

• Have to keep in sync if C version too


FFI

• Ruby API/DSL for calling native code

• Runs on all Ruby impls

• Maintained by JRuby team!

• Solves "access" use case

• Works well for coarse-grained calls


Ruby FFI exampleclass Timeval < FFI::Struct layout :tv_sec => :ulong, :tv_usec => :ulongend

module LibC extend FFI::Library ffi_lib FFI::Library::LIBC attach_function :gettimeofday, [ :pointer, :pointer ], :intend

t = Timeval.newLibC.gettimeofday(t.pointer, nil)


But...

• Struct binding issues

• Across OSes (also 32+64 bit archs)

• Across library versions

• Library compile option mismatches

• Fine-grained perf sometimes suffers


Ruby FFI Generator

• https://github.com/neelance/ffi-gen

• Clang-based Ruby FFI generator

• Used to generate clang binding it uses

• It's meta!


https://github.com/neelance/ffi-gen

https://github.com/neelance/ffi-gen

require "ffi/gen"

FFI::Gen.generate( module_name: "Clang", ffi_lib: "clang", headers: ["clang-c/Index.h"], cflags: `llvm-config --cflags`.split(" "), prefixes: ["clang_", "CX"], output: "clang-c/index.rb")


# A single translation unit, which resides in an index. class TranslationUnitImpl < FFI::Struct layout :dummy, :char end

# Identifies a specific source location within a translation # unit. # # Use clang_getExpansionLocation() or clang_getSpellingLocation() # to map a source location to a particular file, line, and column. # # = Fields: # :ptr_data :: # (Array<FFI::Pointer(*Void)>) # :int_data :: # (Integer) class SourceLocation < FFI::Struct layout :ptr_data, [:pointer, 2], :int_data, :uint end


# Retrieves the source location associated with a given file/line/column # in a particular translation unit. # # @method get_location(tu, file, line, column) # @param [TranslationUnitImpl] tu # @param [FFI::Pointer(File)] file # @param [Integer] line # @param [Integer] column # @return [SourceLocation] # @scope class attach_function :get_location, :clang_getLocation, [TranslationUnitImpl, :pointer, :uint, :uint], SourceLocation.by_value


XNI

• Ruby + plain old C

• Covers access and perf cases

• Cross-implementation support

• Struct mapping in compile phase

• Experimental

https://github.com/wmeissner/xni




hitimes C Ext/** * call-seq: * interval.start -> boolean * * mark the start of the interval. Calling start on an already started * interval has no effect. An interval can only be started once. If the * interval is truely started +true+ is returned otherwise +false+. */VALUE hitimes_interval_start( VALUE self ){ hitimes_interval_t *i; VALUE rc = Qfalse;

Data_Get_Struct( self, hitimes_interval_t, i ); if ( 0L == i->start_instant ) { i->start_instant = hitimes_get_current_instant( ); i->stop_instant = 0L; i->duration = -1.0l;

rc = Qtrue; }

return rc;}


hitimes XNI/** * call-seq: * interval.start -> boolean * * mark the start of the interval. Calling start on an already started * interval has no effect. An interval can only be started once. If the * interval is truely started +true+ is returned otherwise +false+. */bool hitimes_interval_start( RubyEnv* env, hitimes_interval_t* i ){ if ( 0L == i->start_instant ) { i->start_instant = hitimes_get_current_instant( ); i->stop_instant = 0L; i->duration = -1.0l;

return true; }

return false;}


Startup Time


#1 Pain Point


Hard Problem

• MRI boot time is 95% native

• JRuby boot time is 0% native code

• Mostly Java, which needs to warm up

• Parser, interpreter, core classes, compiler

• Even if our code is better, we start slow


Child Processes

• Reduce need for sub-Ruby invokes

• rails -> clean rails env in child

• rails/rake -> bundler relaunch

• rake test -> 4+ processes in Rails

• rake -> rspec in subprocess

• Fix requires changing many libraries


LexicalAnalysisParsing

SemanticAnalysis

Optimization

Bytecode Generation

Interpret

AST

IR Instructions

CFG DFG ...

Existing

MORE

Dalvik Generation ...


Solutions


Nailgun/Drip

• Always running background JVM

• Not quite production quality

• signals, IO

• Small Ruby scripts very fast

• Rails not much faster

• Lots of requires, objects, boot logic


GSoC 2012IR Persistence

IR Instructions

CFG DFG ...

file.ir

file.rbcompile

file.ccompile

file.o


Reflection on GSoC

• Size matters

• # of bytes

• Intern()‘ing of identifiers matter

• Laziness can help a lot


Defined vs Used MethodsCMD DEFINED USED SAVINGS

-e ‘:foo’ 501 33 ~93%

gem install rails

1897 529 ~72%

rails scaffold 9411 1647 ~82%

rake db:migrate

9397 1662 ~82%

rake spec 4595 904 ~80%


New IR Persistence

• Binary format

• Constant pool to intern only once per id

• (currently once per occurrence)

• Incremental loading of method bodies


Ultimate Startup!

rails new fooJRuby Instance

IR Datarails generate

JRuby Instance

rake db:migrate

JRuby Instance

Compile

Use

Use

Background JVM


Ruby is Strong

• Still growing and improving

• MRI too!

• Concurrency can be done

• C extensions are holding us back

• Never surrender!


Thank You!

• Charles Oliver Nutter

• @headius

• [email protected]

• http://blog.headius.com


mailto:[email protected]

mailto:[email protected]

http://blog.headius.com

http://blog.headius.com

the future of jruby - baruco 2013

Technology

truely started

source location

struct layout

dalvik generation

iteration

13 solutions

spain tuesday

calling start