chasing the elephant

26
@iconara CHASING THE ELEPHANT

Upload: iconara

Post on 01-Nov-2014

933 views

Category:

Technology


2 download

DESCRIPTION

JRubyConfEU 2013

TRANSCRIPT

Page 1: Chasing the elephant

@iconara

CHASING THE ELEPHANT

Page 2: Chasing the elephant

Theo / @iconara

Page 3: Chasing the elephant

chief architect at BURT

Page 4: Chasing the elephant

big data analytics with JRuby

Page 5: Chasing the elephant

RUBY

Page 6: Chasing the elephant

RUBY

Page 7: Chasing the elephant

RUBY JRUBY

Page 8: Chasing the elephant

JRUBY IS AWESOMEBECAUSE RUBY IS GREAT,

AND THE JVM IS GREAT

Page 9: Chasing the elephant

hot_bunnies, eurydice, multimeter, mikka, msgpack-jruby

Page 10: Chasing the elephant
Page 11: Chasing the elephant

HADOOP

Page 12: Chasing the elephant

JRUBY SUPERCOMPUTING40 TiB data, 120 EC2 cc2.8xlarge, 1920 cores, 7260 GiB RAM

Page 13: Chasing the elephant

JAVA ALL THE WAY DOWN

Page 14: Chasing the elephant

HADOOP STREAMINGWukong, Dumbo

Page 15: Chasing the elephant

RUBYDOOPrubydoop.org

+ =

Page 16: Chasing the elephant
Page 17: Chasing the elephant
Page 18: Chasing the elephant

main()

Page 19: Chasing the elephant

Class.forName("...")

Page 20: Chasing the elephant

Class.forName("...")

Page 21: Chasing the elephant
Page 22: Chasing the elephant

module WordCount class Mapper def map(key, value, context) value.to_s.downcase.split.each do |word| key = Hadoop::Io::Text.new(word) value = Hadoop::Io::IntWritable.new(1) context.write(key, value) end end endend

Page 23: Chasing the elephant

RUBYDOOP IS LOW LEVELI would love to see someone write something

like Scalding or Cascading on top of it

Page 24: Chasing the elephant

RUBYDOOPrubydoop.org

Page 25: Chasing the elephant

RUBYDOOPrubydoop.org

v1.1.0

Page 26: Chasing the elephant

KTHXBAI@iconara

github.com/iconaraarchitecturalatrocities.com

burtcorp.com