mario donkey kong · distribution, distances, concurrency, commodity, scale: all these are reasons...
TRANSCRIPT
MarioMariovs.
Donkey Kong
…oh, Pardon…
Erlang/OTPvs.
Enterprise Java
PavloBaron
Why am I talking Why am I talking about it
Do you think it’s about heart?
Or is there another reason?
I DON‘TLIKELIKE
JAVA!
The world of IT, as we knew it, has knew it, has changed…
Instead of huge cabinets…
…we can now use lots of cheap commodity hardware
Physics has hit the wall…
…and we must think parallel
Our physically Our physically huge globe…
s
…became a tiny electronic ball. It‘s completely wired
Spontanuous requirements…
…can be covered by the fog (aka cloud)
So what?
We must accept some things…
Functional languages much better suit parallel programming
Extrem scenarios require require
technologies allowing extrem scalability – no scalability – no
forced standards
Application types have pretty much changed
Data became more different, more different, bigger, more fine-grained and more and more frequent
High availability High availability counts for many among must-have parameters
Distribution, distances, concurrency,concurrency,commodity,scale: all these are reasons why parts reasons why parts of a system can anytime crash
Share-nothing approaches scale best
Tell me something new –
Enterprise Java Enterprise Java solves it all. Where does Erlang/OTP Erlang/OTP
come into play?
Exactly!Let‘s take a Let‘s take a look at it
Language
List<Integer> l = Arrays.asList(1, 2, 3, 4, 5);
Java: list processing
List<Integer> l = Arrays.asList(1, 2, 3, 4, 5);List<Integer> r = new ArrayList<Integer>();for (int i : l) {
if (i % 2 == 0) {r.add(i);
}}}
Erlang: list comprehensions
[X || X <- lists:seq(1, 5), X rem 2 == 0].
Where is more boilerplate?boilerplate?
synchronized (this) {if (!crawledSites.contains(site)) {
linkedSites.add(site);}
}
…
Java:„concurrency“
…
public class CountingSemaphore {private int signals = 0;public synchronized void take() {
this.signals++;this.notify();
}
„concurrency“
public synchronized void release()throws InterruptedException {
while (this.signals == 0) wait();this.signals--;
}}
Java concurrency: like two boxers alternating the gloves, thus hitting sequentially
Java concurrency: like a Java concurrency: like a goalkeeper telling the forwards to wait while he takes a pull on a battle
Erlang: functional
Erlang: single assignment
Erlang: single assignment
1> A = 5.52> A = 10.** exception error: no match of right hand side value 103>
Enterprise Java: why not pick „more modern“ weapons?..
Erlang: Share-nothing
-record(state, {start,player1,
Erlang: message passing with state externalization
player1,player2
}).
..
State = #state{start=0}Pid ! {self(), State},Pid ! {self(), State},
receive{Pid, NewState} ->
…end,
import kilim.Mailbox;import kilim.Pausable;import kilim.Task;
public class SimpleTask extends Task {static Mailbox<String> mb = new Mailbox<String>();
Java: „actor model“ This package comes with a bytecode transformation tool called Weaver (package: kilim.tools.Weaver) that post-processes .class fileslooking for the "throws Pausable" annotation.
When a task needs to pause, it
public static void main(String[] args) throws Exception {new SimpleTask().start();Thread.sleep(10);mb.putnb("Hello ");mb.putnb("World\n");mb.putnb("done");
}
public void execute() throws Pausable {while (true) {
When a task needs to pause, it unwinds its stack, squirrels away all state that it'll need later on resumption. This unwinding and rewinding the stack is automatically performed by the code introduced by the Weaver.(Debug information is adjusted so that the transformed code can be debugged inside eclipse) This is identical to what a
while (true) {String s = mb.get();if (s.equals("done")) break;
System.out.print(s);}
System.exit(0);}
}
is identical to what a programmer would havewritten
Erlang: actor based
register(serv, spawn(?MODULE, loop, []))serv ! {self(), “Hello there!”},receive
Spawn, loop, register und message passing
receive{_Pid, Msg} ->
…end.
…
loop() ->receive
{From, Txt}…loop();
So what?In Java, I have
standards for standards for everything. And parallel systems I can solve with I can solve with Scala and Akka
Exactly!Let‘s take a look at this as look at this as well
Java: is it really the ultimate ultimate answer to everything?
Enterprise Java: tries to standardize the half world
This somehow reminds of…
When Enterprise Java?
When not Enterprise Java?
Enterprise Java: suffers Swiss Army knife syndrome
Java: application orientedoriented
Java: especially concurrency and low level technology have generally never been to the fore
Erlang/OTP: strong specialization
Erlang: technology rather than application oriented. „Massive parallel“ from the beginning
-define(IP_VERSION, 4).
Erlang: example of bitstring processing
-define(IP_VERSION, 4).-define(IP_MIN_HDR_LEN, 5).
DgramSize = byte_size(Dgram),case Dgram of
<<?IP_VERSION:4, HLen:4, SrvcType:8, TotLen:16, ID:16, Flgs:3, FragOff:13,TTL:8, Proto:8, HdrChkSum:16,SrcIP:32,DestIP:32, RestDgram/binary>> when HLen>=5, 4*HLen=<DgramSize ->DestIP:32, RestDgram/binary>> when HLen>=5, 4*HLen=<DgramSize ->OptsLen = 4*(HLen - ?IP_MIN_HDR_LEN),<<Opts:OptsLen/binary,Data/binary>> = RestDgram,
...end.
When Erlang/OTP?
Network components controlNetwork components controlExtreme servers
Messaging middlewareDistributed systems
Systems for big data processingBulk calculation systemsBulk calculation systems
Critical systemsEvent control and processing
Bitstream processing
When not Erlang/OTP?
UI-heavy applications
Application / business logic
Number crunchingNumber crunching
Scala devotes itself primarily to… exactly, Scala!
Scala doesn’t have an own VM und lives off the JVM
JVM is by far not established in every IT in the world
Scala‘s basis code inherits Java diseasesinherits Java diseases
Akka development concentrates primarily on Scala
Scala and Akka took over lots of Erlang concepts ☺
Doesn’t matter!Enterprise Java platform is platform is proved, reliable, fault tolerantand optimal and optimal suitable for all situations!
Exactly!Let‘s take a look at this, look at this, too
Java: fault tolerancepublic static void main(String[] args) {
PathDasher dasher = new PathDasher(null); }
…
# A fatal error has been detected by the Java Runtime Environment:## EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006da97783, pid=4772, tid=3416### JRE version: 6.0_15-b03# Java VM: Java HotSpot(TM) 64-Bit Server VM (14.1-b02 mixed mode windows-amd64 )# Problematic frame:# V [jvm.dll+0x207783]
Java: JNI crash will crash the whole JVM
Erlang: fault tolerance
1> self().<0.31.0>
Erlang: fault tolerance
<0.31.0>2> 5/0.** exception error: bad argument in an arithmetic expression
in operator '/'/2called as 5 / 0called as 5 / 0
3> self().<0.34.0>4>
Erlang:supervisor trees
-module(dr_supervisor).-behaviour(supervisor).
Erlang: supervisor tree
-behaviour(supervisor).-export([start_link/1]).-export([init/1]).
start_link(Args) ->supervisor:start_link({local, ?MODULE}, ?MODULE, Args).
init(Args) ->{ok, {{one_for_one, 2, 10}, [{ok, {{one_for_one, 2, 10}, [
{session, {dr_session, start_link, [Args]}, transient, 2000, supervisor, [dr_session]}]}}.
-module(dr_session).-behaviour(supervisor).
-export([start_link/1]).
Erlang: supervisor tree
-export([start_link/1]).-export([init/1]).
start_link(Args) ->supervisor:start_link({local, ?MODULE}, ?MODULE, Args).
init(Args) ->error_logger:info_report("session started"),{ok, {{one_for_one, 2, 10}, [{ok, {{one_for_one, 2, 10}, [{server, {dr_session_server, start_link, [Args]}, transient, 2000, worker,
[dr_session_server]},{state, {dr_session_state, start_link, [Args]}, transient, 2000, worker,
[dr_session_state]},{vm, {dr_session_vm, start_link, [Args]}, transient, 2000, worker,
[dr_session_vm]}]}}.
import org.gridgain.grid.*;import org.gridgain.grid.gridify.*;import org.gridgain.grid.gridify.aop.spring.*;
public final class GridifyHelloWorldSessionExample {private GridifyHelloWorldSessionExample() { //ensure singleton}
@Gridify(taskClass = GridifyHelloWorldSessionTask.class, timeout = 3000)public static int sayIt(String phrase) {
System.out.println(phrase);
Java: distribution
System.out.println(phrase);return phrase.length();
}
public static void main(String[] args) throws GridException {if (args.length == 0) {
GridFactory.start();}else {
GridFactory.start(args[0]);}
To use other AOP implementations (such as JBoss AOP, or Spring AOP), refer to AOP Configuration documentation.The following configuration needs to be applied to enable AspectJ byte code weaving.JVM configuration should include:
try {
int phraseLen = sayIt("Hello World");
System.out.println(„number of characters is '" + phraseLen + "'.");}finally {
GridFactory.stop(true);}
}}
-javaagent:[GRIDGAIN_HOME]/libs/aspectjweaver-1.5.3.jar
Erlang: distributed
Erlang: example of multicall for code reload
%% Find object code for module Mod
{Mod, Bin, File} = code:get_object_code(Mod),
%% and load it on all nodes including this one
{ResL, _} = rpc:multicall(code, load_binary, [Mod, Bin, File,], 5000),{ResL, _} = rpc:multicall(code, load_binary, [Mod, Bin, File,], 5000),
%% and then maybe check the ResL list.
QueueConnectionFactory connFactory = new QueueConnectionFactory();QueueConnection conn = connFactory.createQueueConnection();QueueSession session = conn.createQueueSession(false, Session.AUTO_ACKNOWLEDGE);Queue q = new Queue("world");
Java: message exchange
Queue q = new Queue("world");QueueSender sender = session.createSender(q);TextMessage msg = session.createTextMessage();msg.setText("Hello there!");sender.send(msg);
QueueReceiver receiver = session.createReceiver(q);conn.start();Message m = receiver.receive();if (m instanceof TextMessage) {
TextMessage txt = (TextMessage) m;TextMessage txt = (TextMessage) m;}
session.close();conn.close();
Boilerplate?Even with Spring still symptom still symptom treatment
Erlang: message oriented
Erlang: message passing
Pid ! {self(), “Hello there!”},receive
{Pid, Txt} ->…
end,
VM
JVM threads are too “heavy”. Context switching is extremely expensive to have many thousands of them. Only hacks thousands of them. Only hacks such as CPS simulation over byte code manipulation provide a remedy und illusion of a remedy und illusion of lightweight threads
import kilim.Mailbox;import kilim.Pausable;import kilim.Task;
public class SimpleTask extends Task {static Mailbox<String> mb = new Mailbox<String>();
Java: „concurrency“This package comes with a bytecode transformation tool called Weaver (package: kilim.tools.Weaver) that post-processes .class fileslooking for the "throws
public static void main(String[] args) throws Exception {new SimpleTask().start();Thread.sleep(10);mb.putnb("Hello ");mb.putnb("World\n");mb.putnb("done");
}
public void execute() throws Pausable {while (true) {
looking for the "throws Pausable" annotation.
When a task needs to pause, it unwinds its stack, squirrels away all state that it'll need later on resumption. This unwinding and rewinding the stack is automatically performed by the code introduced by the Weaver.(Debug information is adjusted while (true) {
String s = mb.get();if (s.equals("done")) break;
System.out.print(s);}
System.exit(0);}
}
(Debug information is adjusted so that the transformed code can be debugged inside eclipse) This is identical to what a programmer would havewritten
Java: instance-wide garbage collector
Erlang: concurrent.Hunderts of Hunderts of thousends of lightweight VM processes per node processes per node instead of „native“ threads-> own scheduler
Erlang: heap per process -> garbage collected per process
Hot swapping
Java: ClassLoadertrees can be trees can be complex and unpredictable.Middlewareoften causesoften causeslib conflicts
Java: OSGi is invasive and imports a middleware
Java: app server fatigue is a common phenomenon after
frequent application redeploymentredeployment
Java: zero downtime systems are possible only with human support
% hot_swap:sum(Num) addiert 1 dazu1> c(hot_swap).{ok,hot_swap}
Erlang: live code upgrade
{ok,hot_swap}2> hot_swap:sum(1).2...%hot_swap.erl wurde modifiziert. Jetzt% addiert es 2 dazu……3> c(hot_swap).{ok,hot_swap}4> hot_swap:sum(1).35>
Erlang: non-stop systems arepossible and implemented
Come on!Come on!Enterprise Java offers even more scalabilityscalability
Exactly!This should also be also be considered
Java doesn‘t scale as originally expected
Hacks
Scala & Co.
Java
EnterpriseJava
Erlang: scaling
From sequential program to world-wide distribution
MulticoreProgramming
DistributedProgramming
SequentialErlang
FunctionalErlang
Frameworks
Java: Framework-itis(like others, too ☺)
Erlang: generic behaviors
Erlang/OTP: behaviors
-module(dr_session_state).-behaviour(gen_fsm).-behaviour(gen_fsm).
-export([start_link/1]).-export([init/1]).
start_link(Args) ->gen_fsm:start_link({local, ?MODULE}, ?MODULE, Args, []).
init(_Args) ->init(_Args) ->error_logger:info_report("state started"),{ok, waiting1, []}.
Erlang/OTP:behaviors
-module(dr_session_server).-behaviour(gen_server).
-export([start_link/1]).-export([init/1]).-export([code_change/3]).-export([handle_call/3]).-export([handle_cast/2]).-export([handle_cast/2]).-export([handle_info/2]).-export([terminate/2]).
-record(state, {start,player1,player2
}).
start_link(Args) ->gen_server:start_link({local, ?MODULE}, ?MODULE, [Args], []).
init(_Args) ->error_logger:info_report("server started"),process_flag(trap_exit, true),{ok, #state{}}.
…
Erlang/OTP: behaviors
-module(dr_gen_vm).-export([behaviour_info/1]).
behaviour_info(callbacks) ->[{init,1}];
behaviour_info(_Other) ->undefined.undefined.
NoSQL DSs made in Java don‘t implement Java standards for data access because they are impractical, and use open impractical, and use open standards instead
NoSQL DSs made in Erlangreceive many things as a gift…
Erlang: big data storage possibilities
Erlang: Dets = Disk Erlang Term Storage
{_,D} = dets:open_file(„example“), dets:delete_all_objects(D),dets:insert(D, {mama, mia}),dets:insert(D, {here, we}),dets:insert(D, {go, again}),dets:insert(D, {go, again}),dets:close(D),
Erlang: Mnesia = dets + transactions, quiries and
distributionmnesia:create_schema([node()]),mnesia:start(),mnesia:start(),mnesia:create_table(album,
[ {disc_copies, [node()] },{attributes,
record_info(fields, album)} ]),Insert = fun() ->
mnesia:write(#album{ index=A_Index,#album{ index=A_Index,
artist=A_Artist,title=A_Title } ) end,
{atomic, Result} = mnesia:transaction(Insert),…
Really? ButEnterprise Java Enterprise Java integrates everything in the world!world!
Exactly!Let‘s see…Let‘s see…
Java prefers to integrate non-natively, from afar –natively, from afar –for example per web service. In a pinch JNI will do –at own risk.at own risk.
Erlang: integrated
Erlang: ways to integration
Erlang node Pseudo-OTP
Erlang program
PortPort
OTPnode with the externalprogram
Binary data
Linked-inDriver
Externalprogram
Binary data
Erlang: example JInterfaceimport com.ericsson.otp.erlang.*;
...
OtpNode self = new OtpNode("pseudo@test");OtpNode self = new OtpNode("pseudo@test");OtpMbox mbox = self.createMbox("pingpong");OtpErlangObject o;OtpErlangTuple msg;OtpErlangPid from;
while (true) {try {o = mbox.receive();if (o instanceof OtpErlangTuple) {
msg = (OtpErlangTuple)o;msg = (OtpErlangTuple)o;from = (OtpErlangPid)(msg.elementAt(0));mbox.send(from, msg.elementAt(1));
}catch (Exception e) {...
}}
Erlang: example JInterface
-module(pongclient).-export([pong/0]).
pong() ->{pingpong, "pseudo@test“} ! {self(), "ball"},receive
Result ->io:format("What comes back is : ~p~n", [Result])
end.
And even I can do web services these days ☺
Stop!Stop!I am constrained to the JVM!Period.Period.
BEAM is not an option for every IT
Erjang – Erlang on the JVM. It can already
boot some big Erlangsystems. It’s under systems. It’s under
development.I do help ☺
Right tool for the job
Thank you