devoxx uk 2014 high performance in-memory java with open source

@dbrimley #DevoxxHazelcast

High Performance In-Memory Java with Open Source

David Brimley @dbrimley http://blog.hazelcast.com/ http://www.hazelcast.org/


Who is this guy?

•  Senior Solutions Architect for Hazelcast.

•  17 Years Java Development.

•  Started out as a Cobol Programmer in 1988.

•  Learnt to code Basic on a Vic-20 and BBC Model B.

•  Oracle Coherence and Pivotal Gemfire.


In Memory Java

•  High Performance In Memory Java? It’s an IMDG!

•  Getting to grips with Hazelcast, a tour of features.

•  Some IMDG Tips!


In Memory Java

•  It’s 2 things really, the 2 C.

•  Cache

•  Compute

•  The Magic is the D you stick in front…Distributed!

•  DCache + DCompute = IMDG!


Evolution of IMDG

CACHE IN ONE PROCESS - A MAP, AGGHH OOM!


Evolution of IMDG

CACHE IN ONE PROCESS - WITH EVICTION


Evolution of IMDG

DISTRIBUTED CACHE


Evolution of IMDG

BIG DATA IMDG


Qualities of an IMDG

•  It should be easy to scale, vertically and horizontally.

•  It should be fault tolerant.

•  It should be performant.

•  It should have a easy to understand API.


Enter Hazelcast! •  Java Collections & Queues - Distributed!

•  Java Concurrency - Distributed!

•  Persist to and Read from from Anything! (RDBMS,File,Hadoop,Cassandra,MongoDB)

•  Predicate and SQL like queries!

•  Topics(Pub/Sub)

•  Rich Event Callbacks


<dependency> <groupId>com.hazelcast</groupId> <artifactId>hazelcast</artifactId> <version>3.2.2</version> </dependency>

1. Starting a Hazelcast Cluster Node


HazelcastInstance hz = Hazelcast.newHazelcastInstance();

2. Starting a Hazelcast Cluster Node

@YourTwitterHandle @dbrimley #DevoxxHazelcast


Distributed Compute •  Executors

•  EntryProcessor

•  MapReduce


Executor can run anywhere •  IExecutorService

•  Runnable or Callable

•  Targeted execution in the grid (keys,members)


HazelcastInstance hz = Hazelcast.newHazelcastInstance(); IExecutorService executor = hz.getExecutorService(“myEService"); executor.execute(new EchoTask("foo"));

2. Java Executor, but Distributed!


EntryProcessor •  Execute your code on an entry in an atomic way.

•  You do not need any explicit lock on entry.

•  Runs via IMap.


/** * Applies the user defined EntryProcessor to the entry mapped by the key. * Returns the the object which is result of the process() method of EntryProcessor. */ Object executeOnKey(K key, EntryProcessor entryProcessor);

EntryProcessor via IMap


MapReduce •  New in 3.2

•  Hadoop like API.

•  Aggregators in 3.3

•  Scheduled Tasks in 3.4


IMap<String,String> map = hazelcastInstance.getMap("articles"); KeyValueSource<String,String> source = KeyValueSource.fromMap(map); Job<String, String> job = jobTracker.newJob( source ); ICompletableFuture<Map<String, Long>> future = job .mapper( new TokenizerMapper() ) .combiner( new WordCountCombinerFactory() ) .reducer( new WordCountReducerFactory() ) .submit();

MapReduce


Persist and Read from anywhere. •  Implement a MapStore interface.

•  Configure your Map.

•  Cache misses will read through to the store.

•  Cache updates can persist to store (sync/async).


<map name="aDistributedMap"> <map-store enabled="true"> <class-name>com.company.YourMapStore</class-name> <write-delay-seconds>30</write-delay-seconds> </map-store> <indexes> <index ordered="true">aValueProperty</index> </indexes> </map>

2. Configure (in XML,Spring or API)


public class UserMapStore implements MapStore<Integer, User> { public void store(Integer userKey, User user) { userDao.insertUser(user); } public User load(Integer userKey) { return userDao.getUser(userKey); } …… (more methods like delete, loadAll)

2. Implement MapStore


We like standards. •  Greg Luck, Hazelcast CTO is JSR107 co-spec lead.

•  Hazelcast will be JSR107 compliant this summer.

•  https://github.com/hazelcast/hazelcast/tree/jcache-preview

@YourTwitterHandle @dbrimley #DevoxxHazelcast


Some IMDG Advice •  Don’t force in your RDBMS schema.

•  Do you really need that transaction?

•  Execute near the data.

•  Use the right Serialisation interface.

•  Tune your JVM.

@YourTwitterHandle #DVXFR14{session hashtag} @dbrimley #DevoxxHazelcast


Powerful tool for many Use Cases •  Unscalable NoSQL or RDBMS fix.

•  Compute Intensive Tasks

•  Web Session Replication

•  HA Services

•  WAN Replication


Get Coding! •  hazelcast.org - for open source and downloads

•  hazelcast.com - for pro support,papers & webinars

•  github.com/hazelcast for sources

•  github.com/hazelcast/hazelcast-code-samples

•  follow me on twitter @dbrimley

@YourTwitterHandle #DVXFR14{session hashtag} @dbrimley #DevoxxHazelcast


Thanks / Creative Commons

•  Presentation Template — Guillaume LaForge •  The Queen — A prestigious heritage with some

inspiration from The Sex Pistols and funny Devoxxians •  Girl with a Balloon — Banksy

•  Tube — Michael Keen

devoxx uk 2014 high performance in-memory java with open source

Software

olution of imdg cache

hazelcast cluster

olution of imdg big

imdg tips

ter hazelcast

cache misses

store cache updates

apreduce new