on heap cache vs off-heap cache

39
On-heap cache vs Off-heap cache Radek Grębski @RadekGrebski https://github.com/rgrebski

Upload: rgrebski

Post on 15-Jan-2017

1.347 views

Category:

Software


6 download

TRANSCRIPT

Page 1: On heap cache vs off-heap cache

On-heap cache vs Off-heap cacheRadek Grębski

@RadekGrebskihttps://github.com/rgrebski

Page 2: On heap cache vs off-heap cache

PRESENTATION PLAN

1. Heap vs off-heap2. Heap memory3. Off-heap memory

3.1 Memory mapped file3.2 Unsafe and ByteBuffers3.3 Off-heap memory advantages

4. Cache:4.1 Chronicle4.2 Hazelcast4.3 Redis

5. Comparison

Page 3: On heap cache vs off-heap cache

1. HEAP VS OFF-HEAP

vs

Page 4: On heap cache vs off-heap cache

2. HEAP MEMORY

JVM memoryJDK8

S1Eden S0 Old Generation

Heap (-Xmx)

Metaspace(-XX:MaxMetaspaceSize)

Native memory

JVM Process memory

OS memory

Young Generation

Page 5: On heap cache vs off-heap cache

Objects on heap

Source: http://www.ibm.com/developerworks/library/j-codetoheap/

Integer (64-bit JVM): 7:1 (28 bytes)

9:1 (36 bytes)

Integer (32-bit JVM):

3:1 overhead ratio

16 bytes

String (32-bit JVM):

3.75:1

60 bytes

int[1]

2. HEAP MEMORY

Page 6: On heap cache vs off-heap cache

3. OFF-HEAP MEMORY

Off-heap (native) memoryJVM Process

OS memory

Other processes / unallocated

byte[]

byte[]

Page 7: On heap cache vs off-heap cache

3.1. MEMORY MAPPED FILE

Off-heap memoryJVM Process

OS memory

JVM Process

/tmp/myFile.dat

byte[]

byte[] byte[]

byte[]

Page 8: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

How to allocate memory using standard Java classes?

� java.nio.ByteBuffer:� HeapByteBuffer (on-heap, up to 2gb)� DirectByteBuffer (off-heap, up to 2gb)� MappedByteBuffer (off-heap, up to 2gb, persisted)

� sun.misc.Unsafe� public native long allocateMemory(long allocationSizeInBytes);

Page 9: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

ByteBuffer.allocate ( <2GB )

JvmUtils.verifyJvmArgumentsPresent("-Xmx2g");

ByteBuffer byteBuffer = ByteBuffer.allocate((int) ByteUtil.GB);

byteBuffer.putChar('a') //2bytes, position =0

.putInt(123) //4bytes, position = 2(0 + 2(char))

.put("test".getBytes("UTF-8")); //6 => 2(char) + 4(integer)

byte[] bytesToBeReadInto = new byte["test".getBytes("UTF-8").length];

char charA = byteBuffer.getChar(/*address*/ 0); // 'a'

int int123 = byteBuffer.getInt(/*address*/ 2); //123

byteBuffer.position(6); //set cursor position

byteBuffer.get(bytesToBeReadInto); //"test" as byte[] read into "bytesToBeReadIntoRead"

Page 10: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

ByteBuffer.allocateDirect (off-heap)

JvmUtils.verifyJvmArgumentsPresent(of("-Xmx64m", "-XX:MaxDirectMemorySize=2g"));

long capacity2GB = (2 * ByteUtil.GB) - 1;

assertThat(capacity2GB).isLessThanOrEqualTo(Integer.MAX_VALUE);

//after this line process memory grows up to 2GB+

ByteBuffer byteBuffer = ByteBuffer.allocateDirect((int) capacity2GB)

.putChar('a') //2bytes, position =0

.putInt(123); //4bytes, position = 2(0 + 2(char))

char charA = byteBuffer.getChar(/*address*/ 0); // 'a'

int int123 = byteBuffer.getInt(/*address*/ 2); //123

//perform "GC" on created byte buffer (

ByteBufferUtils.callCleaner(byteBuffer);

byteBuffer.putInt(1);

// # A fatal error has been detected by the Java Runtime Environment:

// #

// # EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x000000006fca432d, pid=9356, tid=11152

Page 11: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

Unallocating ByteBuffer memory

public static void callCleaner(ByteBuffer byteBuffer){

//DirectByteBuffer.cleaner().clean()

Method cleanerMethod = byteBuffer.getClass().getMethod("cleaner");

cleanerMethod.setAccessible(true);

Object cleaner = cleanerMethod.invoke(byteBuffer);

Method cleanMethod = cleaner.getClass().getMethod("clean");

cleanMethod.setAccessible(true);

cleanMethod.invoke(cleaner);

}

Page 12: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

Allocating memory using sun.misc.Unsafe

sun.misc.Unsafe::getUnsafe():

public static Unsafe getUnsafe() {

Class cc = sun.reflect.Reflection.getCallerClass(2);

if (cc.getClassLoader() != null)

throw new SecurityException("Unsafe");

return theUnsafe;

}

Creating an instance of Unsafe:

public static Unsafe createUnsafe() {

Constructor<Unsafe> unsafeConstructor = Unsafe.class.getDeclaredConstructor();

unsafeConstructor.setAccessible(true);

return unsafeConstructor.newInstance();}

Memory allocation:

long memorySizeInBytes = ByteUtil.GB * 4;

long startAddress = unsafe.allocateMemory(memorySizeInBytes);

unsafe.setMemory(startAddress, memorySizeInBytes, (byte) 0);

unsafe.freeMemory(startAddress);

Page 13: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

Memory mapped file using Java API

File mappedFile = new File("/tmp/mappedFile.tmp");

mappedFile.delete();

try (FileChannel fileChannel = new RandomAccessFile(mappedFile, "rw").getChannel()) {

long buffer8MB = 8 * ByteUtil.MB;

MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, 0,

buffer8MB);

long startAddress = 0;

long elementsToPut = 200_000_000;

for (long counter = 0; counter < elementsToPut; counter++) {

if (!mappedByteBuffer.hasRemaining()) {

startAddress += mappedByteBuffer.position();

mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, startAddress,

buffer8MB);

}

mappedByteBuffer.putLong(counter);

}

}Time: 1,068 sFilesize: 1,49 GB

Page 14: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

Memory mapped file content

Page 15: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

Big endian vs Little endian

0xCAFEBABE

Address 00 01 02 03

Big endian CA FE BA BE

Address 00 01 02 03

Little endian BE BA FE CA

Page 16: On heap cache vs off-heap cache

3.2. UNSAFE AND BYTEBUFFERS

ByteBuffer endianess

@Test

public void testEndianess() throws NoSuchMethodException, IllegalAccessException,

InvocationTargetException {

ByteBuffer bigEndianByteBuffer = ByteBuffer.allocate(4);

bigEndianByteBuffer.order(ByteOrder.BIG_ENDIAN);

ByteBuffer littleEndianByteBuffer = ByteBuffer.allocate(4);

littleEndianByteBuffer.order(ByteOrder.LITTLE_ENDIAN);

littleEndianByteBuffer.putInt(0xCAFEBABE);

bigEndianByteBuffer.putInt(0xCAFEBABE);

String bigEndianHexString = Hex.encodeHexString(bigEndianByteBuffer.array());

String littleEndianHexString = Hex.encodeHexString(littleEndianByteBuffer.array());

assertThat(bigEndianHexString).isEqualToIgnoringCase("CAFEBABE");

assertThat(littleEndianHexString).isEqualToIgnoringCase("BEBAFECA");

}

Page 17: On heap cache vs off-heap cache

3.3. OFF-HEAP MEMORY ADVANTAGES

Off-heap storagePros and cons

� No GC!� Manual GC!� Memory leaks� Persistence� IPC� Latency� Durability� Scalability (1TB+)

Page 18: On heap cache vs off-heap cache

4. CACHE

� HashMap� Collections.synchronizedMap(..)� ConcurrentHashMap

Page 19: On heap cache vs off-heap cache

4.1. CHRONICLE

http://chronicle.software

Chronicle Map Chronicle Set Chronicle Queue

Chronicle Logger Java Thread Affinity

Page 20: On heap cache vs off-heap cache

Features

� No GC!� No low-level coding� Persistence� Shared between JVMs� Replication� Loading of Chronicle Map ~10ms � Concurrent across processes� Synchronization is on CPU level� Uses proxy� Zero-copy – getUsing(key, existingValueObject)� close()

4.1. CHRONICLE MAP

Page 21: On heap cache vs off-heap cache

4.1. CHRONICLE MAP

Creating off-heap map instance

private Map<String, OffHeapUser> chronicleMap;

private static final String KEY_SAMPLE = "12345678901234567890";

private static final long MAX_ENTRIES = 10_000_000;

{

chronicleMap = ChronicleMapBuilder.of(String.class, OffHeapUser.class)

.averageKeySize(KEY_SAMPLE.getBytes("UTF-8").length)

.constantValueSizeBySample(new OffHeapUserSample())

.createPersistedTo(new File("/tmp/mappedFile.bin"))

.entries((long) (MAX_ENTRIES))

.create();

}

Page 22: On heap cache vs off-heap cache

4.1. CHRONICLE MAP

Complex structures

public interface OffHeapUser {

String getUsername();

void setUsername(@MaxSize(30) String username);

long getAccountValidUntil();

void setAccountValidUntil(long accountValidUntil);

void setRoleAt(@MaxSize(2) int index, Role role);

Role getRoleAt(@MaxSize(2) int index);

static interface Role {

String getRole();

void setRole(@MaxSize(10) String role);

}

}

Page 23: On heap cache vs off-heap cache

4.1. CHRONICLE MAP

Creating value instance

//fill the data

long accountValidUntil = System.currentTimeMillis() + YEAR_IN_MILLIS;

String username = RandomStringUtils.randomAlphabetic(20);

OffHeapUser offHeapUser = chronicleMap.newValueInstance();

offHeapUser.setAccountValidUntil(accountValidUntil);

offHeapUser.setUsername(username);

OffHeapUser.Role role0 = offHeapUser.getRoleAt(0);

role0.setRole("Role0");

OffHeapUser.Role role1 = offHeapUser.getRoleAt(1);

role0.setRole("Role1");

//put

chronicleMap.put("someKey", offHeapUser);

//get

OffHeapUser offHeapUserActual = chronicleMap.get("someKey");

Page 24: On heap cache vs off-heap cache

4.1. CHRONICLE MAP

Throughput

Key = „u:0123456789”, value = counter

*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map

0

20

40

60

80

100

120

140

160

180

10 000 000 50 000 000 250 000 000 1 250 000 000

Th

rou

ghp

ut

Mu

pd

/s

Map entries

Throughput - ChronicleMap vs ConcurrentHashMap

Cronicle Map

ConcurrentHashMap

OutOfMemory

Page 25: On heap cache vs off-heap cache

4.1. CHRONICLE MAP

Memory usage

OutOfMemory

0,0

20,0

40,0

60,0

80,0

100,0

120,0

140,0

10 000 000 50 000 000 250 000 000 1 250 000 000

Mem

ory

in G

B

Map entries

Memory used - ChronicleMap vs ConcurrentHashMap

Cronicle Map

ConcurrentHashMap

Key = „u:0123456789”, value = counter

*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map

Page 26: On heap cache vs off-heap cache

4.1. CHRONICLE MAP

GC pauses

0,0

5,0

10,0

15,0

20,0

25,0

30,0

35,0

40,0

45,0

50,0

10 000 000 50 000 000 250 000 000 1 250 000 000

Wo

rst

GC

pa

use

in

se

con

ds

Map entries

Worst GC pause [s] - ChronicleMap vs ConcurrentHashMap

Cronicle Map

ConcurrentHashMap

OutOfMemory

Key = „u:0123456789”, value = counter

*ChronicleMap was tested with a 32 MB heap, CHM was test with a 100 GB heap.Source: https://github.com/OpenHFT/Chronicle-Map

Page 27: On heap cache vs off-heap cache

4.1. CHRONICLE

Chronicle Map250M entries comparison

Source: https://github.com/OpenHFT/Chronicle-Map

250 Million entries ConcurrentHashMap ChronicleMap

Throughput 120million updates/s 30 million updates/s

Worst case latency 17s 160µs (avg 0,3µs)

Throughput persisted NA 28 million updates/s

Maximum entries(128 GB)

400 million 2500 million(32mb heap)

Page 28: On heap cache vs off-heap cache

4.2. HAZELCAST

� On-heap cache� Off-heap support (High Density Memory - commercial)� List, Map, Set, Queue� Topics� Executor Service� Dynamic clustering� Transactional

Page 29: On heap cache vs off-heap cache

4.2. HAZELCAST

Example

@Test

public void hazelcastClusterTest(){

Config hazelcastConfig = new Config();

HazelcastInstance hazelcastInstance1 = Hazelcast.newHazelcastInstance(hazelcastConfig);

HazelcastInstance hazelcastInstance2 = Hazelcast.newHazelcastInstance(hazelcastConfig);

Map<String, String> node1Map = hazelcastInstance1.getMap("someMapName");

Map<String, String> node2Map = hazelcastInstance2.getMap("someMapName");

node1Map.put("key", "value");

Assertions.assertThat(node2Map.get("key")).isEqualTo("value");

}

Output:

Members [2] {

Member [192.168.1.23]:5701 this

Member [192.168.1.23]:5702

}

Page 30: On heap cache vs off-heap cache

4.3. REDIS

� REmote DIctionary Server� Key/Value cache + store� ANSI C� Used by:

� StackOverflow� GitHub� Twitter� Instagram� Alibaba

� Clients for almost all programming languages

Page 31: On heap cache vs off-heap cache

4.3. REDIS

Redis data types

� HashMaps (hset, hget)� LinkedList (lpush, ltrim, lrange)� Queue (lpush, rpop, brpop)� Topics (publish, subscribe)� Set (sadd, srem)� SortedSet (zadd, zrem)

Page 32: On heap cache vs off-heap cache

4.3. REDIS

Redis Java clients

� Jedis

� Redisson

� Aredis� JDBC-Redis� Jredis� Lettuce� RedisClient

Page 33: On heap cache vs off-heap cache

4.3. REDIS

Redis HashMap example

@Test

public void testRedisMap(){Jedis jedis = new Jedis("localhost");

// Pipeline pipeline = jedis.pipelined();

// pipeline.multi();

jedis.hset(/*map name*/ "user:1", /*key*/ "firstName", /*value*/"Radek");

jedis.hset("user:1", "lastName", "Grebski");

jedis.hset("user:1", "email", "[email protected]");

// pipeline.sync()

Map<String, String> mapFromRedis = jedis.hgetAll("user:1");

assertThat(jedis.hget("user:1", "firstName")).isEqualTo("Radek");

assertThat(mapFromRedis)

.hasSize(3)

.contains(entry("firstName", "Radek"))

.contains(entry("lastName", "Grebski"))

.contains(entry("email", "[email protected]"));

}

Page 34: On heap cache vs off-heap cache

5. COMPARISON

Chronicle Hazelcast Redis

Deployment model Embedded Embedded / Separate Separate

Replication Yes Yes Master/Slave

Topics (pub/sub) No Yes Yes

Executor Service No Yes No

Persistence Yes (Memory mapped file) Yes (db, file, custom) Yes (periodically)

Java collections support Yes Yes Yes (Redisson)

Clustering No Yes Yes (since 3.0)

Distributed events No Yes Yes

Features

Page 35: On heap cache vs off-heap cache

5. COMPARISON

R/W Performance

Chronicle Hazelcast Redis ConcurrentHashMap

Write (2mln entries) 2 555 583 178 491 110 668 1 100 715

Read (2mln entries) 937 207 214 638 156 177 4 092 769

0

500 000

1 000 000

1 500 000

2 000 000

2 500 000

3 000 000

3 500 000

4 000 000

4 500 000

Op

s/s

R/W Performance

Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)

Page 36: On heap cache vs off-heap cache

5. PERFORMANCE COMPARISON

GC Pauses

Heap: 1gb, G1 GC, Threads: 2, Records: 2mln, Entries: counter, User(username, 2x role, long)

Chronicle Hazelcast Redis ConcurrentHashMap

GC pauses summarized 39 1452 480 726

GC pause max 8 28 15 360

0

200

400

600

800

1000

1200

1400

1600

Tim

e in

mill

is

Page 37: On heap cache vs off-heap cache

Which cache should I use in my app ?

Page 38: On heap cache vs off-heap cache

Questions?

Page 39: On heap cache vs off-heap cache

Thank You !

Radek Grębski@RadekGrebski

https://github.com/rgrebski