dr. low latency or: how i learned to stop worrying about pauses and love the memory
DESCRIPTION
Presented by Jaromir Hamala, C2B2 Senior Consultant, at JDays Conference on the 27th of November 2013. Cheap RAM means we can use more memory than an average disk size was not that long time ago. It's very tempting to increase size of our JVM heap and store more data in-memory. As always, there is a trade-off: Large heaps lead to higher latencies due to Garbage Collector overhead. This session will show different strategies for accessing off-heap memory and using the advantage of large RAM without paying the performance penalty.TRANSCRIPT
© C2B2 Consulting Limited 2013
All Rights Reserved
@author Jaromir Hamala, [email protected] @author(type=Twitter) @jerrinot
Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the
Memory Heap
© C2B2 Consulting Limited 2013
All Rights Reserved
Who Am I?
■ Jaromir Hamala
■ A curious developer
■ Who likes to break play with stuff
© C2B2 Consulting Limited 2013
All Rights Reserved
Safe Harbor
This talk may contain non-sense!
© C2B2 Consulting Limited 2013
All Rights Reserved
Facts
• Memory is Cheap
• We have a lot of data
• Data in Memory == Fastest Possible Data (apart from CPU
caches)
http://www.npr.org/blogs/thetwo-way/2012/06/06/154416480/ho-hum-dull-and-boring-are-now-a-pair
© C2B2 Consulting Limited 2013
All Rights Reserved
On-Heap Allocation
• CoolObject cool = new CoolObject()
• Great:
– It’s easy and usually *very* fast! • Allocation can be actually faster than in C
– GC goodies!
• Not so great:
– GC leads to Stop-the-World Pauses
– Bigger Heap -> Longer Pauses
© C2B2 Consulting Limited 2013
All Rights Reserved
© C2B2 Consulting Limited 2013
All Rights Reserved
© C2B2 Consulting Limited 2013
All Rights Reserved
Why to go Off-Heap?
• Latency caused by GC
• Locality
• Sharing with non-Java
http://digboston.com/boston-lulz/2013/08/trolley-trollop-first-comes-love/attachment/why-god-why-kitten1/
© C2B2 Consulting Limited 2013
All Rights Reserved
GC Implementations
• (Old Gen) GC in HotSpot (Oracle/Sun JDK, OpenJDK)
– ParOld – STW pauses by design!
– CMS – fragmentation -> STW pause
– G1 – Immature (yet?), not fulfilling the expectations -> occasional STW pauses
© C2B2 Consulting Limited 2013
All Rights Reserved
Latency can kill you!
© C2B2 Consulting Limited 2013
All Rights Reserved
What is Off-Heap?
• Area of memory not maintained by GC
– Manual memory control
• No Notion of Java object
– serialization
http://nickshell1983.wordpress.com/2010/06/29/taking-a-god-nudged-leap-of-faith/
© C2B2 Consulting Limited 2013
All Rights Reserved
Off-Heap Options?
• Direct Memory Buffer
• sun.misc.Unsafe
• JNI
• Memory Mapped File
• ….
http://en.wikipedia.org/wiki/File:Road_to_Hell_One_Sheet.jpg
© C2B2 Consulting Limited 2013
All Rights Reserved
Direct Memory Buffer
• ByteBuffer directBuf = ByteBuffer.allocateDirect(int noBytes);
• Part of Java NIO
• Max. 2GB / buffer
• Pure Java = Portable
• getX()/putX() methods
© C2B2 Consulting Limited 2013
All Rights Reserved
sun.misc.Unsafe
– long addr = unsafe.allocateMemory(noBytes)
– Not a standardized part of JDK!
– Implemented by most JVM vendors
• Sun / Oracle / IBM
– It can disappear from JDK without warning!
© C2B2 Consulting Limited 2013
All Rights Reserved
Cassandra I.
public class NativeAllocator implements IAllocator
{
static final Unsafe unsafe;
static {
try {
Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (sun.misc.Unsafe) field.get(null);
}
catch (Exception e) {
throw new AssertionError(e);
}
}
public long allocate(long size) {
return unsafe.allocateMemory(size);
}
public void free(long peer) {
unsafe.freeMemory(peer);
}
}
© C2B2 Consulting Limited 2013
All Rights Reserved
JNI
• Java Native Interface
• Native library means worst portability between platforms
• JNI Overhead
• Can re-use already existing (native) allocator
© C2B2 Consulting Limited 2013
All Rights Reserved
Cassandra II.
• Java Native Interface via JNA public class JEMallocAllocator implements IAllocator {
public interface JEMLibrary extends Library {
long malloc(long size);
void free(long pointer);
}
private final JEMLibrary library;
public JEMallocAllocator() {
library = (JEMLibrary) Native.loadLibrary("jemalloc", JEMLibrary.class);
}
public long allocate(long size) {
return library.malloc(size);
}
public void free(long peer) {
library.free(peer);
}
}
© C2B2 Consulting Limited 2013
All Rights Reserved
Memory Mapped File
• Part of NIO
• Pure Java -> Portable
• Allows sharing data between processes
• Persisted by OS
© C2B2 Consulting Limited 2013
All Rights Reserved
private static int count = 1010241024; //10 MB
public static void main(String[] args) throws Exception {
RandomAccessFile memoryMappedFile = new RandomAccessFile("largeFile.txt", "rw");
//Mapping a file into memory
MappedByteBuffer out = memoryMappedFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0, count);
//Writing into Memory Mapped File
for (int i = 0; i < count; i++) {
out.put((byte) 'A');
}
System.out.println("Writing to Memory Mapped File is completed");
//reading from memory file in Java
for (int i = 0; i < 10 ; i++) {
System.out.print((char) out.get(i));
}
System.out.println("Reading from Memory Mapped File is completed");
}
Javin Paul,
http://javarevisited.blogspot.com/2012/01/memorymapped-file-and-io-in-java.html
© C2B2 Consulting Limited 2013
All Rights Reserved
Use cases for Off-Heap?
• Caches / Data Grids
– A lot of data
– Objects with well defined lifecycle
• (Extremely) Latency-Sensitive tasks
– High-Frequency trading…
• Sharing Data with non-Java
© C2B2 Consulting Limited 2013
All Rights Reserved
Apache DirectMemory
• “…is a off-heap cache for the Java Virtual Machine”
• Easy to integrate
• Friendly License
• Experimental
• http://directmemory.apache.org/
© C2B2 Consulting Limited 2013
All Rights Reserved
DirectMemory Usage
CacheService<Object, Object> cacheService =
new DirectMemory<Object, Object>()
.setNumberOfBuffers( 10 )
.setSize( 1000 )
.setInitialCapacity( 100000 )
.setConcurrencyLevel( 4 )
.newCacheService();
put( K key, V value )
put( K key, V value, int expiresIn )
© C2B2 Consulting Limited 2013
All Rights Reserved
Is Off-Heap The Future of Java?
• Oh God! Please no!
• R&D should be aimed to more effective GC
– Zing JVM (based on HotSpot, C4 - the fully concurrent GC)
– RedHat Shenandoah?
© C2B2 Consulting Limited 2013
All Rights Reserved
Final Warning(s)
• Stay On-Heap unless you have no other choice!
• If you think you have no other choice, think twice
• Do NOT re-invent the wheel! (Memory Management)
• Measure, measure, measure!
© C2B2 Consulting Limited 2013
All Rights Reserved
Further Sources
• http://mechanical-sympathy.blogspot.com/ – Martin Thompson. There is a brilliant mailing list as well!
• http://psy-lob-saw.blogspot.com – Nitsan Wakart
• http://vanillajava.blogspot.com – Peter Lawrey
• http://insightfullogic.com/blog/ – Richard Warburton
• http://ashkrit.blogspot.com – Ashkrit
© C2B2 Consulting Limited 2013
All Rights Reserved
© C2B2 Consulting Limited 2013
All Rights Reserved
Thank you