how to get the most out of your hibernate, jboss eap 7 application (ståle pedersen)
TRANSCRIPT
![Page 1: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/1.jpg)
![Page 2: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/2.jpg)
How to get the most of of your Hibernate, JBoss EAP 7 application
Ståle Pedersen
![Page 3: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/3.jpg)
Agenda
Lessons of what we've learned….
● Methodology● Tools● Profiling● Benchmarking● EAP 7 Performance improvements
![Page 4: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/4.jpg)
Methodology
Performance optimization Methodology
![Page 5: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/5.jpg)
Methodology
What
● What prevents my application from running faster?○ Monitoring
![Page 6: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/6.jpg)
Methodology
What -> Where
● What prevents my application from running faster?○ Monitoring
● Where does it hide?○ Profiling
![Page 7: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/7.jpg)
Methodology
What -> Where -> How
● What prevents my application from running faster?○ Monitoring
● Where does it hide?○ Profiling
● How can we improve performance?○ tuning/optimizing
![Page 8: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/8.jpg)
Methodology
Top down approach
● System Level○ Disks, CPU, Memory, Network, ....
![Page 9: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/9.jpg)
Methodology
Top down approach
● System Level○ Disks, CPU, Memory, Network, ....
● JVM Level○ Heap, GC, JIT, Classloading, ….
![Page 10: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/10.jpg)
Methodology
Top down approach
● System Level○ Disks, CPU, Memory, Network, ....
● JVM Level○ Heap, GC, JIT, Classloading, ….
● Application Level○ APIs, Algorithms, Threading, ….
![Page 11: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/11.jpg)
Methodology
What are we looking for?
![Page 12: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/12.jpg)
Methodology
What are we looking for?
● Lot of %sys○ Networking, scheduling, swapping
![Page 13: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/13.jpg)
Methodology
What are we looking for?
● Lot of %sys○ Networking, scheduling, swapping
● Lot of %iowait○ Disk activity, not enough disk block/caches
![Page 14: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/14.jpg)
Methodology
What are we looking for?
● Lot of %sys○ Networking, scheduling, swapping
● Lot of %iowait○ Disk activity, not enough disk block/caches
● Lot of %irq, %soft○ Interacting with devices
![Page 15: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/15.jpg)
Methodology
What are we looking for?
● Lot of %sys○ Networking, scheduling, swapping
● Lot of %iowait○ Disk activity, not enough disk block/caches
● Lot of %irq, %soft○ Interacting with devices
● Lot of %idle○ Few (runnable) threads, big GC pauses
![Page 16: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/16.jpg)
Methodology
What are we looking for?
● Lot of %sys○ Networking, scheduling, swapping
● Lot of %iowait○ Disk activity, not enough disk block/caches
● Lot of %irq, %soft○ Interacting with devices
● Lot of %idle○ Few (runnable) threads, big GC pauses
● Lot of %user○ No need to use profiling until you have high %user time
![Page 17: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/17.jpg)
Tools
![Page 18: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/18.jpg)
Tools
Tool types
● Observability, safe, depending on overhead● Benchmarking, load testing. Difficult to do in a production environment● Tuning, changes could hurt performance, perhaps not now or at the current load, but later
![Page 19: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/19.jpg)
Tools
● Tool types○ Observability, basic
■ top■ dmesg■ free■ vmstat■ iostat■ mpstat■ sar
![Page 20: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/20.jpg)
top
System summary, list processes or threads
● Can miss processes that do not last long enough to be caught● Will most often consume a fair bit of CPU● Default summary CPU usage, use “1” to get a better view of cores
![Page 21: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/21.jpg)
dmesg
Print kernel ring buffer
● Display system messages● An error tool that can give info why an application is slow/dies● Eg: OOM issues, filehandles, TCP errors, ++
![Page 22: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/22.jpg)
free
Memory usage
● Give a quick overview over memory usage● Virtual page, and block devices I/O cache
![Page 23: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/23.jpg)
vmstat
Virtual memory stat
● r - number of processes running (waiting to run), should be < num cpus● b - number of processes sleeping● free – idle memory, should be high● si, so – memory swapped to/form disk, should be 0● cpu, percentages of total CPU time
○ user time, system time, idle, waiting for I/O and stolen time○ system time used for I/O
![Page 24: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/24.jpg)
mpstat
Processors statistics
● Gives a good overview of CPU time● Comparable to top (with 1 pressed to show all CPU's), but use less CPU
![Page 25: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/25.jpg)
sar
System activity information
● Check network throughput● rxpck/s, txpck/s – total number of packets received/transmitted per second● Use EDEV for statistics on failures the network devices are reporting
![Page 26: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/26.jpg)
sar
System activity information
● Reports TCP network traffic and errors● active/s – number of local initiated TCP connections per second● passive/s – number of remote initiated TCP connections per second● retrans/s – number of segments retransmitted per second
![Page 27: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/27.jpg)
iostat
Monitor system I/O devices
![Page 28: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/28.jpg)
Java Tools
Observing Java applications
![Page 29: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/29.jpg)
Java Tools
Observing Java applications
● Not as many tools out of the box
![Page 30: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/30.jpg)
Java Tools
Observing Java applications
● Not as many tools out of the box○ but they are powerful
● jstack, jstat, jcmd, jps, jmap, jconsole, jvisualvm
![Page 31: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/31.jpg)
jstack
Prints Java thread stack traces for a Java process
● Useful to quickly look at the current state of the threads
![Page 32: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/32.jpg)
jmap
Prints Java shared object memory maps or heap memory
● Several different options○ histo[:live] – Prints a histogram of the heap, list classes, number of objects, size○ heap – Prints information regarding heap status and usage○ dump:<dump-options> – Dump java heap○ No options will print shared object mappings
![Page 33: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/33.jpg)
jstat
Monitor JVM statistics; gc, compilations, classes
● Many useful options: class, compiler, gc, gccapacity, gccause, gcnew, gcold, gcutil, ... ○ class – number of classes loaded/unloaded, Kbytes loaded/unloaded○ compiler - number of compiled, failed, invalid compilations, with failedtype/method○ gcutil – garbage collection statistics; survivor space 0,1, eden, old space, permanent,
young generation, gc collection time, …
![Page 34: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/34.jpg)
Byteman
A tool that simplifies Java tracing, monitoring and testing● http://byteman.jboss.org/
RULE ServerLogger_serverStoppedCLASS org.jboss.as.server.ApplicationServerServiceMETHOD stopAT ENTRYIF TRUEDOopenTrace("stop","/tmp/stop.log");traceStack("stop: "+Thread.currentThread.getName()+"\n","stop");ENDRULE
![Page 35: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/35.jpg)
Java Tools
What have we learned● Java have decent monitoring tools out of the box● Not as easy to use, but powerful● Gives a lot of information as a (fairly) low cost
● Also included○ VisualVM○ JConsole
![Page 36: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/36.jpg)
Tools
Final thought
● Which tool you use/prefer is not important
![Page 37: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/37.jpg)
Tools
Final thought
● Which tool you use/prefer is not important○ What is important is that you can measure everything
![Page 38: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/38.jpg)
Profilers
![Page 39: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/39.jpg)
Profilers
High CPU load
Where is it?
![Page 40: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/40.jpg)
Profilers
High CPU load
Can profilers help?
![Page 41: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/41.jpg)
Profilers
High CPU load
Can profilers help?
Profilers (try) to show “where” application time is spent
![Page 42: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/42.jpg)
CPU Profiling Limitations
● Many problems are not CPU Bound○ Networking○ Database○ Remote Services○ I/O○ Garbage Collection○ ….
![Page 43: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/43.jpg)
Different CPU Profilers
● Instrumenting○ Adds timing code to application
● Sampling○ Collect thread dumps periodically○ VisualVM, JProfiler, Yourkit, ….
![Page 44: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/44.jpg)
Sampling CPU Profilers
[survey made by RebelLabs]
![Page 45: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/45.jpg)
Sampling Profilers
Samples
MainThread.run()
Backend.findUser()
Controller.doAction()
new User() fetchUser()
display()
someLogic() moreLogic()
evenMoreLogic()
![Page 46: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/46.jpg)
Sampling Profilers
More samples
MainThread.run()
Backend.findUser()
Controller.doAction()
new User() fetchUser()
display()
someLogic() moreLogic()
evenMoreLogic()
![Page 47: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/47.jpg)
Sampling Profilers
Stack tracing
● Most profilers use JVMTI
○ Using GetCallTrace
■ Triggers a global safepoint
■ Collect stack trace
● Large impact on application
● Samples only at safepoints
![Page 48: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/48.jpg)
What is a safepoint?
Java threads poll a global flag
○ At method exit/enter○ At “uncounted” (int) loops
![Page 49: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/49.jpg)
What is a safepoint?
A safepoint can be delayed by
○ Large Methods○ Long running “uncounted” (int) loops
![Page 50: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/50.jpg)
Sampling Profilers
Safepoint bias
MainThread.run()
Backend.findUser()
Controller.doAction()
new User() fetchUser()
display()
someLogic() moreLogic()
evenMoreLogic()
![Page 51: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/51.jpg)
Safepoint summary
JVM have many safepoint operations
• To monitor its impact use– -XX:+PrintGCApplicationStoppedTime– -XX:+PrintSafepointStatistics– -XX:+PrintSafepointStatisticsCount=X
![Page 52: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/52.jpg)
Sample Profilers
Java profilers that work better!
● Profilers that use AsyncGetCallTrace
○ Java Mission Control■ Very good tool, just don’t use it in production (without a license)
○ Oracle Solaris Studio■ Not well known, it’s free and it works on Linux
○ Honest Profiler■ Open source, cruder, but works very well
![Page 53: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/53.jpg)
Java Mission Control/Flight Recorder
● Designed for usage in production systems● Potentially low overhead
○ Be wary of what you record■ Capture everything is not a good strategy■ Capture data for one specific thing (memory, contention, cpu)
● Capturing data will not be skewed by safepoints○ Make sure to use:
■ -XX:+UnlockDiagnosticVMOptions -XX:+DebugNonSafepoints● Need a license to be used in production, free to use for development
![Page 54: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/54.jpg)
Java Profilers problem
● Java stack only
● Method tracing have large observer effect
● Do not report:
○ GC
○ Deopt
○ Eg System.arrayCopy ….
![Page 55: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/55.jpg)
![Page 56: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/56.jpg)
Perf
● A system profiler● Show the whole stack
○ JVM○ GC○ Kernel
● Previously not very useful for Java○ Missing stacks○ Method symbols missing
![Page 57: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/57.jpg)
Perf
● Solution○ Java8u60 added the option -XX:PreserveFramePointer
■ Allows system profilers to capture stack traces
● Visibility of everything○ Java methods○ JVM, GC, libraries, kernel
● Low overhead● Flamegraph
![Page 58: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/58.jpg)
System C++Java
![Page 59: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/59.jpg)
Flamegraph
Top edge show which method is using CPU and how much.
MainThread.run()
a()
b()
e()
c()
d()
![Page 60: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/60.jpg)
![Page 61: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/61.jpg)
Limitations of Perf Mixed Mode
● Frames might be missing (inlined)
● Disable inlining:
○ -XX:-Inline
○ Many more Java frames
○ Will be slower
● perf-map-agent have experimental un-inline support
![Page 62: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/62.jpg)
Profiling at Red Hat
● Perf for CPU profiling
● Java Mission Control
○ Contention
○ Thread analysis
○ Memory usage
![Page 63: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/63.jpg)
Benchmarking
![Page 64: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/64.jpg)
Benchmarking
Most benchmarks are flawed, wrong and full of errors.
![Page 65: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/65.jpg)
Benchmarking
Most benchmarks are flawed, wrong and full of errors○ Used cautiously it can be a very valuable tool to reveal performance issues, compare
applications, libraries, ...
![Page 66: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/66.jpg)
Benchmarking
● Java Benchmarking libraries○ Microbenchmarks
■ Java Microbenchmark Harness● http://openjdk.java.net/projects/code-tools/jmh/
■ JMH is the only library on Java we recommend for microbenchmarks■ Still possible to write bad benchmarks with JMH
![Page 67: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/67.jpg)
Benchmarking
Load testing frameworks
○ Gatling■ http://gatling.io/
○ Faban■ http://faban.org/
![Page 68: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/68.jpg)
Benchmarking
A Hibernate microbenchmark lesson
● We wanted to compare cache lookup performance between Hibernate 4 and 5○ We also wanted to test a new cache we added to Infinispan, Simple cache
● Created several JMH benchmarks that tested different use cases○ Default cache with and without eviction○ Simple cache with and without eviction
![Page 69: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/69.jpg)
Benchmarking
A Hibernate microbenchmark lesson
4.3 w/ eviction 292050 ops/s
4.3 w/o eviction 303992 ops/s
5.0 w/ eviction 314629 ops/s
5.0 w/o eviction 611487 ops/s
5.0 w/o eviction (simple cache) 706509 ops/s
![Page 70: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/70.jpg)
Benchmarking
We progressed to a large full scale EE benchmark.
● We expect similar or better results vs EAP6 (Hibernate 4.2)
![Page 71: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/71.jpg)
Benchmarking
We progressed to a large full scale EE benchmark.
● We expect similar or better results vs EAP6 (Hibernate 4.2)○ The response times are 5-10x slower!○ And we are now CPU bound
![Page 72: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/72.jpg)
Benchmarking
We progressed to a large full scale EE benchmark.
● We expect similar or better results vs EAP 6 (Hibernate 4.2)○ The response times are 5-10x slower!○ And we are now CPU bound
● How is that possible?
![Page 73: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/73.jpg)
Benchmarking
We progressed to a large full scale EE benchmark.
● We expect similar or better results vs EAP 6 (Hibernate 4.2)○ The response times are 5-10x slower!○ And we are now CPU bound
● How is that possible?○ We had several microbenchmarks showing better performance on Hibernate 5○ Could the problem lie in WildFly 10?○ Let’s profile!
![Page 74: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/74.jpg)
Benchmarking
![Page 75: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/75.jpg)
Benchmarking
![Page 76: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/76.jpg)
Benchmarking
What had happened?
● Why did we spend so much CPU in this method and why do it not reflect the results from the microbenchmarks?
![Page 77: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/77.jpg)
Benchmarking
What had happened?
● Why did we spend so much CPU in this method and why do it not reflect the results from the microbenchmarks?○ We started debugging
■ -XX+PrintCompilation● Show when/which methods are compiled/decompiled● Showed that the method only was compiled once, but twice on EAP 6 (Hib4.2)● Why?
![Page 78: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/78.jpg)
Benchmarking
What had happened?
● We had made several changes internally in Hibernate 5 to reduce memory usage
![Page 79: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/79.jpg)
Benchmarking
What had happened?
● We had made several changes internally in Hibernate 5 to reduce memory usage○ One specific class was refactored to an interface○ Changing a monomorphic call site to a bimorphic call site
■ Which can prevent compiling/inlining of methods!
![Page 80: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/80.jpg)
Benchmarking
What had happened?
● We had made several changes internally in Hibernate 5 to reduce memory usage○ One specific class was refactored to an interface○ Changing a monomorphic call site to a bimorphic call site
■ Which can prevent compiling/inlining of methods!
● Direct method calls will “always” be optimized (no subclassing)● In Java subclassing/inheritance might cause the JVM to no optimize method calls
![Page 81: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/81.jpg)
Benchmarking
Lessons learned
● Code changes might have unexpected consequences○ In this case, reduced memory, increased CPU usage
● Benchmarks are good, but should always be questioned○ When you benchmark, analyze (all/most) possible use cases
![Page 82: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/82.jpg)
Benchmarking
Lessons learned
● Code changes might have unexpected consequences○ In this case, reduced memory, increased CPU usage
● Benchmarks are good, but should always be questioned○ When you benchmark, analyze (all/most) possible use cases
● Yes, we ended up fixing the CPU usage without causing it to use more memory!
![Page 83: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/83.jpg)
EAP 7 Performance Improvements
Performance highlights
● JPA● JCA● Web
![Page 84: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/84.jpg)
EAP 7 Performance Improvements
Hibernate 5 performance improvements
● Memory usage○ Reduced memory usage 20-50%
![Page 85: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/85.jpg)
EAP 7 Performance Improvements
Hibernate 5 performance improvements
● Memory usage○ Reduced memory usage 20-50%
● Cache improvements○ Simple Cache
■ 132% Improvement with immutable objects
![Page 86: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/86.jpg)
EAP 7 Performance Improvements
Hibernate 5 performance improvements
● Memory usage○ Reduced memory usage 20-50%
● Cache improvements○ Simple Cache
■ 132% Improvement with immutable objects
● Pooled Optimizer○ In persistence.xml add property
■ hibernate.id.optimizer.pooled.preferred value="pooled-lotl”
![Page 87: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/87.jpg)
EAP 7 Performance Improvements
Hibernate 5 performance improvements
● Bytecode enhancements○ hibernate.enhancer.enableDirtyTracking○ hibernate.enhancer.enableLazyInitialization
● Improved caching○ hibernate.cache.use_reference_entries○ hibernate.cache.infinispan.immutable-entity.cfg = “immutable-entity”
![Page 88: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/88.jpg)
EAP 7 Performance Improvements
IronJacamar performance improvements
● Contention issues○ New default ManagedConnectionPool
![Page 89: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/89.jpg)
EAP 7 Performance Improvements
IronJacamar performance improvements
● Contention issues○ New default ManagedConnectionPool
● Disabled logging○ Not everything is disabled
■ <datasource …. enlistment-trace="false" >
![Page 90: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/90.jpg)
EAP 7 Performance Improvements
IronJacamar performance improvements
● Contention issues○ New default ManagedConnectionPool
● Disabled logging○ Not everything is disabled
■ <datasource …. enlistment-trace="false" >
● Fairness○ Set to true at default
■ <datasource> <pool><fair>false</fair></pool>
![Page 91: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/91.jpg)
EAP 7 Performance Improvements
Undertow vs JBossWeb
● Undertow is nonblocking● JBossWeb have a thread for each connection
![Page 92: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/92.jpg)
EAP 7 Performance Improvements
Simple HelloServlet Benchmark
EAP 6.4 EAP 7RPS Mean 95% 99% Mean 95% 99%5000 2 1 4 0 1 1
15000 3 4 56 1 2 625000 13 18 247 1 3 740000 46 246 414 12 23 8455000 361 647 1889 65 136 635
![Page 93: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/93.jpg)
EAP 7 Performance Improvements
● Lightweight● Embeddable● Scalable● HTTP/2● HTTP Upgrade
![Page 94: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/94.jpg)
EAP 7 Performance Improvements
Overall EAP 7 Performance tricks
● Remote byte pooling○ jboss.remoting.pooled-buffers
● Disable CDI○ CDI is enabled by default in EAP 7
![Page 95: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/95.jpg)
EAP 7 Performance Improvements
Overall EAP 7 Performance tricks
● Remote byte pooling○ jboss.remoting.pooled-buffers
● Disable CDI○ CDI is enabled by default in EAP 7
![Page 96: How To Get The Most Out Of Your Hibernate, JBoss EAP 7 Application (Ståle Pedersen)](https://reader034.vdocuments.site/reader034/viewer/2022050722/58ed36c71a28ab2a1f8b45bb/html5/thumbnails/96.jpg)
Q&A