jcmd #javacasual
TRANSCRIPT
![Page 1: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/1.jpg)
jcmd 便利
KUBOTA Yuji @sugarlife <[email protected]>
NTT OSS Center
2016/Nov/07
Copyright©2016 NTT corporation
![Page 2: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/2.jpg)
Who I amKUBOTA Yuji (@sugarlife)
* Java Technical Support Engineer at NTT
* OpenJDK author, Icedtea committer, Speaker
Copyright©2016 NTT corporation
HeapStats/heapstats
![Page 3: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/3.jpg)
•Introduction to Troubleshooting in JDK 9
• jcmd と jhsdb を使おう
•スライド:https://goo.gl/4jaujJ
JDK 9 からの診断ツール
Copyright©2016 NTT corporation
![Page 4: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/4.jpg)
Java 8
Copyright©2016 NTT corporation
![Page 5: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/5.jpg)
/jdk9/jdk/src/jdk.jcmd/share/classes/sun/tools% tree.|-- common| `-- ProcessArgumentMatcher.java|-- jcmd| |-- Arguments.java| `-- JCmd.java|-- jinfo| `-- JInfo.java|-- jmap| `-- JMap.java|-- jps| |-- Arguments.java| `-- Jps.java|-- jstack| `-- JStack.java`-- jstat
|-- Alignment.java:
Copyright©2016 NTT corporation
jdk9/jdk/src/jdk.attach/linux/classes/sun/tools/attach/VirtualMachineImpl.javajdk9/hotspot/src/share/vm/services/diagnosticCommand.cpp
Dynamic Attach (on Linux)
Unix domain socket
1. Send SIGQUIT
2. Start
AttachListener thread
4. Processing
3. Send request
5. Receive response
Serviceability
Tool(jcmdを含む)
Target
JVM
process
![Page 6: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/6.jpg)
•Introduction to Troubleshooting in JDK 9
•JDK 9 では jcmd / jhsdb を使おうという話
•スライド:https://goo.gl/4jaujJ
JavaOne 2016% jstack # JDK 8:Options:
-F to force a thread dump. Use when jstack <pid> does not respond (process is hung)-m to print both java and native frames (mixed mode)-l long listing. Prints additional information about locks-h or -help to print this help message
% ./jstack # JDK 9:Options:
-l long listing. Prints additional information about locks-h or -help to print this help message
# -F was removed by JDK-8155091: Remove SA related functions from tmtools
% ./jstack -mError: -m option usedCannot connect to core dump or remote debug server. Use jhsdb jstack instead% ./jhsdb jstack --mixedYou have to set --pid or --exe.
--locks to print java.util.concurrent locks--mixed to print both java and native frames (mixed mode)--exe executable image name--core path to coredump--pid pid of process to attach
Copyright©2016 NTT corporation
![Page 7: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/7.jpg)
•CLI の Local JVM process 診断ツール
• jcmd [-l | -h | process command]• -h : help, -l | no argument: jps
•process: <pid | main-class>• pid に 0 を指定すると全部• main-class は一致したプロセス全部
• command: <command[ options] | -f file>• command は Domain.suffix 形式
jcmd
Copyright©2016 NTT corporation
![Page 8: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/8.jpg)
• jcmd process help•実行可能なコマンド一覧 (対象プロセス依存)
• jcmd process help command•コマンドのヘルプ(使い方、負荷、オプション等)
Help
Copyright©2016 NTT corporation
% jcmd 999 help GC.run999:GC.runCall java.lang.System.gc().
Impact: Medium: Depends on Java heap size and content.
Syntax: GC.run
![Page 9: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/9.jpg)
Command: Domain.suffix (Java 8)
DomainNum.
suffixAbstract
PerfCounter 1 各種 JVM 統計情報 (※)
ManagementAgent 3 JMX エージェント操作
Thread 1 スレッドダンプ
GC 7 GC 機構 (Heap / GC / Finalization)
VM 7 ランタイム操作、情報取得
Copyright©2016 NTT corporation
※: 本来はコマンドとは別枠だが、見易さのため並べて記載している
![Page 10: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/10.jpg)
Command: Domain.suffix (Java 9)
DomainNum.
suffixAbstract
PerfCounter 1 - 各種 JVM 統計情報 (※)
ManagementAgent 4 +1 JMX エージェント操作
Thread 1 - スレッドダンプ
GC 7 +2
-1
GC 機構 (Heap / GC / Finalization)
VM 15 +8 ランタイム操作、情報取得
Compiler 7 - コンパイラ操作、情報取得
JVMTI 2 - JVMTI エージェント操作
GC.rotate_log が VM ドメインへ移動
Copyright©2016 NTT corporation
※: 本来はコマンドとは別枠だが、見易さのため並べて記載している
![Page 11: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/11.jpg)
•JVM 統計情報
• jstat や jinfo はこれを基に加工して出力してる• jdk/src/jdk.jcmd/share/classes/sun/tools/jstat/resources/jstat_options
•無加工は辛いので加工スクリプトに渡して出力すると便利
PerfCounter
Copyright©2016 NTT corporation
suffix Description
print JVM 統計情報を出力
![Page 12: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/12.jpg)
•Management Agent 起動したり終わらせたり• JMX (Management Extensions)と JDP(Discovery Protocol)
• JMXリモートアクセスを忘れてても後から設定可能
ManagementAgent
suffix Description
status 状態表示
start リモートエージェントを有効化
start_local ローカルエージェントを有効化
stop リモートエージェントを無効化
Copyright©2016 NTT corporation
JDK 9 から
![Page 13: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/13.jpg)
$ jcmd 999 ManagementAgent.start jmxremote.port=777 jmxremote.ssl=true999:Commandexecuted successfully$ jcmd 999 ManagementAgent.statusAgent: enabled
Connection Type: remoteProtocol : rmiHost : fedora24.novalocalURL : service:jmx:rmi://XXX…XXXProperties :com.sun.management.jmxremote.authenticate = true [default]com.sun.management.jmxremote.ssl.need.client.auth = false [default]com.sun.management.jmxremote.ssl = true [default]:com.sun.management.jmxremote.port = 8070
Copyright©2016 NTT corporation
[config.file | jmxremote.* | jdp.*]
![Page 14: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/14.jpg)
•Thread
•mixed モードが欲しい場合は% jhsdb jstack --mixed --pid <pid>
Thread
suffix Description
print Thread dump
Copyright©2016 NTT corporation
![Page 15: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/15.jpg)
•GC機構 (Heap/GC/Finalization)
GC
suffix Description
class_stats* クラスメタデータ一覧 (-helpで各項目説明)
class_histogram クラスヒストグラム (jmap -histo:live)
heap_info ヒープ・メタスペース使用量
heap_dump ヒープダンプ (jmap -dump, -allで参照切れも)
finalizer_info Finalize 待ちキュー表示
run_finalization Finalization 実行 (System#runFinalization)
run GC 実行 (System#gc)Copyright©2016 NTT corporation
*:要 -XX:+UnlockDiagnosticVMOptions参考:hotspot/src/share/vm/runtime/globals.hpp
![Page 16: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/16.jpg)
•ランタイム周り。情報出力系
VM
suffix Description
info Fatal Error Log (クラッシュ時等で出すログ)
uptime 起動時間
command_line VM 引数
system_properties システムプロパティ
version バージョン
flags 設定した(された)フラグ (-all で全フラグ)
dynlibs メモリマップ付き Dynamic library リストCopyright©2016 NTT corporation
![Page 17: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/17.jpg)
•ランタイム周り。情報出力・動作変更系
VM
suffix Description
print_touched_methods* JVM 実行中に触ったメソッドを表示
classloader_stats メタデータ状況等の全クラスローダ情報
class_hierarchy 読込クラスをクラスローダ付き階層表示
native_memory** Native Memory Tracking (NMT)
stringtable StringTable (-verboseで全Stringダンプ)
symboltable SymbolTable (-verboseで全シンボル)
set_flag flgname [value] JVM フラグの設定 (一部のみ)Copyright©2016 NTT corporation
*:要 -XX:+UnlockDiagnosticVMOptions
**: 要 -XX:NativeMemoryTracking 設定
![Page 18: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/18.jpg)
•Native メモリの状態を記録する1. Run Java with -XX:NativeMemoryTracking=summary
2. jcmd 999 VM.native_memory baseline
3. jcmd 999 VM.native_memory summary.diff
•詳細に見たい場合は上の summary を details に変える
•NativeMemoryTracking was available since Java 8• https://docs.oracle.com/javase/8/docs/technotes/
guides/troubleshoot/tooldescr007.html
Native Memory Tracking
Copyright©2016 NTT corporation
![Page 19: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/19.jpg)
$ jcmd 999 VM.native_memory baseline999:Baseline succeeded# --- Run method which wants to check the memory consumption ---$ jcmd 999 VM.native_memory summary.diff999:Native Memory Tracking:Total: reserved=5664359KB -36096KB, committed=371555KB -36096KB- Java Heap (reserved=4108288KB, committed=258048KB)
(mmap: reserved=4108288KB, committed=258048KB)- Class (reserved=1056953KB +15KB, committed=5433KB +15KB)
(classes #767 +3)(malloc=185KB +3KB #741 -4)(mmap: reserved=1056768KB, committed=5248KB)
- Thread (reserved=36128KB -112KB, committed=36128KB -112KB)(thread #36 +1)(stack: reserved=35980KB +325KB, committed=35980KB+325KB)(malloc=107KB +2KB #197 +4)(arena=41KB -130KB #70 +2)
:
Native Memory Tracking
±n shows diff from baseline
Copyright©2016 NTT corporation
![Page 20: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/20.jpg)
•ランタイム周り。ログ。
• -XX:+PrintGCDetails -Xloggc:gc.log.`date +%Y-
%m-%d_%H-%M-%S` -XX:+PrintGCTimeStamps
• -Xlog:gc*=debug:gc.log.%t:time,level
• -Xlog:[<selection>]:[<output>]:[<decorators>]
VM
suffix Description
log JVM Unified Logging (JEP158) 制御
Copyright©2016 NTT corporation
![Page 21: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/21.jpg)
• jcmd 999 VM.log [options]• -Xlog:[<selection>]:[<output>]:[<decorators>]
GC ログを設定するケース
Copyright©2016 NTT corporation
Options Description
output output (出力先)
output_options output (その他の設定)
what selection (ロギング対象とそのレベル)
decorators decorators (出力内容調整)
disable Clears all configurations and turns off all loggings
list Lists current log configurations
rotate Rotate all logs (instead of GC.log_rotate in Java 8)
![Page 22: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/22.jpg)
•Confirms current loggings
GC ログを設定するケース
Copyright©2016 NTT corporation
$ jcmd 999 VM.log list999:Available log levels: off, trace, debug, info, warning, errorAvailable log decorators: time (t), uptime (u), timemillis (tm), …Available log tags: add, age, alloc, arguments, annotation, barrier, …:Described tag combinations:logging: Logging for the log framework itself
Log output configuration:#0: stdout all=warning uptime,level,tags#1: stderr all=off uptime,level,tags
default configurations
![Page 23: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/23.jpg)
•Turns on a new logging: GC.log
GC ログを設定するケース
Copyright©2016 NTT corporation
$ jcmd 999 VM.log output="file=gc.log" output_options="filecount=5,filesize=10m"what="gc=debug" decorators="time,level"
999:Command executed successfully$ jcmd 999 VM.log list999:: Log output configuration:#0: stdout all=warning uptime,level,tags#1: stderr all=off uptime,level,tags#2: gc.log gc=debug time,level added a new GC logging
gc=log_level
name or path
options (log
rotation)
![Page 24: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/24.jpg)
•Turns off all existing loggings
• If you want to turn off #2 only• jcmd 999 VM.log output="#2" what="all=off"
GC ログを設定するケース
Copyright©2016 NTT corporation
$ jcmd 999 VM.log disable999:Command executed successfully$ jcmd 999 VM.log list999::
Log output configuration:#0: stdout all=off uptime,level,tags#1: stderr all=off uptime,level,tags
set all=off at stdout / stderr, and removed
the other configuration(s) such as #2
output nothing
![Page 25: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/25.jpg)
• jcmd 999 VM.log rotate
•Rotates logs satisfying the following1. Configured ‘output’ as file
2. Configured ‘filecount’ (by ‘output_option’)
手動ログローテート
$ jcmd 999 VM.log list:Log output configuration:#0: stdout all=warning uptime,level,tags#1: stderr all=off uptime,level,tags#2: gc.log gc=debug filecount=5,filesize=20M level,tags
#2 will be rotated,
not #0 and #1
Obsoleted GC.rotate_log
Copyright©2016 NTT corporation
![Page 26: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/26.jpg)
•JIT Compiler• JEP 165: Compiler Control (JDK 9 Feature)
Compiler
Copyright©2016 NTT corporation
suffix Description
directive_add Adds directives of JIT by JSON format same as -XX:CompilerDirectivesFile
directive_remove Removes a latest directive
directive_clear Clears all directives
directive_print Prints current directives
codecache Prints summary of code cache
codelist Lists all code cache
queue Lists methods in waiting queue for JIT
![Page 27: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/27.jpg)
% cat disable_c2_at_string-hashCode.json[
{ "match":[ "*String.hashCode" ],"c2":{ "Exclude":true }
}] % jcmd 999 Compiler.directives_add disable_c2_at_string-hashCode.json999:1 compiler directives added% jcmd 999 Compiler.directives_print999:
Directive:matching: *String.hashCodec1 directives:inline: -Enable:false Exclude:false BreakAtExecute:false Log:false PrintAssembly:false …
c2 directives:inline: -Enable:true Exclude:true BreakAtExecute:false Log:false PrintAssembly:false …
Switch false to true
Copyright©2016 NTT corporation
![Page 28: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/28.jpg)
• jcmd 999 Compiler.codelist•Prints JIT compiled methods (in code cache)
How JIT Compiler works?
% jcmd 999 Compiler.codelist999::18 3 java.lang.String.hashCode()I [0x00007ff301594090, 0x00007ff301594280 - 0x00007ff3015946e8]:
ID
Compile level of -XX:+TieredCompilation
0: Interpreter
1-3: C1
1: C1 without profiling (full optimization)
2: C1 with basic profiling (invocation and back-edge)
3: C1 with full profiling
4: C2
Method name nmethod start address code start address
code end address
Copyright©2016 NTT corporation
![Page 29: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/29.jpg)
• jcmd 999 Compiler.codelist•Prints JIT compiled methods (in code cache)
How JIT Compiler works?
% jcmd 999 Compiler.codelist999::18 3 java.lang.String.hashCode()I [0x00007ff301594090, 0x00007ff301594280 - 0x00007ff3015946e8]:
ID
Compile level of -XX:+TieredCompilation
0: Interpreter
1-3: C1
1: C1 without profiling (full optimization)
2: C1 with basic profiling (invocation and back-edge)
3: C1 with full profiling
4: C2
Method name nmethod start address code start address
code end address
Copyright©2016 NTT corporation
![Page 30: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/30.jpg)
How JIT Compiler works?
•JVM JIT compilation overview•http://www.slideshare.net/ZeroTurnaround/vladimir-ivanovjvmjitcompilationoverview-24613146
• Intrinsic Methods in HotSpot VM•http://www.slideshare.net/RednaxelaFX/green-teajug-hotspotintrinsics02232013
Copyright©2016 NTT corporation
![Page 31: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/31.jpg)
•JVMTI Agent 制御
JVMTI
suffix Description
agent_load library_path [option] Attach JVMTI agent of library path
data_dump Fire DataDumpRequest JVMTI
event
Copyright©2016 NTT corporation
% jcmd 999 JVMTI.agent_load /path/to/libheapstats.so% jcmd 999 JVMTI.agent_load ¥
/path/to/libheapstats.so=heapstats.conf
![Page 32: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/32.jpg)
Copyright©2016 NTT corporation
•DiagnosticCommandMBean (jconsole / jvisualvm)
リモートプロセスに jcmd
same as jcmd
commands
click then run jcmd
VM.system_properties
result
![Page 33: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/33.jpg)
HeapStatsLightweight JVMTI agent
Copyright©2016 NTT corporation
![Page 34: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/34.jpg)
•Gathers runtime information continuously, then draws up time-series graphs
•Lightweight agent to run on the production systems
•New solution to troubleshooting for OpenJDK users.
HeapStats
Copyright©2016 NTT corporation
![Page 35: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/35.jpg)
Overhead of HeapStats
467.59 461.43
0
50
100
150
200
250
300
350
400
450
500
Without HeapStats With HeapStats
(ops/m)
1.32 %
Overhead Rate
SPECjvm2008 Composite Result
Copyright©2016 NTT corporation
![Page 36: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/36.jpg)
$ jcmd 999 JVMTI.agent_load libheapstats.so999:Commandexecuted successfully
or
java.lang.OutOfMemoryError: Java heap space
$ java –jar heapstats-analyzer.jar
$ rpm -ivh heapstats-2.0.1-0.*.rpm$ java -agentpath:heapstats … Your_Application
then
How to use HeapStats
Copyright©2016 NTT corporation
![Page 37: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/37.jpg)
Copyright©2016 NTT corporation
![Page 38: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/38.jpg)
biegebyte[]
Copyright©2016 NTT corporation
![Page 39: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/39.jpg)
byte[]
Most doubtful class which has many references to byte[]
Copyright©2016 NTT corporation
![Page 40: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/40.jpg)
Monitor wait
RunSleep
Legend
Copyright©2016 NTT corporation
![Page 41: jcmd #javacasual](https://reader033.vdocuments.site/reader033/viewer/2022051706/586e8d221a28aba0038b8781/html5/thumbnails/41.jpg)
•https://github.com/HeapStats/heapstats
•http://icedtea.classpath.org/wiki/HeapStats
Your contributions are welcome!
Copyright©2016 NTT corporation