© 2007 ADP© 2007 ADP1
Java Capacity Planning & Performance Measurements
Dr. Carl J. De [email protected]
Electronic Numerical Integrator and
Computer Electronic Discrete Variable Automatic Computer
© 2007 ADP© 2007 ADP2
Java Capacity Planning & Performance Measurements
• Agenda– Capacity vs. Performance– Java capacity planning
• Java black box– Tools
• Java application– Tools
– Performance Measurements• Coding standards
• Byte code instrumentation– Method performance measurements using management library
– Method performance measurements using ARM & management library
• Tools
– Conclusions
© 2007 ADP© 2007 ADP3
Java Capacity Planning & Performance Measurements
Hardware Does Not Always Solve Capacity Problems • Capacity of a given hardware configuration is subject to the
application constraints
Capacity = f(Hardware, Application Performance)• Assume a given hardware configuration
– is 70% utilized, – Provides a 3 second application response,– When the arrival rate is two hundred transactions per second.– Management upgrades the hardware configuration with one that is
twice as fast
What could be the affect on the response time?
© 2007 ADP© 2007 ADP4
Java Capacity Planning & Performance Measurements
What could be the affect on the response time?• Response Time Could Improve
– If there isn’t an application CPU bottleneck– If there isn’t an application threading bottleneck– If there isn’t a network or I/O bottleneck
• Response Time Could Degrade– Work arrives faster
• Saturates non CPU peripherals• CPU experiences I/O Waits
– If there is an application threading bottleneck– If SQL is a poorly written
• More physical I/O executed than required• table scans/small reads
– If there is a Network Bottleneck• Small packet size
© 2007 ADP© 2007 ADP5
Java Capacity Planning & Performance Measurements
• Black box Java Capacity Planning– The Java Process
• CPU
• Disk
• Network
• Memory
– No application visibility
– Many java classes running in one JVM
• Application Capacity Planning– Process java
• Application Class/Method Data– CPU time by method
– Elapsed time by method
– Method invocation count
– Memory usage heap
– Workflow trace
– SQL timing
© 2007 ADP© 2007 ADP6
Java Capacity Planning & Performance Measurements
• Capacity Planning Black Box (java) - No Application Visibility– Collect process data– Identify busy period– Analyze data– Review reports to identify hot spots
• Computer CPU utilization
• Computer summary
• Disk IO summary
• Network summary
• Memory summary
• Process summary – Java will usually be the top resource consumer
© 2007 ADP© 2007 ADP7
Java Capacity Planning & Performance Measurements
• Capacity Planning Black Box (java) - No Application Visibility– Identify and eliminate all bottlenecks (analytically)– Apply and monitor changes to physical infrastructure– Once bottlenecks are eliminated
• Recollect performance data
• Use Analytical modeling to help determineCapacity = f(Hardware, Application Performance)
– Tools • Opnet IT GURU
• Hyperformix Capacity Manager
• Team Quest
• Performance Assurance
© 2007 ADP© 2007 ADP8
Java Capacity Planning & Performance Measurements
• Application Capacity Planning (java) - Application Visibility– Coding Standards
• Application Specific Log– Non standard format
– Each application log must be parsed by capacity planner
– May or may not contain necessary data
– Better than nothing
– Tools• Byte Code Engineering Library (BCEL)
• Java ClassLoader
© 2007 ADP© 2007 ADP9
Java Capacity Planning & Performance Measurements
Sample Codepublic final class test {
test () { }
public void runtest () {
for ( int i = 0 ; i < 9999999 ; i++ ) {}
}
public static void main(String[] argv) {
try {
for (int i = 0 ; i < 1; i++){
test t1 = new test();
t1.runtest();
for (int j = 0 ; j < 2 ; j++)
new oohrah ( argv[0], argv[1] ) ;
}
} catch ( Exception e ) {
e.printStackTrace();
}
}
}
Sample Codepublic class oohrah
{
public oohrah( String a, String b ) {
spin();
}
public static void spin() {
int i;
new hello();
for (i=0; i < 500000000; i++);
}
}
public class hello
{
public hello( ) {
printHello();
}
public static void printHello() {
System.out.println ("HelloWorld (from hello)");
}
}
© 2007 ADP© 2007 ADP10
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation - JDK 1.5Start
End
Parse Java Class
Create ClassGen, Get Constant Pool, Get Methods
Get Method
Are All Methods processed
If Main
Crete Print Count Method
Yes
Instrument MethodsNo
Yes
No
© 2007 ADP© 2007 ADP11
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation - JDK 1.5Start Class Name=test,Method Name main Elapsed 1180537593616 CPU 515625000Start Class Name=test,Method Name runtest Elapsed 1180537593632 CPU 515625000End Class Name=test,Method Name runtest Elapsed 1180537593663 CPU 546875000ClassName=test runtestCount==> 1ClassName=test runtestCpu==> 31250000ClassName=test mainCount==> 1ClassName=test mainCpu==> 0Start Class Name=oohrah,Method Name spin Elapsed 1180537593679 CPU 562500000Start Class Name=hello,Method Name printHello Elapsed 1180537593679 CPU 562500000HelloWorld (from hello)End Class Name=hello,Method Name printHello Elapsed 1180537593679 CPU 562500000ClassName=hello printHelloCount==> 1ClassName=hello printHelloCpu==> 0End Class Name=oohrah,Method Name spin Elapsed 1180537595053 CPU 1890625000ClassName=oohrah spinCount==> 1ClassName=oohrah spinCpu==> 1328125000Start Class Name=oohrah,Method Name spin Elapsed 1180537595053 CPU 1890625000Start Class Name=hello,Method Name printHello Elapsed 1180537595053 CPU 1890625000HelloWorld (from hello)End Class Name=hello,Method Name printHello Elapsed 1180537595053 CPU 1890625000ClassName=hello printHelloCount==> 2ClassName=hello printHelloCpu==> 0End Class Name=oohrah,Method Name spin Elapsed 1180537596366 CPU 3203125000ClassName=oohrah spinCount==> 2ClassName=oohrah spinCpu==> 2640625000End Class Name=test,Method Name main Elapsed 1180537596381 CPU 3218750000ClassName=test runtestCount==> 1ClassName=test runtestCpu==> 31250000ClassName=test mainCount==> 1ClassName=test mainCpu==> 2703125000
© 2007 ADP© 2007 ADP12
Java Capacity Planning & Performance Measurements
• Application Capacity Planning (java) - Application Visibility• ARM Instrument
– Elapsed time
– Invocation counts
– Correlation
– ArmTransactionsWithMetrics
• CPU time
– Tools• Byte Code Engineering Library (BCEL)
• Java ClassLoader
© 2007 ADP© 2007 ADP13
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation - ARMStart
End
Parse Java Class
Create ClassGen, Get Constant Pool, Get Methods
Get Method
Are All Methods processed
ARM Instrument Method
Yes
No
© 2007 ADP© 2007 ADP14
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_start_transaction()
time=03:09:34 06/13/07
transaction id=02000000 00000000 00000000 00000000
parent correlator: none
flags=NONE
application name=ApplicationDefinition test
transaction name=TransactionDefinition test.main
transaction instance handle=1
© 2007 ADP© 2007 ADP15
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_start_transaction() time=03:09:34 06/13/07 transaction id=04000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.runtest transaction instance handle=2
arm_stop_transaction() time=03:09:34 06/13/07 transaction instance handle=2 transaction status=GOOD flags=NONE application name=ApplicationDefinition test transaction name=TransactionDefinition test.runtest
© 2007 ADP© 2007 ADP16
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_start_transaction() time=03:09:34 06/13/07 transaction id=06000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition test app handle=1 tran name=TransactionDefinition test.main tran instance id=1 flags=NONE application name=ApplicationDefinition oohrah transaction name=TransactionDefinition oohrah.spin transaction instance handle=3
© 2007 ADP© 2007 ADP17
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_start_transaction() time=03:09:34 06/13/07 transaction id=08000000 00000000 00000000 00000000 parent correlator: length=44 format=126 flags=0 app name=ApplicationDefinition oohrah app handle=3 tran name=TransactionDefinition oohrah.spin tran instance id=3 flags=NONE application name=ApplicationDefinition hello transaction name=TransactionDefinition hello.printHello transaction instance handle=4
© 2007 ADP© 2007 ADP18
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_stop_transaction()
time=03:09:34 06/13/07
transaction instance handle=4
transaction status=GOOD
flags=NONE
application name=ApplicationDefinition hello
transaction name=TransactionDefinition hello.printHello
arm_stop_transaction()
time=03:09:35 06/13/07
transaction instance handle=3
transaction status=GOOD
flags=NONE
application name=ApplicationDefinition oohrah
transaction name=TransactionDefinition oohrah.spin
© 2007 ADP© 2007 ADP19
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_start_transaction()
time=03:09:35 06/13/07
transaction id=0a000000 00000000 00000000 00000000
parent correlator:
length=44
format=126
flags=0
app name=ApplicationDefinition test
app handle=1
tran name=TransactionDefinition test.main
tran instance id=1
flags=NONE
application name=ApplicationDefinition oohrah
transaction name=TransactionDefinition oohrah.spin
transaction instance handle=5
© 2007 ADP© 2007 ADP20
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_start_transaction()
time=03:09:35 06/13/07
transaction id=0c000000 00000000 00000000 00000000
parent correlator:
length=44
format=126
flags=0
app name=ApplicationDefinition oohrah
app handle=5
tran name=TransactionDefinition oohrah.spin
tran instance id=5
flags=NONE
application name=ApplicationDefinition hello
transaction name=TransactionDefinition hello.printHello
transaction instance handle=6
© 2007 ADP© 2007 ADP21
Java Capacity Planning & Performance Measurements
• Dynamic Byte Code Instrumentation – ARMarm_stop_transaction()
time=03:09:35 06/13/07
transaction instance handle=6
transaction status=GOOD
flags=NONE
application name=ApplicationDefinition hello
transaction name=TransactionDefinition hello.printHello
arm_stop_transaction()
time=03:09:35 06/13/07
transaction instance handle=5
transaction status=GOOD
flags=NONE
application name=ApplicationDefinition oohrah
transaction name=TransactionDefinition oohrah.spin
arm_stop_transaction()
time=03:09:35 06/13/07
transaction instance handle=1
transaction status=GOOD
flags=NONE
application name=ApplicationDefinition test
transaction name=TransactionDefinition test.main
© 2007 ADP© 2007 ADP22
Java Capacity Planning & Performance Measurements
• Analyzing the Raw Data
TT -hput)me)/ThrougthodBusyTi((Users/Me sedTimeMethodElap
s/TCompletion Throughput
otalVisitspsedTime/T MethodEla Service
psedTime/T MethodElan Utilizatio
nPeriodObservatio T
nt MethodCou sCompletion
© 2007 ADP© 2007 ADP23
Java Capacity Planning & Performance Measurements
• Operational Analysis – Method Raw Data Process
© 2007 ADP© 2007 ADP24
Java Capacity Planning & Performance Measurements
• Operational Analysis – Overloading• Process Method• Method Process• Valid for
• Black box, and application capacity planning
Denning, and J. Buzen, The operational analysis of queuing network models. Computer Surveys, 10(3),
225 - 261 (1978)
© 2007 ADP© 2007 ADP25
Java Capacity Planning & Performance Measurements
• Third Party Performance Analysis Tools- Unable to implement custom application logging - Unable to implement Dynamic Byte Code
Instrumentation- Third party alternatives
• Mercury
• Opnet Panorama/ACE
• Quest Perform Assure
• Wily
© 2007 ADP© 2007 ADP26
Java Capacity Planning & Performance Measurements
• ConclusionBy adding a bit of sophistication during application development, performance testing, or production deployment the often overlooked non-functional
requirement of “application performance” requirement can be addressed