SAIP 2012 Performance Engineering Author: Jens Edlef Møller (jem@cs.au.dk) Instructions for installation, setup and use of tools.

Introduction For the project assignment a number of tools will be used. The tools are open source tools which are freely available through the internet. Although the tools are free they are of professional quality and used by many professional performance engineers for performance testing in “the real world”. The tools are based on Java and will therefore be usable on most mainstream platforms, including Windows, Linux and Mac OS. This installation guide has been verified with Windows XP and Windows 7.

Mandatory Tools - Installation

Apache JMeter This is an open source performance testing tool which can generate load on your system and measure the response times of different transactions. Installation

1. Go to http://jmeter.apache.org 2. Go to “Download Releases” section and download latest binary version (e.g. zip-file). (Your

teacher has tested this with version 2.7) 3. Extract the zip-file into a directory on your hard drive <jmeter-dir> 4. Add <jmeter-dir>\bin directory to the PATH environment variable

JMeter Plugins This is an Open source plugin for JMeter which adds a number of useful features to the basic JMeter tool. For this project we will use the features to collect CPU measurements and to optionally upload test results to a cloud-based performance test analysis tool. This plugin contains several components which are used by the performance test:

- JMeter test elements (are automatically available from JMeter test plans) - Measurement agent, which is a java-program which can measure resource usage of processes on

the operating system level (e.g. CPU, memory, I/O, etc.). The agent must be started on each machine that hosts processes to be measured during the performance test.

Note: The measurement agent is supported on these platforms: http://support.hyperic.com/display/SIGAR/Home#Home-overview Installation

1. Go to http://code.google.com/p/jmeter-plugins/downloads/list 2. Download newest release (zip-file). (Your teacher has tested this with version 0.5.4) 3. Extract the zip-file into a directory on your hard drive <jmeter-plugins-dir> 4. Copy JMeterPlugins.jar from <jmeter-plugins-dir> to <jmeter-dir>\lib\ext

Verification of JMeter and JMeter-Plugins installation

JMeter JMeter is started by invoking the jmeter script file in <jmeter-dir>\bin (this is jmeter.bat on Windows). This will launch the JMeter Graphical User Interface:

JMeter-Plugins Verify that JMeter-Plugins is correctly installed into JMeter by selecting the “Edit – Add – Listener” submenu and verify that there are a number of listener types beginning with the string “jp@gc”:

To verify that the operating system measurement agent works you should start the agent on your local machine:

1. Open a command prompt and go to the directory <jmeter-plugins-dir>\serverAgent 2. Invoke the startAgent script to start the agent (this is startAgent.bat on Windows)

The measurement agent by default uses port number 4444 to communicate with JMeter. In case this port is used by another process on your machine this will generate an error in the console window. Refer to a later section of this document for information on how to change port numbers.

Optional Tools - Installation

Loadosophia.org Cloud-based tool which can generate test reports for performance tests executed by JMeter. It is powerful, user friendly, free and offers seamless integration with JMeter-Plugins extension to JMeter. To use loadosophia.org you must create a workspace where you will upload your test results and generate reports. Loadosophia uses Google account for authentication purposes. Installation

1. Ensure you have Google Account (which you have if you have gmail…). If you don’t have a Google account you can create one from this link: https://accounts.google.com/SignUp

2. Go to http://loadosophia.org/ and click “My Workspace” in upper left corner of the page:

3. Proceed to Google Authentication to login (press button):

4. Create a new project called “TM12” by pressing the “Add Project” link:

5. Issue an upload token which can be used to automatically upload test results from JMeter to loadosophia.org when a test ends. The token is issued through this link: https://loadosophia.org/service/upload/token/ . Follow the instructions in the link to generate the token, and then copy it from the webpage into the clipboard, and paste it into JMeter as outlined in point 7.

6. Open the TM12 test plan in JMeter (Menu File, Open / ctrl-O). This test plan is designed by us (: SAiP teachers) for use on the TM12 project, and you will have to download it and put it in some suitable directory, like the root of the TM12 project.

7. Insert this token into the JMeter Test Plan by clicking on the testplan’s jp@gc - Loadosophia.org Uploader entry in the Test Plan tree, and paste it into the ‘upload token’

textbox.

jvisualvm This tool is a powerful java code profiler which is distributed with JDK. It has a nice graphical user interface and is very easy to use. It is launched by executing jdk\bin\jvisualvm.exe. Installation

1. You should already have the JDK installed in order to execute the TM12 application. If you do not have this installed, you can download it from http://www.oracle.com/technetwork/java/javase/overview/index.html

Performance Testing with the TM12 Test Plan A default JMeter Test Plan will be handed out for you to use for the performance test of TM12. This Test Plan assumes that you have installed and configured the tools as described above and that all processes execute on the same machine (i.e. “localhost”). This can all be changed, but in order to verify the installation of the tools it is advisable to try with this default test plan first. You can use either the memory based TM12 server (“ant server”) or the persistent server (“ant serverm”) in which case you must also have the mongo db running.

The performance test is controlled through the JMeter UI, but before you start the performance test you should ensure that all relevant components are started with no errors:

- TM12 Server process must be running locally - Server agent must be running locally (serverAgent.bat)

In JMeter you can load the TM12 Test Plan through the “File – Open” menu:

The test is launched by pressing the green “Start” button:

While executing the Test Plan it is important to monitor to ensure that test results are valid, i.e. that

- No unexpected errors occur - Throughput and response times look reasonable

The recommended way of doing this is to select the “Aggregate Graph” test element and observe that there are no errors reported in the top right corner of the JMeter UI and that there are no errors reported for the 2 transaction types in the test suite:

The red marks in the picture show that no errors are occurring in the test. The green marks in the picture show that all threads are running (11 of 11), that response times are reasonable (around 14 milliseconds for the threads reading data) and throughput is reasonable (around 2,2 transactions per second). In case you get errors, your test is not working and you need to investigate why. Likely problems can be that the server is not running on the expected address and port. In case you get very high response times and/or low throughput you probably have a bad configuration of your system affecting the test results. Also this must be investigated. As a supplement to monitoring through the “Aggregate Graph” element above you can also look at the reported CPU by selecting the “jp@gc - PerfMon Metrics Collector”:

The blue bar is reporting the total CPU usage of the machine – and the peaks are occurring due to launch of a major application unrelated to the performance test.

Adapt TM12 Test Plan If your test environment differs from the default plan you must adapt the JMeter test plan. This is very easy to do through the graphical UI. A few common cases are described below – for other changes please consult the JMeter online manual at http://jmeter.apache.com and the JMeter-Plugins manual at http://code.google.com/p/jmeter-plugins/.

Executing TM12 server on another machine For a more realistic test setup you should execute JMeter from a different machine than the TM12 system. In order to do this you should make the following changes:

- Start TM12 server on a different machine - Configure JMeter Test Plan to connect to system on a different machine

Starting TM12 server on a different machine is not further elaborated here. You need to know the IP address of the machine and test that the machine executing JMeter can reach the machine executing the TM12 server. This is best tested through the brower by entering this URL in the address bar:

http://tm12server/tm12?pid=pid01 where “tm12server” should be substituted by an IP address or a hostname known by the machine executing JMeter. You must also start the serverAgent on the server executing TM12. In the JMeter Test Plan you need to make these changes:

1. Change address of TM12 server (IP or hostname) for both the “Readers” and the “Writers” thread groups

2. Change the address for the server agent of the JettyServer process

Executing Server Agents on other ports If port numbers 4444 are in use by processes on your machine which you cannot stop during the performance test you need to make the following changes:

- Start serverAgent to listen on another port number - Configure JMeter Test Plan to connect to another port number

To make serverAgent listen on another port number you can specify other port numbers through command line options. It is recommended that you use the same port number for TCP and UDP protocols: startAgent --udp-port 5000 --tcp-port 5000 To configure JMeter to connect to another port number you should change the values in the “jp@gc - PerfMon Metrics Collector” test element:

In order to see who are using which port numbers, you can invoke the “netstat -a” command in a command prompt.

FAQ Issue Solution JMeter generates many errors There can be several reasons for this.

Check that the server is running and that you can access it using the browser. Also check that server addresses and port names are correct.

JMeter stops shortly after starting test suite Try disabling most elements from the test plan and gradually build it up – enabling one element at a time. Of course you should check the obvious: is the server running etc.

Perfmon Metrics Collector can not connect to measurement agent(s)

Check that the agent is running on the relevant machines. Also check the console window of the measurement agent for errors.

Perfmon Metrics Collector does not fetch any data for a counter

Check measurement agent window for error messages (query error). The connect string used by the Perfmon Metrics Collector might not point to the relevant processes – or these might not be running. This can be investigated using the shell of the SIGAR library, which is used by the serverAgent process (Invoke “java –jar sigar.jar” from the serverAgent sub-directory of JMeter-Plugins installation directory) For more information look at this page: http://support.hyperic.com/display/SIGAR/PTQL

Error uploading data to Loadosophia Check that you have created an account on the website and a project named TM12. Also, check that you have generated an upload token and inserted into the Test Plan.