michel duijvestijn rotterdam university of applied science testscript can be used to test...

Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

2011 European Users Conference Brussels

Automated testing of assessment delivery performance

using Jmeter

Michel Duijvestijn

Rotterdam University of Applied Science

2011 European Users Conference Brussels Copyright © 1995-2011 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved.

Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.

How YOU(!) can use a tool (JMeter) to test the delivery of QMP-assessments

Why you may want to do that

How it works and why it‟s important to know a bit about that

What parameters there are to set

How to interprete the test results

How we set up a complete testing environment

Our results until now

What to expect?



Introduction

How to get started

Running the script as a black box


Into the black box: how things work

Parameter settings

Setting up an integrated test environment

Some test results

Discussion

Agenda



Software-tool to build and execute JMeter testscripts

JMeter testscript can be used to test webapplications (like QMP assessment delivery)

Open source

Java based

Multi-platform (windows, linux, Mac?)

Very active community

What is Jmeter?



With a proper testscript you can simulate an assessment delivery in QMP: „Student‟ goes to website

Is logging on

Chooses the right assessment

Gives a „valid‟ answer to all of the questions

takes a look at the feedback

Returns to the homepage

Is logging off again

In connection to QMP:



Performance: How many concurrent users

While using random questions: all questions performing well?

May be helpfull for debugging incidently occurring questions

Regression test after server maintenance?

Controlling 24x7 server availability

Possible applications



Introduction

How to get started Running the script as a black box



Parameter settings


Some test results

Discussion

Agenda



An all-inclusive ZIP-file containing JMeter installation

“working Demo” script for QMP version 5

A conversion of this script to work with QMP v4 (NB: with limited number of question types!)

“getting started” guide for this script

“reference guide” with complete documentation

Extended script for use in combination with…

…the “work in progress” Access database to control the proces (NB: only partly documented)

Support as much as possible, but no guarantee!

What do I have to offer?

Slide 8



Schedule assessment in QMP

Unzip the file (no need to install) Optional: plugin

Start JMeter.bat and load the demoscript

Provide script with your own settings

Run it!

To get started

DEMO…



Import group of testusers into QMP (all having the same password)

Make a copy of the assessment to test

Show feedback at end of assessment

Make a note of Assessment ID (see assessment properties)

Schedule Assessment (JM-version) for testusers

Preparations within QMP



JMeter software in C:\jakarta-jmeter-2.4?

Plugin-jar in C:\jakarta-jmeter-2.4\lib\ext?

JMeterDemo in \bin\scripts?

Link on desktop to \bin\scripts\JMeter.bat (including –t-link to script?!)

File with usernames at proper spot

Other preparations



„testplan‟-level sCustomerid (YOURID) (= repository) Path (/your/serverpath/perception.php) Password (YourParticipantPassword)

„HTTP-request defaults level Server url (your.perceptionserver.nl) Http protocol + poort (http/https; 80/443)

User parameters level Assessment-id (at sSession) Assessment-name (optional) Check path to file with user names

Adjust script settings



Introduction

How to get started




Parameter settings


Some test results

Discussion

Agenda



<ctrl+e> clear results

<ctrl+r> run

Start it

Slide 14



Following progress with View Results Tree

Slide 15



Following progress with Aggregate report

Slide 16



<ctrl+.>

If things go wrong: STOP!

Slide 17



Introduction

How to get started




Parameter settings


Some test results

Discussion

Agenda



Green response? Not necessarily everything OK!

So: check if every step gives the right response!

No errors???

Slide 19



Shows responstime in ms

Can rise to tens of seconds!

Response time dependent of assessment timing (more about that later)

The complexity of the assessment

Of course number of concurrent users

Important: Error%!

Interpretation „aggregate report‟



Introduction

How to get started




Parameter settings


Some test results

Discussion

Agenda



Listeners (show results) View result tree (result of individual requests) Aggregate report (response times, errors)

Controllers (logical structure) If/then While

Samplers http-request Debug sampler

Timers

Error-handling: assertions

Parts to discuss



To start an assessment you need a set of parameters The good news: this information is available in the

previous page!

The magic: how to get this information out of the page?

How to start an assessment?

customerid=HSRPROD&

action=StartAssessment&

login=5637881790119879316&

authentication=Password&

progress=0&

syslang=nl-NL&

assessmentID=7939180316566014&

lang=-&

expires=2011-03-05T18:09:43.818186+01:00&

resume=false&

access=CEC090C693D0D5C330468B0BC967B27C&

monitor=false&

useQS=false&

isOpen=false&

assessmentName=JM_Stresstest - Stoas&

scheduleID=864719902&

scheduleGroupID=161686849&

forDownload=false

customerid=HSRPROD">

<input name="action" type="hidden" value="StartAssessment">

<input name="login" type="hidden" value="5637881790119879316">

<input name="authentication" type="hidden" value="Password">

<input name="progress" type="hidden" value="0">

<input name="syslang" type="hidden" value="nl-NL">

<input name="assessmentID" type="hidden" value="7939180316566014">

<input name="lang" type="hidden" value="-">

<input name="expires" type="hidden" value="2011-03-05T18:09:43.818186+01:00">

<input name="resume" type="hidden" value="false">

<input name="access" type="hidden" value="CEC090C693D0D5C330468B0BC967B27C">

<input name="monitor" type="hidden" value="false">

<input name="useQS" type="hidden" value="false">

<input name="isOpen" type="hidden" value="false">

<input name="assessmentName" type="hidden" value="JM_Stresstest - Stoas">

<input name="scheduleID" type="hidden" value="864719902">

<input name="scheduleGroupID" type="hidden" value="161686849">

<input name="forDownload" type="hidden" value="false“>



Answer: regular expression extractor



Retrieve information with “post-processor” at loginpage…



… and apply it with the request for the next page (= assessment)



The same way: determining which answers to give



QMP generating an error message = for JMeter correct received data, so nothing wrong?!

Solution: define when JMeter shoud “fail” : assertions

If things go wrong within QMP: response assertion



Yes and no…

You can use it as a black box but:

You may need to dig in the code in case anything is not working properly

Extensive reference guide available (at community website)

Sounds complicated? Do I need to know?



Introduction

How to get started




Parameter settings Setting up an integrated test environment

Some test results

Discussion

Agenda



Without measurements: everything ASAP = how to crash your server with DDoS

Not a good representation of the actual situation

Solution: use timers Constant delay offset

Gaussian: constant delay and deviation

Start of the test: „ramp-up‟ time (= time in seconds within all threads should start up)

Assessment timing



Introduction

How to get started




Parameter settings


Some test results

Discussion

Agenda



Manually changing parameters

No storage of results

In case of parallel use: no combination of results

limitations of basic script

Slide 34



New script version Same basic principle, but

All input and output via text files

Some adaptations for „parallel use‟ (max. 12 x 20)

Access database Parameter preset tables

VBA code to export necessary parameter files

VBA code to monitor progress

VBA code to import results

Reports to display results

Available, but only partly documented

Integrated test environment (how we dit it)

Slide 35



Select testparameters

Slide 37



Monitoring progress

Slide 38



Display results

Slide 39



Introduction

How to get started




Parameter settings


Some test results Discussion

Agenda



Some results (1)



Some results (2)



Introduction

How to get started




Parameter settings


Some test results

Discussion

Agenda



Questions…

michel duijvestijn rotterdam university of applied science testscript can be used to test...

Documents