alliance alliance performance status - creq régis elling july 2011

AllianceAlliance Performance Status - CREQ

Régis ELLING

July 2011

Alcatel-Lucent – InternalProprietary – Use pursuant to Company instruction. 2 | Performances enhancements| February2011

Alliance – Performance Status – Objectives

Describe the performance enhancements with objectives:

Current results of the Stress tests with highlights on slow features.

Remaining or on-going actions needed to enhance performances.

Stress Tests – CREQStrategy & results


Alliance – Performance Status – Stress Tests - Strategy

Strategy:

Stress tests have been updated and completed to have a global overview of the Alliance

OEFE application, testing AFE Order, B2B and now also CDR & CREQ.

Considering the high number of scenarios and also to ease the analysis, we have: Executed the runs on a feature by feature basis (per module): all the scenarios related to a given

functionality of the application are executed. This way it is also possible to optimize a “module”

(errors, response times, load …).

To execute all the scenarios at the same time: only once all the modules of the Alliance application

have been validated.

The following slides present the results for the CREQ module.

The last stress test and following results are from the R4.0.


Alliance – Performance Status – Stress Tests results - CREQ (1/7)

CREQ Management

The following features are included in this part of the test: CREQ: Create Generic & Specific CREQ, search with high number of results, low number of results,

Reply to Generic and Specific, Accept CREQ.

AFE management: Open Order, line creation by file upload.

Strategy applied:2 hours of execution of scenarios while increasing the number of users up to 50 during 1st hour, then

up to 100 for the 2nd hour.

Crash test not performed considering that there is a problem remaining on the search

CREQ that is increasing quickly and highly the load on the SQL Server CPU.



CREQ Management - Scenarios# Name + description Number of users 17 Search collaborative requests with many results (CRSM)

Execute a search on the collaborative requests that returns a lot of results (approx 200); for a collaborative request chosen randomly in the list of returned results display the main and the attachment tabs.

From 4 to 7

18 Create generic collaborative request (CRCG) Search the sales orders that were created by the “Submit CDR” scenario (these can be easily found given the search criteria). Open a sales order that has been chosen randomly and duplicate it. For the newly created sales order create a generic collaborative request and submit it.

From 14 to 27

19 Reply to generic collaborative request (CRRG) The user (who has been assigned the collaborative request (previous scenario)) searches the generic collaborative requests that were created by the previous scenario. A randomly chosen collaborative request is opened, completed and accepted.

From 13 to 27

20 Create specific collaborative request (CRCS) Search the sales orders that were created by the “Submit CDR” scenario (these can be easily found given the search criteria). Open a sales order that has been chosen randomly and duplicate it. For the newly created sales order create a CAE specific collaborative request and submit it.

From 6 to 13

21 Reply to specific collaborative request (CRRS) The user (who has been assigned the collaborative request (previous scenario)) searches the CAE specific collaborative requests that were created by the previous scenario. A randomly chosen collaborative request is opened, a configuration file is attached to it and completed (this “sends” the request back to the person who has created it).

From 6 to 13

22 Accept specific collaborative request (CRAS) The user (who has been assigned the collaborative request (previous scenario)) searches the CAE specific collaborative requests that have been modified by the previous scenario (in fact this user is the user who has created the specific collaborative request). A randomly chosen collaborative request is opened and accepted.

From 7 to 13



Errors during the stress test

The error rate is very low by scenario.

Many application errors occurred , “Timeout expired” errors happened during

searches on collaborative requests.

Transaction Description Executions Fails RateCRAS06_Accept Click on the “Accept” button in the CAE specific

CREQ197 1 0,50 %

CRRG03_SearchSubmit Search Generic CREQ for reply 493 2 0,41 %

CRRG04_OpenCR Open Generic CREQ for reply 443 1 0,22 %CRRG08_Accept Accept 442 1 0,21 %CRRS03_SearchSubmit Search collaborative requests 226 1 0,44 %

CRSM03_SearchSubmit Search collaborative requests 111 1 0,90 %

CRRS06_Upload Upload file to create lines 205 1 0,48 %



Global Response times during the stress test



Top 30 highest response timesTransaction Average response

time (in s) R3.4

Average response

time (in s) R4.0 Delta

CRRS06_Upload 80,6 4,1 94,97%CRRS03_SearchSubmit 62,3 102,5 -64,50%CRSM03_SearchSubmit 61,5 56,2 8,56%CRCS12_Continue 60,5 100,00%CRCG12_OK 60,3 100,00%CRAS03_SearchSubmit 59,8 95,6 -59,88%CRRG03_SearchSubmit 57,2 103,4 -80,72%CRRG06_Complete 46,1 4,1 91,21%CRCS03_Search 38,4 3,7 90,43%CRCG03_Search 35,1 3,8 89,16%CRRG08_Accept 33,5 3,1 90,89%CRCS05_Duplicate 30,6 8,1 73,58%CRCG05_Duplicate 30,4 7,9 74,12%CRCG15_Continue 28,2 5,4 80,75%CRSM06_AttachTab 27,8 7,5 73,03%CRCG14_Submit 24,4 4,9 79,77%CRCS14_Submit 23,9 4,5 81,28%CRRG04_OpenCR 23,5 2,3 90,02%CRSM07_MainTab 23,4 100,00%CRSM05_OpenCR 23,2 2,5 89,16%CRAS04_OpenCR 22,1 2,7 87,60%CRCS15_Continue 21,7 5,1 76,36%CRRS09_Complete 17,9 1,7 90,70%CRCS04_OpenOrder 16 4,0 75,03%CRCG04_OpenOrder 15,3 4,1 73,23%CRCG10_OpenCR 7,5 1,8 75,63%CRCS10_OpenCR 7,4 1,8 75,45%CRRS04_OpenCR 6,9 2,4 64,71%CRAS06_Accept 6 2,6 55,86%CRAS01_Search 3,7 2,7 27,72%

Overall, the module response times

have decreased by 63% compared to

the previous release.

Apart from the search CREQ which have

increased, all response times have greatly

decreased.

All response times are low (under 5

seconds) apart from:

Search CREQ (under analysis).

Duplicate (some additional actions are also

done in the steps)

Attach file to CREQ (5 seconds think time)

Continue (0,1 second over the 5)


Alliance – Performance Status - Stress Tests results - CREQ (6/7)

Activity during the Stress tests (100

users)

Scenarios / Nb executions R3.5-Pass R3.5-Fail R4.0-Pass R4.0-Fail Activity delta

CRAS01_Search 271 0 269 0 -0,74%

CRAS03_SearchSubmit 271 0 269 0 -0,74%

CRAS04_OpenCR 248 23 198 71 -0,74%

CRAS06_Accept 248 0 197 1 -20,16%

CRAS08_CloseCR 248 0 196 0 -20,97%

CRCG03_Search 266 0 495 0 86,09%

CRCG04_OpenOrder 264 2 495 0 86,09%

CRCG05_Duplicate 264 0 495 0 87,50%

CRCG09_SaveHeader 264 0 495 0 87,50%

CRCG10_OpenCR 264 0 495 0 87,50%

CRCG11_CreateCReq 264 0 495 0 87,50%

CRCG12_OK 264 0 0,00%

CRCG14_Submit 264 0 495 0 87,50%

CRCG15_Continue 264 0 495 0 87,50%

CRCG16_CloseCR 264 0 495 0 87,50%

CRCG18_CloseOrder 264 0 495 0 87,50%

CRCS03_Search 125 0 233 0 86,40%

CRCS04_OpenOrder 125 0 233 0 86,40%

CRCS05_Duplicate 125 0 233 0 86,40%

CRCS09_SaveHeader 125 0 233 0 86,40%

CRCS10_OpenCR 125 0 233 0 86,40%

CRCS11_CreateCAE 125 0 233 0 86,40%

CRCS12_Continue 125 0 0,00%

CRCS14_Submit 125 0 233 0 86,40%

CRCS15_Continue 125 0 233 0 86,40%

CRCS16_CloseCAE 125 0 233 0 86,40%

CRCS18_CloseOrder 125 0 233 0 86,40%

CRRG01_Search 401 0 495 0 23,44%

CRRG03_SearchSubmit 401 0 493 2 23,44%

CRRG04_OpenCR 347 0 443 1 27,95%

CRRG06_Complete 347 0 443 0 27,67%

CRRG08_Accept 347 0 442 1 27,67%

CRRG09_CloseCR 347 0 442 0 27,38%

CRRS01_Search 167 0 227 0 35,93%

CRRS03_SearchSubmit 166 1 226 1 35,93%

CRRS04_OpenCR 165 1 206 0 24,10%

CRRS06_Upload 164 1 205 1 24,85%

CRRS09_Complete 163 1 205 0 25,00%

CRRS10_CloseCR 163 0 205 0 25,77%

CRSM01_Search 123 0 112 0 -8,94%

CRSM03_SearchSubmit 123 0 111 1 -8,94%

CRSM04_Sort 89 0 99 0 11,24%

CRSM05_OpenCR 89 0 99 0 11,24%

CRSM06_AttachTab 89 0 99 0 11,24%

CRSM07_MainTab 89 0 0,00%

CRSM08_CloseCR 89 0 99 0 11,24%

CRSM09_ExportAll 99 0 100,00%

Average activity increase 43,60%

The 0% or 100% figures are because

either the scenario was not run previously

(application could not manage the load in

R3.4) or is not executed in R4.0 (functional

change).

The activity decreases are as follows:

CRASXX scenarios: decrease is related to

the fail of the previous step of search CREQ.

As a consequence the steps after the search

have been less executed.

CRSMXX scenarios: Little decrease of the

number of executions.

The CREQ module in R4.0 supports an

average of 43% more activity compared

to R3.4.


Alliance – Performance Status - Stress Tests results - CREQ (7/7)

CPU load during stress test

The load is higher on the SQL server than on the web servers: SQL server: around 95 % for 50 users / up to 60 % for 80 users and more

Web servers: from 10 % to 20 % for 50 users / between 20 % and 40 % for 100 users

From a memory usage perspective, there is a usage of 250 Mb RAM on each server out of the 2500

available. This part is also good.

The collaborative request module generates a too high load on the SQL server This is

mainly due to the CREQ search feature which falls into timeout after 80 users.


Alliance – Performance Status - Stress Tests - CREQ - Conclusions

Conclusions

The collaborative request module still generates a too high load on the SQL server (AS

web server results are now good); this is due to the SQL queries on the collaborative

request searches (high response times and timeouts using a lot of SQL server

resources).

Apart from the Search CREQ, there are no errors and the response times are good and

stable.

A final action is needed on the CREQ Search feature.

Remaining or on-going actions


Alliance – Performance Status – Action Plan

Needed or on-going actions

Needed actions: Analyse the Search CREQ feature to define optimizations to reduce the SQL server CPU usage and

response times.

Re-execute a Stress test to validate the results, if good then launch a crash test.


Alliance – Performance Status – Search Collaborative Request

Search Collaborative requests

The search collaborative requests is one key limitation in the module that also affects

the resource usage of the server.

Even though a optimizations have already been performed in R3.4 and R4.0, it is not

enough when this search facility is under heavy activity.

Investigations are as follows:Analyze the first part of the query which extracts the data of the Collaboration matrix to inject the

user rights to define the results. This part has been identified as CPU and time consuming and

increasing with the number of elements in the matrix itself.

Compare (after having done the run on the CDR) the behavior of the search CDR which deals with

the same kind of complexity, but is technically built differently (CDR is a dynamic query, CREQ is a

static SQL query). If results are better and stable with the CDR search, it means the search CREQ

must be re-written dynamically.

Alcatel-Lucent – InternalProprietary – Use pursuant to Company instruction. 16 | Presentation Title | Month Year

www.alcatel-lucent.comwww.alcatel-lucent.com

alliance alliance performance status - creq régis elling july 2011

Documents

creq module

search creq

performance enhancements

cdr creq

generic specific creq

opencropen generic creq

alliance application

company instruction