soasta cloudtest release notes build...

SOASTA CloudTest™ Release Notes Build 5554.36 May 14, 2012

Table of Contents

Build 5554.36........................................................................................................1

Bugs Fixed ........................................................................................................1

Build 5554.19........................................................................................................2

Bugs Fixed ........................................................................................................2

Build 5554.10........................................................................................................3

Bugs Fixed ........................................................................................................3

Build 5554.3..........................................................................................................5

Test Creation.....................................................................................................5

Change in behavior for HTML parsing affects XPath ....................................5

Test Execution...................................................................................................6

Cloud Provider Account Support for Rackspace (CloudTest Pro) .................6

Setup Rackspace (OpenStack) as a Cloud Provider .................................6

Async Play Start Improvements ....................................................................9

Bugs Fixed ......................................................................................................10

Build 5483.23......................................................................................................14

Bugs Fixed ......................................................................................................14

Build 5483.3........................................................................................................18

Test Creation...................................................................................................18

Introducing TouchTest™ .............................................................................18

Welcome Page Improvements ...................................................................19

Double Click to Open ..................................................................................20

Double Click to Open in Result Details ....................................................20

Double Click to Open in Analysis Widgets...............................................23

Double Click to Open in Composition Editor ............................................25

Double Click to Open in Clip Editor..........................................................25

User-Friendly Display of Properties & ISSEs...............................................26

Clip Editor Improvements ............................................................................29

Using the New Record Toolbar Button ....................................................29

Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

Browser Recording in the New Clip Editor ...............................................31

Record Browser (new window) .............................................................32

Record Browser (inline frame) ..............................................................33

Open in Composition Button Now a Drop-Down......................................35

Open in Test Composition ....................................................................35

Play in Test Composition ......................................................................36

Debug in Test Composition ..................................................................36

List View Improvements ..............................................................................37

List View Repeats ....................................................................................37

Breakpoints in List View Gutter.............................................................39

Renamed Column Headings.................................................................39

Composition Editor Improvements ..............................................................41

Locations in a TouchTest™ .....................................................................42

Locations in an HTTP(S) Test..................................................................43

Track Properties, General Tab.................................................................44

Default Track, General Tab Properties .................................................45

Adding Multiple Locations or Distributions to a Track..................................46

Adding Locations To a Single Distribution ...............................................47

Adding Locations To Distinct Distributions...............................................48

Changes to Repeat Method Drop-Down..................................................48

Using Repeats with Multiple Track Locations ..............................................50

Seed Data Improvements............................................................................52

Tying Seed Data to a Virtual User(s) .......................................................53

Server Allocation......................................................................................54

Separate Slice Distribution ...................................................................54

Single Slice Distribution........................................................................55

Data Section ............................................................................................55

Column and Row Count........................................................................55

Row Traversal Types............................................................................56


EOF Action ...........................................................................................56

Row Representation .............................................................................57

Assignment of seed data values to individual properties .........................58

Seed Data Result Events.........................................................................58

Multiple Custom Properties...................................................................59

Mobile App Targets .........................................................................................60

Creating a Mobile App Target ..................................................................60

Test Execution ....................................................................................................63

Tenant Recognition .....................................................................................63

Test Analytics......................................................................................................65

Result Details Improvements.......................................................................65

General ....................................................................................................68

Composition, Band, Track, and Clip Level............................................68

Target Level..........................................................................................68

Message / Action Level.........................................................................69

Waits and Validations ...........................................................................69

Custom Properties ...................................................................................69

Input.........................................................................................................70

Target Demographics ...........................................................................72

Waterfall Chart ............................................................................................73

Creating a Custom Waterfall Dashboard ..............................................74

Miscellaneous..................................................................................................75

Automatic Update Improvements for EC2 ...................................................75

Jenkins Plug-in Error Message Improvements ............................................75

Support Browsers........................................................................................75

Bugs Fixed ......................................................................................................76

Build 5554.36

Bugs Fixed 30083: Functional test "stops recording" on site A problem occurred doing browser recording on cross-domain sites due to a “Cross-domain frame lock” in use in Firefox 8+ browsers. Due to this cross-domain frame lock, a cross-domain window would not get registered, and therefore, wouldn’t record as expected. As of this release, when trying to determine whether a new window should be registered for recording or not, CloudTest will try and determine if this window falls under the domain of the original recorder window (the one under which recording was started). In some cases where this doesn’t succeed, it will be necessary to click to stop recording, and then resume recording using the respective URL directly (by entering it when the “Enter starting URL for recording” box appears on resume). 30082: Functional recording giving "Invalid URL" for valid URLs A properly formed URL with parameters would produce this error in the Clip Editor, Enter starting URL for recording box. 29994: Duplicate property created in clip A clip that had custom properties whose names were equivalent once special characters were stripped became corrupt after being saved or opened in a composition. 29985: Unable to record login For the latest Mac OS X Java version, a problem could occur with the Origin header being in the restricted headers list that would result in recording failure. As of this release, Conductor will set the "sun.net.http.allowRestrictedHeaders" JVM system property to "true" to turn off the HttpURLConnection blacklist. 29779: Graph widgets when exporting CSV show time as 16:01:00 Results exported to the CSV format unexpectedly failed to post the correct time format when “Relative time” was used. Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

1

Build 5554.19

Bugs Fixed 29963: Re-starting environment with grid up results in error during result creation This "could not create result" grid error occurred in a grid on an Environment that had been brought down and then brought back up while the grid was up. 29882: Rackspace OpenStack starts two copies of JBoss An attempt to use Rackspace OpenStack unexpectedly resulted in a redundant start of JBoss. 29857: Deadlock if there is an error while stopping a Composition containing Mobile Apps If a Composition actively playing Mobile Apps was stopped, and an error occurred in the Mobile App processing during that stop, sometimes a deadlock would occur, causing the Composition to remain in the "Stopping" or "Aborting" state forever. 29783: Error executing database query With a deeply nested result that had filters applied, it was possible to get this exception in the Hierarchical Collection Analysis. 29669: Error Report Submission should be background processing Error report submission could take too long in some situations. As of this release, CloudTest wil pass error report submission into a background thread, thus permitting the user to continue without being held up. 27810: Tracks with 1 user ignore leading Start time In prior releases, CloudTest would ignore leading Start times when repeats were set to 1. As of this release, those lead times are utilized as set.


2

Build 5554.10

Bugs Fixed Activity Map Widget fixed The Activity Map widget has been upgraded to utilize the Maps API v3 for this release. 29846: Error executing database query An error occurred while trying to filter the Event Log to display error events when a given container element didn’t exist in the dimension. This fix also includes a new feature to export Events to CSV. 29821: Support for Rackspace OpenStack API fixed Launching Rackspace OpenStack servers from a grid would fail with the error "Server returned invalid JSON." 29806: Dashboard not showing correct result With the async start of tests, the dashboard, when switching from the Edit tab to the Play tab, would sometimes show the previous result instead of the new one because the new result had not been created yet. As of this release, the play tab is only switched to after we get a result ID. 29824: Uncaught ReferenceError: ConcertoLayout is not defined; JS line 101 This error would occur during an attempt to export chart data to. 29815: Error switching to the debug tab while a composition is playing This script error would occur clicking the Debug tab for the first time while a composition played. 29781: Grid wizard comes up disabled for existing Grids Opening an existing grid in the Grid Manager resulted in a disabled grid (i.e. the Deploy Instances button was disabled as well as the navigation arrows).


3

29774: Results Viewer not up to date when Composition ends In some cases, after a Composition completed, the Result Details would present an incomplete display of events. This fix takes care of the missing events and status, etc.


4

Build 5554.3

Test Creation

Change in behavior for HTML parsing affects XPath As part of a fix introduced for Build 5483.23, the underlying HTML parser used by CloudTest was upgraded to a new version that now imposes the TBODY tag wherever TABLE is detected. This tag is already commonly imposed by some browsers, and is technically a required HTML tag, so this is a familiar issue for CloudTest users doing browser tests whenever specifying XPath in Property Sets, Validations, of when using the Browser Action Editor’s Locator tool. SOASTA recommends that a given XPath or JS DOM be specified so that it'll work with or without a TBODY in the target XML. This can be accomplished in a number of ways. For example, by using an XPath that selects past the TBODY, such as: //td[@id='xxx']

Or, in cases where it’s desirable to select on the table (or parent), and on a child: //table[@foo='bar']//td[2]

As is the case whenever specifying XPath, if an XPath doesn't allow for a TBODY, it's the path that's wrong, and not the DOM.


5

Test Execution

Cloud Provider Account Support for Rackspace (CloudTest Pro) As of this release, CloudTest Pro users can create Cloud Provider Accounts for the Rackspace Cloud (OpenStack).

By entering valid cloud provider account information for Rackspace, locations are automatically created and can then easily be used to provision servers via the Grid Manager.

Setup Rackspace (OpenStack) as a Cloud Provider Use the following steps to enter your valid Rackspace credentials as a CloudTest Pro/Enterprise Cloud Provider Account.

1. Select Central > Cloud Provider Accounts and then click New to launch the Cloud Provider Account dialog box.

2. Enter a name for the account. This name is used in locations and also appears in the Cloud Provider Account drop-down in the Grid Manager.

3. Change the “Type” drop-down to “Rackspace Cloud (OpenStack)”. The dialog box changes to reflect the available Rackspace data centers (e.g. a tab for DFW1 appears).

4. Check the box for the first Rackspace cloud site this account uses and then enter the Rackspace Account Name and API key, and then enter the API key a second time in the Confirm API key field. Repeat this step if other Rackspace data centers will be used with this account.

5. To find your API key, a. Log in to the Rackspace Cloud management portal.


6

b. Click the Your Account sidebar link, and then API Access in the sub-menu that appears.

c. Click the "Show Key" button (or "Generate New Key" if you don't already have one).

d. Note your username, and copy your API Key. 6. You can optionally configure an access control list to restrict which

CloudTest users are able to launch Rackspace servers with this account. This step is not required if you are the only CloudTest user that will be launching servers. To configure an access control list:

a. Select the CloudTest user you want to grant access to in the drop-down at the bottom of the dialog.

b. Click the Add button. c. A new row will appear in the list above the drop-down. d. To allow the user to view the Cloud Provider Account object (but

not necessarily use it), check the Read box. Note that even with this box checked, the user will not be able to view secure information (such as the Rackspace shared secret).

e. To allow the user to modify, delete, or re-name the Cloud Provider Account, check the Write box.

f. To allow the user to launch new servers using this Cloud Provider Account, check the Use box.

g. Repeat steps a-f for each CloudTest user that you want to grant access to.


7

7. Click OK to complete configuration of the Cloud Provider Account. The

account information will be validated by issuing a call to Rackspace. If the credentials are invalid, a warning will be shown. You can choose to go back and correct the credentials, or continue.

8. After the Cloud Provider Account is saved, a new Location for each Rackspace data center will be automatically created, if it does not already exist.

This location must be specified during Grid Manager configuration of grids For more information about Locations, refer to Using Locations.


8

http://cloudlink.soasta.com/t5/Knowledge-Base/Using-Locations/ba-p/384

Async Play Start Improvements This release includes back-end Async Play Start improvements that will result in the following user experience improvements: (1) greater reliability and better reporting of problems during Composition start, especially with multi-server environments, and (2) more status information during play start (this will result in additional Status Indicator messages such as “Creating Result”) and (3) more status information during Composition termination.


9

Bugs Fixed 29756: Playing back again leaves Result Details in this state Result Details sometimes did not show any data on subsequent test runs depending on timing. 29677: Multiple text entries with the same PlaceHolder don't record proper locators Multiple text entries with the same PlaceHolder didn't record proper locators. 29579: Cannot get back to real time mode After making a selection in the Result Details, Navigation Tree, attempts to return to “real time” mode would prove unreliable. 29560: Cannot select clip elements from the tree while playing back In some cases, attempts to select a clip element in the Result Details, Navigation Tree would fail if the composition was playing. 29550: Support record and playback of logging in by hitting the Go/Done iOS keyboard button This case adds support for record and playback of logging in by hitting the Go/Done iOS keyboard button. 29537: iPod screen shot is squished Screenshots produced as a result of the Output, captureScreenshot, were incorrectlu constrained in some cases. 29536: Ability to simulate (and record) Enter key

A given mobile app with a Login screen didn’t use a “Submit” button, but relied upon the Enter key instead. This fix adds new action called "keyboardReturn" rather then recording a type with an ascii code such as \13. 29524: Unexpected token e An error occurred when Result Details was showing a result that had been deleted. A useful error message will now display in this circumstance. 29522: A result with ID 'ERROR: relation "eventfact2958" does not exist This error would occur if a result from a Draft Composition was open and the Cover Flow was in use and a second user closed the Composition Editor, inadvertently deleting the draft. A user-friendly error message now handles this circumstance.


10

29511: not able to use TouchTestIDs In some cases, valid TouchTest IDs on a given button were not captured as expected during recording. 29402: Invalid argument This error likely occurred when drawing cross-hairs in a given chart and the underlying action had coordinates of “NaN”. As of this release, CloudTest will not draw cross-hairs if there aren't valid coordinates. 29222: a.GetItemFlow().Size is not a function; JS line 25 While watching a result in “real time” mode, the user double-clicked a Target to open it in the editor, and this error resulted when the view was switched back to the Dashboard tab. 29009: Typed in zip code not playing back A numeric zip code entry interface in a mobile app didn’t play back as expected. 28854: Import doesn't work in Google Chrome An attempt to upload an import file in Google Chrome 18 failed due to some change in Chrome. 28803: attempt to run compile-and-go script on a cleared scope; JS line 891 This error occurred running a script on cleared scope. As of this release, this error is ignored. 28763: Cannot read property 'uniqueIDPrefix' of null Additional null checking code has been added to detect further occurrences of this script error in the Dashboard Edit panel. 28756: com.soasta.common.xml2.XmlUtilities.transformDocFromFile An error occurred while creating a new Mobile App object. 28699: Clip with nested mobile clips does not offer device choice The Location drop-down didn't display the available Device Agents when a clip contained nested clips. Nested mobile targets are now handled by overriding them via their Device Agent property in the Composition Editor, Property Sets tab for the given target. 28638: this.__el_crosshair is undefined; JS line 13 An error occurred drawing a crosshair on a chart, likely due to an unknown or undefined coordinate. 28621: Rackspace teardown fails if server was externally deleted A change in the Rackspace API wherein a response code changed from 500 to 404 resulted in this error. Both codes are now handled.


11

28615: The referenced monitorspec '276' does not exist An error occurred when a deleted or invalid monitor was still in use in a widget. As of this release, we don't allow invalid/deleted monitors to be used in widgets. 28570: SCommand fails with NullPointerException if credentials are invalid If the credentials for SCommand were invalid, then a NullPointerException resulted instead of an "Invalid credentials" error. This also affected the Ant tasks, which in turn affected Jenkins. 28559: ISSE insertion via Property Chooser always inserts at the end in IE8 ISSE insertion didn’t work as expected in Internet Explorer 8 and 9 versions. 28275: Should be able to get back to the Property Chooser dialog from the blue ISSE oval After using the Property Chooser dialog box to create and place an ISSE. However, double-clicking the resulting blue-bordered ISSE would not bring the Property Chooser back up with the selection displayed as expected. 28240: Report Error – CloudTest Lite An attempt to create a simple report failed in CloudTest Lite only. 27437: Dependencies tab needs to show full path The Dependencies tab for a given test clip didn’t show the full path to compositions in which it was in use, make it difficult if not impossible to determine the actual dependencies (for example, the composition might exist in multiple folders with the same name). The full path is now shown. 25726: EC2 server list setup fails if there is an extra "Multi-Server Environment" folder EC2 server list setup failed if there were additional system monitor folders. 22685: Executive summary doesn't show grey box that's in the original source docx A generated report didn’t correctly reflect all the layout specifications of its report template.


12


13

Build 5483.23

Bugs Fixed Charts Not Rendering in Internet Explorer 8 Some charts unexpectedly failed to render when shown in Build 5483.3 in Internet Explorer 8. 28771: Problem with MakeAppTouchTestable The Make App TouchTestable utility would fail in projects whose main file included no header files, and also if the AppDelegate file was named ApplicationDelegate.mm. This is now handled. 28643: Xpath fails and causes user to restart CloudTest Lite There were problems parsing some complex HTML responses when applying XPath against the HTML for Property Sets or Validations, that caused the XPath to not match when it should. 28642: Central > Mobile Apps list, Owner column empty The Central, Mobile Apps list, Owner column was not populated with any values as expected. 28636: Result import fails An attempt to import data exported from an build didn’t succeed as expected due to a missing column for results. 28619: Choose a Device Agent box pre-selects app but not device While prior selection was in effect for the mobile app a user had previously selected, the Device Agent was not also selected in the Choose a Device Agent and Mobile App box. 28618: Selenium script import fails if there are invalid XML characters in the Open command A selenium script import would fail if Open contains invalid XML. For example, in the form of URI=xxx.


14

28610: Tapping causes too many taps Tapping in a mobile app would capture multiple taps instead of the one expected tap. As of this release, only one tap per event (as opposed to per gesture recognizer) is recorded. 28582: Exception: java.lang.NullPointerException When de-nesting a container, we could find that an intermediate container is not yet in the Results Service. 28581: waitForElementEnabled mismatch between arguments & help text The right panel help text for the waitForElementEnabled action indicated support for a pattern argument, but the argument in question was not included in the Waits form. 28579: waitForElementPresent documented arguments The right panel help text for the waitForElementPresent action indicated support for a pattern argument, but the argument in question was not included in the Waits form. 28570: SCommand Fails with NullPointerException if credentials are invalid If the credentials for SCommand were invalid, then a NullPointerException would result instead of an "Invalid credentials" error. 28552: Missing help text The reference text was missing for all of the orientation actions. 28551: Inconsistent icons in Results The Widget Selection Panel, Dashboard Data section incorrectly showed a Success (green checkmark) icon and a Fail (red X) icon for two identical runs of the same mobile app test that had identical results. 28545: Unexpected token at Flipper.init The Error dialog didn't capture an underlying error for display as expected. 28541: Clip: 'TC Tag Creation #1' appAction6 doesn't happen TouchTest recording wouldn’t recover after a problem with identical locators on a page occurred. For example, where more than one feature on a page had ‘text=Hashtag’ as its Locator. As of this release, the ability to record locators with an index such as text=Hashtag[2] has been added. 28540: Nothing is recorded after clicking 'Feedback' The issue here is that we call the "type" action as a pending action. So, unless we get the notification that type action has been completed, all the subsequent recorded actions are sent to the pending actions queue. In the case of a given field in a mobile app, the “type” action never finished (or the type completion notification never comes through) and thats why no more actions are sent to


15

CloudTest. As of this release, hiding the keyboard now cancels the pending action so that recording can resume. 28524: Error executing database query This problem occurred due to invalid SQL being generated when applying a filter on the Fundamentals rate metrics. 28477: The device agent has timed out message should go away when you click on the record button again After a Device Agent timed out, thus producing a Clip Editor message as expected, clicking Record after resolving the agent issue wouldn’t result in the resumption of recording as expected. 28472: TouchTestDriver does not retry failed network requests Once a network request failed, the request was not retried as robustly as expected. As of this release, we now retry up to 5 times if there is a network error. 28469: The device agent has timed out In prior releases, TouchTest Agent would receive a server task to launch the mobile app on the device and then go into the background; however, before this occurred, one or two server tasks would be intercepted by TouchTest Agent that were intended for TouchTest Driver, and this connection timeout would result. 28452: Type not recorded in the mobile app After entering edit mode in a given mobile app, the subsequent “type” action was not captured as expected. As of this release, we are now listening for UITextViewTextDidBeginEditingNotification and for UITextViewTextDidEndEditingNotification. 28371: TouchTest Agent should reject devices w/ iOS earlier than 5.0 In earlier versions, TouchTest Agent didn’t clearly indicate when an unsupported iOS version was encountered. 28054: UIPageControl doesn't record and playback In earlier versions, TouchTest didn’t record or playback the UIPageControl used in some mobile apps.


16

27129: App action “type” in the User ID field fails In earlier versions, TouchTest recording didn’t handle cases where a “type” is a combination of clear, sendKeys and submit. So that, when the submit sent "\n" to the text field it would the text that was just added with sendKeys. This is now handled. 26967: Seed Data object not handling '&' correctly Entering an ampersand in a Seed Data row would incorrectly result in its hex code equivalent (&) being inserted into the test. 26636: Attempting to duplicate a Playlist yields internal error Using a right-mouse click in Central > Library to Duplicate a Playlist would result in this "Internal error". 26633: Can't open Playlists from Library view In prior release in Central > Library, Playlists appeared in the list on the right. However, neither right-mouse-Open nor double-click would open them. 26491: Reports created in Gran Cabrio cannot be opened in MS Word 2007 Reports generated from Gran Cabrio (5303.x) builds wouldn’t open in Microsoft Word 2007 as expected. 25862: Results Databases should allow teardown after abnormal termination In prior releases, if a database was externally terminated outside CloudTest, the Results Database dialog’s Tear Down Instances button would remain inactive. As of this release, the Tear Down Instances button will become active if an externally terminated database is detected. 25840: Volume list should be filtered by CPA In prior releases, the Results Database dialog box’s Volume list included every volume in the selected Location, regardless of which CPA owned it. As of this release, this list will only show those Volumes owned by the selected Cloud Provider Account.


17

Build 5483.3

Test Creation

Introducing TouchTest™ CloudTest’s new TouchTest™ technology delivers, for the first time, complete functional test automation for multi-touch, gesture-based mobile applications. TouchTest™ technology delivers fast, precision functional testing while increasing the stability of automated tests across releases. CloudTest® can launch mobile apps that are under test through a lightweight software agent, SOASTA TouchTest™ Agent. Devices can be dedicated to testing in the lab, used as part of an external test, or crowd-sourced as part of a high volume, globally distributed test. Using SOASTA TouchTest™ Driver, which is compiled into the app under test, support is provided for recording, playback and validations of user actions within any iOS 5.0 device including iPhone, iPad, and iPod Touch. There is no need to jailbreak the iOS device and the device can be untethered. Along with release notes, this release includes a complete TouchTest Tutorial TouchTest™ with complete end-user instructions for test creation, as well as a TouchTest™ Developer Guide where important developer integration steps are presented. CloudTest User Interface Improvements for TouchTest™ include:

• Welcome Page improvements that facilitate TouchTest™ integration by providing relevant steps as a wizard.

• Clip Editor Improvements that facilitate inline TouchTest™ recording and editing.

• Composition Editor Improvements that facilitate TouchTest™ playback.


18

http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Tutorial.pdf?v6


http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Developer_Guide.pdf

Welcome Page Improvements The CloudTest Welcome page (Central > Welcome) has a new, lighter, and easier-to-read look and feel for this release. Additional wizard steps are available to TouchTest™. Also, new TouchTest™ items have been added to both the Downloads and Documentation & Samples sections.

The first page of the TouchTest wizard present three simple first steps to make a mobile app TouchTestable, to install that app on a device, and to run a device agent on that device using Safari browser.


19

The remaining three steps of the wizard register that device (via the Device Agent in Safari), and proceed right away to clip creation, and finally, compositon creation.

Double Click to Open This release adds “Double Click to Open” support for items in the Composition Editor and Clip Editor, Analysis widgets, and in the Result Details dashboard, where its addition will have a dramatic impact on test editing due to the ability to open any part of a test clip precisely at the selection point. This ability to open the clip at an exact location will make test editing far easier. Double-click support that was provided in the Composition Editor in prior releases is now more widely in effect throughout. For example, in CloudTest linked scripts, nested clips, and in result items in both the Navigation Tree and Cover Flow.

Double Click to Open in Result Details Double-clicking from within Result Details is comprehensive: every part of the test, including the composition itself, can be opened in a new tab.

Click a clip to open it in a new Clip Editor tab with no item selected (shown below) in the parent clip.


20

Double click any clip element to open it in its respective editor at the same location as displayed in Result Details.

For example, double-clicking a selected page (shown below) will open

the parent clip in the Clip Editor with the page selected (as with the page shown above and below).


21

Items in playing compositions can be selected and then double-clicked while the test is streaming.

Whenever an item within a container is opened the container is also expanded.


22

Double Click to Open in Analysis Widgets Double click any item in a composition hierarchy in an Analysis widget to open it in a new tab in its respective editor. Double-click to open is provided in all Analysis widgets whenever a result is selected, whether in a stand-alone dashboard or a dashboard displayed from within the Composition Editor tabs. All analysis widgets are supported, including Composition Analysis, Clip Analysis, Clip Element Analysis, as well as collection and property value analysis widgets. No action is taken when an item in a widget displays an item other than a composition element (i.e. a URL). In the Composition Analysis widget, items can be opened in their respective editors.

Click the Composition, Band, or any Track to open the composition in the Composition Editor.


23

Drill down and click any clip or clip element to open it in a new tab in the Clip Editor.

When you do so, the new Clip Editor tab opens on top and the clicked item, in this case the “knowledgebase” page (shown above and below) container, is selected.

When a script in an Analysis widget is clicked, the Script Editor opens (rather than opening the clip in the Clip Editor at the insertion point for that script).


24

Double Click to Open in Composition Editor Double-click actions were already supported in the Composition Editor for clips in tracks (double-clicking a clip in a track opens the Clip Editor). As has been the case for many releases of CloudTest, double-clicking a clip in the Composition Builder adds it to first or selected track.

Double Click to Open in Clip Editor Double click a Nested Clip to open it in a new tab was introduced when Nested Clips were introduced.


25

The new tab is on top after a double-click.

User-Friendly Display of Properties & ISSEs Properties, property paths, and In Situ Subsitution Expressions (ISSEs) are now presented as blue rounded boxes throughout CloudTest. They are displayed as “bubbles” also in the lower panel. There is a new, lower-panel Expert button that allows users to expand the bubble to toggle between full text and bubble view). This improves both their visibility in the workspace and their accessibility. Simply double-click or hover (single-click) to inspect or edit their data and info. Double-click any property box to open it in its appropriate editor.

Placed properties now appear as rounded blue boxes throughout the CloudTest user interface making them easier to find later. For example, the Clip Editor, List View, Parameter 2 column shown above has a property, username, placed into its Query String.

Property boxes also appear wherever Property entry fields exist, such as in If-Then-Else or Switch containers shown in the Clip Editor.


26

Property boxes are also displayed in the Composition Editor whenever a property field or value is in display. For example, whenever a Repeat By Property repeat method is specified for either a track or clip repeat. When Repeat By Property is first selected from the Repeat Method drop-down, it will display [not specified] until its value is specified via the Property Chooser.

For more complex ISSEs, user-friendly display means hiding the underlying ISSE syntax in favor of a clickable, and more readable format as shown in the Group 1 container below, where a user-friendly ISSE is deployed.


27

A closer view shows that the user-friendly version of this expression has the same rounded-blue border as the other items above.

To inspect the full expression syntax for any user-friendly ISSE, double click to inspect it in the lower panel.

Note that the Locator field above shows the concatenated expression. To show the full ISSE, toggle the Expert button to the right of the field where the ISSE has been entered.


28

Clip Editor Improvements As part of the introduction of TouchTest™, the Clip Editor now provides the ability to record mobile app actions inline, just as browser actions are recorded inline. Target creation for mobile apps and WebUIs now occurs on-the-fly, the underlying targets are accessible when creating or modifying system or custom properties.

Using the New Record Toolbar Button The new Record drop-down is now prominently part of the main Clip Editor toolbar. To begin recording, simply click the Record drop-down and choose the recording type (e.g. Record Mobile App).

As of this release, the lower panel tabs are all minimized by default in order to increase the usable workspace in the editor. When a new Untitled Test Clip is created, the Clip Editor launches with a Record prompt such as the one shown above.

1. Once ready, click the Record button and then select Record Mobile App.

TIP: Browser recording options, which appeared in all browsers in

prior releases, now appear only in the supported Firefox.


29

The Choose a Device Agent and Mobile App wizard appears. Choose the Device Agent and Mobile App from among those configured and click Record on the lower right.

As a result of selection above, a target is created. Target creation via a separate wizard is no longer required.

You can discover the name of this automatically created target by looking at the Clip Editor > Messages Actions lower panel, Included Targets field.


30

• To change either the Device Agent or Mobile App click the corresponding

Target Editor, Section Info, Settings field for the given item.

Browser Recording in the New Clip Editor When in Firefox, if the necessary Browser Recorder extension has yet to be installed, an additional drop-down command appears.

Otherwise, two new browser recording commands appear, Record Browser (new window) and Record Browser (inline frame). These commands replace the Pop in/Pop out interface used in prior releases. All recording is performed in either a new window or inline frame within a given session.


31

Record Browser (new window) 1. Choose Record Browser (new window) and then enter the starting URL.

If the URL entered is already in a target, then that target is used. Otherwise, a new WebUI target is created.

2. Click Record to proceed with your recording in a new window. When you do so, the target URL opens in a new Firefox window and the Record button indicates recording is underway.

3. Perform the planned browser actions in the new window. Actions are

added to the Clip Editor as you go.


32

4. Click the Record button (in the original window Clip Editor tab) to stop

recording.

Record Browser (inline frame) 1. Choose Record Browser (inline frame) and then enter the starting URL.

If the URL entered is already in a target, then that target is used. Otherwise, a new WebUI target is created.


33

2. Click Record to proceed with your recording inline in the Clip Editor workspace.

When you do so, the target URL opens inline and the Record button indicates recording is underway.


34

3. Perform the planned browser actions in the workspace. Actions are added to the Clip Editor as you go.

4. Click the Record button to stop recording.

Open in Composition Button Now a Drop-Down The Clip Editor toolbar Open in Composition button is now a full-fledged drop-down that includes three commands.

Open in Test Composition • With a test clip open in the Clip Editor, choose this command to place the

clip into Track 1 of a new, draft Untitled Composition. Copyright 2012. CloudTest is a registered trademark of SOASTA, Inc. and/or its affiliates. Other names may be trademarks of their respective owners.

35

When you do so, CloudTest switches automatically to a new Composition Editor tab. The composition is not loaded and play doesn’t commence automatically.

Play in Test Composition • With a test clip open in the Clip Editor, choose this command to place this

clip into Track 1 of a new, draft Untitled Composition, which will be loaded and automatically played with the Play tab visible.

Debug in Test Composition With a test clip open in the Clip Editor, choose this command to place this clip into Track 1 of a new, draft Untitled Composition, which will be loaded and automatically played in “Debug mode” with the Debug tab visible.


36

List View Improvements The Clip Editor, List view improvements in this release include the display of repeats in rows instead of in a vertical column, as well as showing breakpoints in the gutter, and renamed column headings.

List View Repeats The Clip Editor, List view now displays repeats as a separate line-item in the list above the clip element that has the repeat. This replaces the Repeat column in prior releases. Use Repeat By to specify a specific number of repeats for a given item. The Repeat By will appear above the item to which it is attached.

List view shows user-friendly paths for properties whenever they occur in List view. Property fields are shown in containers, as well as when a property is specified for a repeat (shown below). Changing the Repeat Method drop-down shown above from Repeat By to Repeat By Property introduces the Property Chooser dialog box. When a repeat method that requires a property to be set is first selected, it will show as [not specified].


37

Use the Property Chooser to make a property selection from among existing properties or to define a new property.

View property detail info in popups via single-clicking in or hovering over the given item.

Additional repeat types include For Each (formerly Property: For Each)

and While (formerly Property: While).

The Continuous repeat setting found in prior releases has been renamed Infinite.


38

Breakpoints in List View Gutter The List view also now shows breakpoints in the gutter. In the prior release, these breakpoints were displayed in the column to the right of the clip element name.

Renamed Column Headings Column headings in the List view have been renamed for this release. The default column sort order is Name, Operation, Parameter 1, Parameter 2, [Scope], Type (for Containers), and finally, Target Name.

Right-click the column header to display a list of columns to show. Uncheck any item in the list to suppress its display in List view.

User preferences set here are honored for all clips until changed again. Columns can also be reordered using drag and drop. For example, the Type column, second-to-right by default, has been placed in the position beween Name and Operation in the screenshot below.


39


40

Composition Editor Improvements Composition Editor improvements include a new track distribution user interface that supports multiple track distribution. A distribution is a grouping of one or more locations within a given track that share repeat settings. In the Composition Editor, the Track drop-down has a new look and feel that also reflects new underlying functionality. Locations are now more identifiable, with sections more visible in the list, and are selected by radio button.

• Locations can now be grouped in a distribution that shares repeat settings,

or, configured so that each location can have its own distribution with its own repeat settings. Repeat settings can be enabled or disabled per distribution.

If lower-panel location settings are already in effect, the Locations drop-down indicates so. Advanced settings can be specified in the Track, Property, General tab in the lower panel.

Note: Right-click a track and select Edit Properties to easily access

its lower panel settings.


41

Locations in a TouchTest™ TouchTest Agents appear in the Locations drop-down in TouchTests.

For TouchTests, the listed locations are the device agents to specify for test playback. Select the Connected target device from the Locations list to proceed (if it is not Connected, use Safari to login from the mobile device). This selection can also be made at the clip level by clicking Select at Clip (shown above). Once Select at Clip is chosen, the clip(s) in the track will display the Location drop-down.

If multiple clips are in the given track, each displays the Location drop-down.


42

Locations in an HTTP(S) Test • When a track with only an HTTP(S) clip is selected, the Location drop-

down familiar from prior releases appears (in its new context).

• When clips with a Mobile target(s) & a Server target(s) are combined into

one track, the Location drop-down refelects that both devices and servers are available to be configured.


43

If Select at Clip is selected, then the Mobile Device can be selected at the clip level on the given track.

Track Properties, General Tab For this release, the Track properties General and Repeat tabs have been merged into one General tab to accommodate all track properties related to server allocation, server location, distribution, and repeats.

• Former General tab settings appear at the top of the newly combined tab: Track Name, Percent Allocation, and Errors should.

• In addition to the former General tab, Maestro and Copy Count fields, the

merged General tab includes all of the former Repeat tab settings.

• These merged settings appear in the new General tab, Maestro Settings section (below) and can now be applied to the entire track, if grouped, or be individually set per location.

• Each row in the Maestro Settings form has its own Add/Delete icons, whether within or between distributions.


44

As in prior releases, repeat settings are disabled by default and appear in the tab only when a given Enable Repeat box is checked.

Default Track, General Tab Properties When a new composition is created, the following default track properties are set:

• Track Name: Track 1

• Percent Allocation: inactive (this field is inactive until a clip is added to the composition)

• Errors should: fail the band (this is the default for all composition elements)

• Maestro Settings (section): o Location: Any Location o Copy Count: Inactive by default (make Copy Count active by

selecting a Dedicated Load Server via the Location drop-down) o Enable Repeat: repeat settings are disabled by default (check the

box to enable repeat settings for the given Location(s) or Clip(s).


45

Adding Multiple Locations or Distributions to a Track The track properties General tab offers two approaches to track distribution—Locations sharing repeat settings can be grouped into a distribution (represented visually as an inner (darker gray section, with inner icons) within the Maestro Settings form,

With Distributions, one track can be configured per functional unit of clips, and it can run on as many servers and repeat in as many different ways for different sets of servers as required. In addition to providing a quick way to propagate repeat settings within a test composition; distributions will improve overall performance in some tests where many tracks slow test load times and complicate the interpretation of test results.

Note: The Delete icon(s) are inactive if only one item is listed in the distribution or in the overall track.


46

Or, Locations can be added to a new distribution (lighter gray, outer icons) that has its own repeat settings (shown above).

Adding Locations To a Single Distribution Locations can be grouped together into one distribution by using the inner (leftmost) Plus icon. Use this in order to have repeat settings that apply to all the locations grouped in that distribution.

1. Click the green Plus icon in the inner form (darker gray) to add a new Location to the given section.

When you do so, the Location is added to the distribution (inner form). One Enable Repeat checkbox is provided to apply repeat settings to the locations grouped in this section.

2. Repeat to arrive at the desired number of Locations in this distribution. 3. Next, select a Location, or server, for each row.

If Enable Repeat is checked, those settings will apply to all locations in this distribution.


47

Adding Locations To Distinct Distributions In this setup, each distribution has its own repeat settings. This is accomplished by by using the outer (rightmost) Plus icon to add Locations to a new distribution—in order to have repeat settings that apply to each location(s) grouped in a distribution.

1. Click the green Plus icon in the outer form (light gray) to add a new Location section to the track. Like with the top section, any section can have one or more locations defined.

2. Repeat to arrive at the number of Locations to use. 3. Next, select a Location, or server, for each row. 4. If Enable Repeat is checked, those settings will apply individually to each

distribution.

Changes to Repeat Method Drop-Down Fields in the Repeat Method drop-down have been renamed for this release to make them more user-friendly. Formerly, the top two listed methods were Fixed and Property: Fixed both of which were used to set a fixed value for a given repeat, one of which called a property to do so.

• The Fixed repeat method has been renamed Repeat By for this release.

Repeat By gets an integer value in all its possible contexts (e.g. whether parallel or serial repeats are in use). This value can also be set on the track surface and such changes are reflected in the lower panel (as shown above).


48

• The Property: Fixed repeat method has been renamed Repeat by Property for this release. This method calls a property. If that property is not yet specified, this is shown.

For Sequenced Band compositions, only parallel is enabled, unless Renew Parallel Repeats is checked. Checking Renew Parallel Repeats activates some fields below including the Maximum Duration option, as well as Paced and Inter-repeat delay in the Advanced Options section.

Note: Properties are now displayed as blue highlighted objects with rounded corners rather than by their path as in prior releases. Use the Property Chooser as before to select any existing property or to create a custom property on-the-fly.

• The Property: prefix has been dropped from For Each and While, and the

Continuous repeat setting has been renamed Infinite. As in prior releases, these settings are reflected on the track surface.


49

Using Repeats with Multiple Track Locations Locations can be grouped together within a distribution by using the inner (leftmost) Plus icon—in order to have repeat settings that apply to all the locations in that distribution. Enable Repeats must be checked per distribution to be in effect.

Alternately, use the outer (rightmost) Plus icon to configure track locations each with its own repeat settings.

On the track surface, the Copy Count number shows an accurate Copy Count, but it turns into plain non-input text in any additional location (e.g. when there is more than one location in a distribution and one of them doesn’t use Dedicated Load Server).


50

As in prior releases, the Track surface displays information about its current configuration. When more than one distribution is configured in the Track lower panel, the track surface will show [various] as its Virtual Users state. Clicking Locations here will show “Specified in lower panel.”


51

Seed Data Improvements This release includes further refinement of the Seed Data support via the Custom Properties interface introduced in recent releases. Instead of one simple Advanced section to define the seed data property, as well as its slice, and its distribution; new, separate Allocation and Data sections are now provided with more sophisticated server allocation and row traversal types.

As a result of these new settings, more robust random distribution support is available in this release, alongside slicing improvements, and seed data structs (for both Seed Data: Repository and Seed Data: URL--when Column names are in the first row is checked). In prior releases, a single Advanced section consisted of the following fields:

• Maximum rows per server

• Row count

• Column count

• EOF Action: (Wrap, Null, Error, Stop Track and Drain, and Stop Composition)

• Use a Subset of the seed data: (Starting Row, Number of Rows) New Allocation and Data sections with settings to define seed data distribution replace the simpler Advanced section found in the prior release. All of the above settings are present in the new sections and are joined by new settings.


52

Tying Seed Data to a Virtual User(s) It is now possible to pair a given virtual user to a distinct slice of data via the new Property value is tied to the virtual user setting with a subordinate setting, New value per virtual user iteration.

• Property value is tied to the virtual user When checked, all Custom Properties within the same virtual user that reference the same seed data source will get the same single seed data value.

• New value per virtual user iteration When checked, then each iteration of the virtual user will get a new value. In the example above, the two Custom Properties in the two Messages will get the same value within each iteration of the virtual user, but each iteration will have a different value. Note: The top-level Save value for analytics setting was available in

prior releases. In Cloud Test, an instance of a Track is generally used to represent a “virtual user”. Parallel repeat renewal of the Track is generally used to represent “iterations” of a virtual user. If there are multiple references to the same seed data source from different Custom Properties within the same virtual user, each such Custom Property will get a different value from the seed data.


53

For example, if Custom Properties defined in two different Messages in the same Clip both refer to the same seed data source, each will get its own value from the seed data.

Server Allocation The Allocation section mixes a new Server allocation setting with the Maximum rows per server and Use a subset of the seed data settings present in the former Advanced section to arrive at a more powerful way to slice data. The “Server allocation” setting controls how the data is divided among the servers in a multi-server Composition.

• Separate slices of the seed data will be sent to each server Choose this to divide the data evenly among all of the servers, and send a unique slice of the data to each server.

Separate Slice Distribution To cause every instance of a property to get a random value, then set “Server allocation” to “Separate slices of the seed data will be sent to each server” in order to cause different sets (slices) of the data to be sent to each server. Each server then selects randomly from within its slice. For example, for a Composition that plays on 5 servers with the option Separate slices of the seed data will be sent to each server, if the “Maximum rows per server” is specified as 1,000, then if there are at least 5,000 rows in the source then 1,000 rows of seed data will be distributed to each server. However, if the source only contains 600 rows, then only 120 rows would be distributed to each server (600/5). If there are not enough rows of seed data to distribute at least one row to each server, the load will fail.

• The same single slice of seed data will be sent to all servers” is Choose this to single slice data that is then sent to all servers, and each server will be using the same seed data.


54

For example, with this setting all 5 servers would share the same single set of seed data (either 5,000 or 600 rows in the above examples).

Single Slice Distribution To cause every server to randomly select from the entire set of data, when it is not required that every value be used exactly once across all servers, then set “Server allocation” to “The same single slice of seed data will be sent to all servers” to cause the entire set of data to be sent to all servers. All of the servers will then be randomly selecting from the entire range of values.

Data Section The second new section brings together new row and column settings that replace the former row count and column count settings.

Column and Row Count Rows and columns are defined in a new way for this release. Row count is expressed as an integer, while column use is specified either as All or First x.

• As in the prior releases, Row count setting specifies the number of rows of seed data to distribute to each individual instance of the Custom Property. If the value is greater than one, then the Custom Property’s value will be an array value, one entry in the array for each row.

Note: If both “Column count” and “Row count” are greater than one, then the property value will be either a two-dimensional array, where the first index is the row and the second index is the column (if “Array” was chosen), or an array of structs (if “Struct” was chosen).


55

Row Traversal Types The “Row traversal type” setting specifies how each server will traverse its slice of the seed data.

• Sequential

The server traverses through it’s set of data sequentially. The “EOF Action” setting applies and indicates what to do if the end of the server’s set of data is reached.

• Random with replacement

Whenever a new value is needed to fill in a new property value, one is chosen randomly from the server’s set of data. The “EOF Action” setting does not apply, since each selection is just made randomly from the entire set of data for that server, and therefore the server can never “run out” of values.

• Random without replacement

Whenever a new value is needed, one is chosen at random and marked as used. Thus a unique value is chosen each time from the server’s set of data. Since in this case the server can run out of data, the “EOF Action” setting applies.

EOF Action As in the prior release, the EOF Actions specifies what to do if any server reaches the end of it’s slice of seed data during play of the Composition.

• Wrap

The server will start over at the beginning of its slice of seed data.

• Null

The values of the Custom Properties will be set to nulls. If according to the settings the property value is to be an array, it will be set to an array of nulls of the appropriate dimension and size.


56

• Error

The item containing the Custom Property will fail. Error handling will proceed according to the regular “Failure action” settings of the item, it’s parents, and the Composition.

• Stop Track and Drain

The current instance of the item’s parent Track on the current server (and everything in it) is stopped. No new repeats or parallel repeat renewals will occur for that Track on this server. Other instances of the Track on this server that already exist will continue to their normal end, but no new instances will be started on this server. In other words, if using the recommended approach of having each Track represent a “Virtual User”, and if using “Parallel Repeat Renewal” to cause each Virtual User to be automatically replaced when it ends, then this setting will cause no new Virtual Users to be created on this server when its seed data runs out.

• Stop Composition

The entire Composition (on all servers) will be stopped immediately (as if the “Stop” button had been pressed).

If multiple Custom Properties refer to the same seed data source, each such reference can have it’s own EOF Action, they do not all need to be the same.

Row Representation

The “Column count” setting specifies the number of columns to retrieve from the seed data. If the setting is one, the property is set to a single value. If the setting is greater than one, the property value type depends upon the option chosen for the “Row representation” setting.

• If “Array” is chosen the property is set to an array value, with one array member for each column in the seed data. If there are fewer columns in the seed data than specified in “Column count”, then the extra values in the array are set to null. The array will always be the size specified by “Column count”.

• If “Struct” is chosen, the property is set to a struct value, with one member for each column in the seed data. The “Struct” option can only be used when accessing seed data via URL and the “Column names are in the first


57

row” option is chosen, or when a Repository seed data object is being used.

Note that Struct is available with Seed Data: URL only when the Column names are in the first row box is also checked.

Assignment of seed data values to individual properties When the Composition is playing, any time that a new item starts playing, such as a Clip, Chain, or Message, if that item contains Custom Properties that reference seed data, the server on which the item is playing extracts the next available row from the slice of seed data that server has been allocated and sets the Custom Property’s value. Depending upon the settings, this may result in multiple rows being extracted for each Custom Property value.

Seed Data Result Events Various events are logged to the Result that indicate how the seed data was allocated and used. These events can be viewed in the “Result Details” and “Event Log” Dashboard widgets. All such events are logged at the “Composition” level and thus will be present even when the Composition is played in “Load” mode. When play starts, a “Seed Data distribution summary” event is logged that shows a consolidated summary of the slices distributed to each server involved in the Composition:

When play starts on each server involved in the Composition, a “Seed Data slices distributed” event is logged for the server that shows information about the slices distributed to that server:

When play completes on each server involved in the Composition, a “Seed Data usage” event is logged that shows how much data that server used from the slice, as well as whether it was necessary to wrap back to the start of the data (if allowed to do so by the Custom Property settings). In this example, the first slice wrapped twice as the slice size was a single row and a total of 3 rows were allocated:


58

Multiple Custom Properties If a Composition contains references to multiple seed data sources, each server will have a separate, independent slice for each such seed data source. If multiple Custom Properties refer to the same seed data source, they will all extract data from the same single slice (each will get different unique rows from that slice). If multiple Custom Properties refer to the same seed data source but specify different values for “Maximum rows per server”, the highest specified value will be used. If multiple Custom Properties refer to the same seed data source, all references must specify the same “Starting row” and “Number of rows” settings, if those settings are specified. No security (such as Basic Authentication or other user ID or passwords) is supported when accessing seed data via URL. However, Repository Seed Data objects can be encrypted with a password. At Composition load time, you will be prompted to enter any such necessary passwords. If multiple Custom Properties refer to the same seed data source, each such reference can have it’s own EOF Action, they do not all need to be the same.


59

Mobile App Targets This release introduces a new “Mobile/Native” target type to the Target Definition Wizard for use with TouchTest devices. In most cases, this target will be automatically created during Device Agent and Mobile App selection in the Clip Editor. However, mobile app targets can also be created and specified manually. Mobile app target creation can also be achieved via the Target Definition Wizard. Targets created manually can then be associated with the correct mobile device via its Device Agent and a Mobile App on that device.

Creating a Mobile App Target 1. Select Central > Targets and then click New. 2. Select the Native / Mobile App target type. 3. Click Next to advance to the Settings screen.


60

4. In the Settings screen, give the mobile target a name. 5. Optionally, select a Target Image appropriate for the mobile app.

6. Click Next to advance to the Finish screen and then click the Finish button

to complete target creation. When you do so the target is opened in a new Target Editor tab in CloudTest.

Mobile targets are considerably simpler than WebUI targets, since they lack Location Strategies and many of the other Ajax-related fields required for browser-based functional tests.


61

Use the remaining steps to ensure that your target is used for a given Device Agent / Mobile App combination.

7. Specify the Device Agent by name from among those configured. Note: Refer to the TouchTest Tutorial for a complete step-by-step

guide to registering your mobile device for use with the TouchTest Agent.

8. Specify the Mobile App to use from among those configured. Note: Mobile apps are configured by the Mobile Administrator. If

your mobile app doesn’t appear, discuss it with your Admin. Refer to the TouchTest Developer Guide for additional instructions.

9. Check the Terminate app at clip completion box to ensure that the mobile app is returned to its original state at the end of the test (or between clip plays in cases where the clip is used more than once in a given test composition).

10. Save the target after making any changes.


62


http://cdn.soasta.com/productresource/download/SOASTA_TouchTest_Developer_Guide.pdf

Test Execution

Tenant Recognition CloudTest now recognizes a tenant boundary with respect to server selection. Maestro will now only pick servers that belong to the tenant in which the given composition is about to run. This tenant boundary will eliminate some confusion for grid administrators, as well as during Location selection in compositions, as well as in Play Options. The Central > Servers list (and all other lists that show servers) now includes an additional Tenants column as part of the Maestro server’s new ability to distinguish between tenants.

Only eligible servers, based on tenancy, will appear in the CloudTest user interface. For example, such as in the visible list of Locations presented in the Composition Editor’s Play With Options dialog box (as well as Play Options displayed in Central via the main toolbar).


63


64

Test Analytics

Result Details Improvements This release introduces a new, more interactive version of the Result Details Dashboard with animated display during Play, numerous navigation improvements, better organization of information into sections that are interactive and can be minimized/maximized to improve readability, all of which we believe will result in faster analysis. As noted in the Double Click to Open section above, all items in the Navigation Tree and Cover Flow can now be double-clicked. In the case of clip elements, the parent clip will be opened to the selection, thus making it easier to use Result Details in the test editing process.

• The Result Details tree now launches expanded, with the latest container

open, and focus set on the last item (for a playing composition). The data display is updated to that last item as results become available.


65

The next item in the stream will first appear in the Navigation Tree (e.g. AppAction8) just prior to the focus changing to it. Then, the lower panel sections change to match the context of the newly selected item. Once play is completed, the last clip (where more than one exists) remains expanded. The Cover Flow has also been redesigned and is wider and dynamically sizes based on the buffered items shown.

• The Cover Flow also keeps the last item in focus, unless some user

interaction overrides it. Hovering over an item will pop it up to show the operation.

• Lower panel section display changes based upon the composition element selected.


66

At the Composition level selection show below, the General section, and Custom Properties appear. Additional sections will appear where applicable. Sections can be maximized and minimized easily by clicking the top-right section button.

For example, additional sections relevant to TouchTest™ AppAction’s such as those shown above will now appear in the Summary tab based on the context, including Waits and Validations, Custom Properties as well as for Inputs and Outputs. Click the Minimize/Maximize button to expand or collapse any section to use the available dashboard space.


67

General The General section display is context-sensitive and varies depending on the selected item. Error detail information for the given selection is displayed in the General, Error(s) section (where applicable).

In addition to posting error(s) where applicable, the General section displays the following summary information (for the given selection):

Composition, Band, Track, and Clip Level • Error(s) – At the composition level, error(s) are posted, if any exist.

• Status – At the composition level, status is displayed as: Playing, Completed, Stopped, Aborted, etc.

• Effective Duration – At the composition level, the effective duration of a test is posted.

• Avg. Response Time – At the composition level, the average response time of messages and/or actions is posted.

• Total Message Bytes – At the composition level, the total message bytes is posted. If the composition was run in Preview mode, this is zero for both.

• Effective Message Throughput – The throughput as a measure of message or action bandwifth is posted.

Target Level • Error(s) – At the target level, error(s) are posted, if any exist.

• Name – The name of the selected target.

• Avg. Response Time – At the target level, the average response time of messages and/or actions is posted.

• Total Message Bytes – At the targetn level, the total message bytes is posted.


68

Message / Action Level • Error(s) – At the message/action level, error(s) are posted, if any exist. • Operation – Where applicable, the type of operation is shown. • Name - The message/action name

Waits and Validations Both built-in and custom waits and validations are posted in the new Waits and Validations section.

When you click on a wait or validation, the appropriate wait or validation will be highlighted in the Input or Output box below.

Custom Properties Custom Properties for the item selected in either the Navigation Tree or Cover Flow are displayed in this section, which appears in context.


69

Input The Inputs of the given selection are displayed in this section. For example, for an HTTP message, the form will always have two fields – URL Values and Query String.

For a browser action or app action (as shown below), the Input form fields will vary.


70

Output The Output of the selected item(s) is displayed in this section. If custom outputs are defined for that action, they are displayed. A custom output specifies what is it to be shown in the Result Details widget for a given message or action. Typical outputs include captureScreenshot, outputElementText, and outputInnerHTML. A single message or action can have an unlimited number of outputs.


71

Target Demographics The Target Demographics section appears when a target item is selected in Result Details. This section provides target detail info that varies by target type. For HTTP(S) targets, the URL is posted:

For Browser Action targets, the Conductor, Browser, URL, Wait Timeout, Wait Interval, Mouse Speed, Allow Native Xpath setting, and the Fire Blur on Type setting are all posted:

For App Action targets, the iOS Device Agent’s name, the Mobile App name, as well as the terminate behaviors shown below are posted.


72

Waterfall Chart This release introduces a new system Waterfall Dashboard (shown below) that appears during dashboard selection.

The Waterfall Chart works in conjunction with the Clip Element Hierarchy chart and both must be present if they are to be used in a custom dashboard. The Select an item in the Clip Element Hierarchy list to display the load times for various paged clip elements on the right.


73

in the Widget Type > Basic section that shows incremental information for pages within a composition. This chart will display the load time based on the selected clip element those pages for the given columns: Domain, URL, Timeline, Sent, and Received.

Creating a Custom Waterfall Dashboard 1. To create a custom Waterfall Dashboard, open the Widget Selection

Panel, Widget Type, Basic section. 2. Select the Clip Element Hierarchy widget and drag it into the dashboard

workspace. 3. Select the Waterfall Chart and drag it into the dashboard workspace. This

chart is the final entry listed in this section.


74

Miscellaneous

Automatic Update Improvements for EC2 Additional Automatic Update improvements in this release will make the upgrade process smarter for Amazon EC2-launched instances. Instead of just modifying the current environment, updates will examine the EC2 environment's volume, and then update any other environments using that volume as well.

Jenkins Plug-in Error Message Improvements Jenkins and sCommand now include improved error messages than those presented in Gran Cabrio. Probably some algorithm like "find the first item that had an error, chronologically, and display all of the errors (text and details) for that item as multiline text

Support Browsers As of this release, Mozilla Firefox 3.6 is no longer supported. The updated browser list includes the following browser version:

• Internet Explorer 8 and 9

• Google Chrome 4+

• Firefox 4+

• Safari 4+


75

Bugs Fixed 28468: Unable to rename result Renaming a result by renaming it from the Results list in Central and using the method provided didn’t work as expected. 28319: Average Response Time is zero for clip level and higher The Average Response Time and other statistics weren’t being posted in browser recording compositions as expected. 26098: Invalid argument.; JS line 2370 An error occurred in Internet Explorer 8 due to an “invalid width.” 25944: Error Analysis Widget not showing all data The Error Analysis widget didn’t show the complete tree view of the errors while a test ran, and also didn’t permit tree navigation as expected. 25893: A Soap fault has occurred. faultString: For input string: "undefined" An error occurred invoking the Session Template Wizard after selecting a string to replace. 25715: Attempting to access Target cookies for a Clip without Targets that support cookies yields null pointer exception A Script attempting to access cookies via a WebUI target using a script, such as: $context.currentClip.targets[0].cookies = null; var cookies = $context.currentClip.targets[0].cookies; would be aborted with a "Null pointer" Java exception. 25685: Changing a Clip to Timed caused a new, empty Clip to be displayed Attempting to Change Clip Timing after adding an HTTP message to an unsaved clip incorrectly allowed timing to be changed; however, a new Clip Editor tab was unexepectedly opened as a result. 25681: No more waits can be added if last wait is removed If a wait in a noOP action was removed by the user, there would be no subsequent way to add any more waits via the user interface. 25679: Can't get target-level header definitions or property analytics to work


76

A problem with targets whose underlying XML had incorrect bindings would prevent property analytics and other features from working as expected. 25677: Clip Editor keeps closing my Clip with "deleted" message while debugging Opening a clip in a new tab from within a composition, and then adding the open clip to a new draft composition, would result in this error closing the Clip Editor tab. 25562: IP address didn’t show up in the IP address column The HTTP(S) recording wizard included an extraneous IP Address column that in some cases wasn’t populated at all. This column has been removed as of this release. 25258: Very difficult to view results when stopped in the debugger After stopping the Debugger, attempting to view the result in Result Details for the given item was impossible as the Result Details widget was constantly refreshing and closing up every expanded detail. 25199: WebUI Targets Didn’t Handle Preview Mode Properly When used with a WebUI target, Preview Mode didn’t work as expected. The WebUI test attempted to find a Conductor and to cause that Conductor to perform actions not in line with what is expected in a preview.


77

soasta cloudtest release notes build...

Documents