final data driven testing ebook
TRANSCRIPT
-
8/9/2019 Final Data Driven Testing eBook
1/18
-
8/9/2019 Final Data Driven Testing eBook
2/18
What is Data-Driven Testing?
Contents
Introduction
Creating a New Project
Recording an Automated Test
Modifying the Data-Driven Test
Changing out Operations Based on Data
Executing a Data-Driven Test and Checking Results
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 1
2
5
6
10
11
14
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
3/18
Introduction
Data-driven testing is one of the most powerful items in a
tester’s toolkit – and for good reason: it is a process which can
maximize time savings and increase test coverage. Simply
put, it is the process of designing a test, parameterizing it and
running it in an iterative manner with varying data. This allows
the tester to increase coverage by ensuring a wide range of
input variables pass. Using an automated testing tool, such as
TestComplete, allows a tester to supercharge this process by:
• Generating a large volume of randomized data
• Managing an automated test from design to
execution utilizing the generated data
• Report and analyze the results – all in a matter of
minutes!
A key aspect of data-driven testing is using data that isexternal to your functional tests, which is then loaded and
used to extend your automated test cases. You can take the
same test case and run it with as many different inputs as you
like, thus getting better coverage from a single test. True to the
name, it’s the data that drives the test, which is what makes
it so powerful. This data can be stored in an Excel document,
a MySQL database or another data source. Since automatingrepetitive tests means that you don’t need to test each piece
of data manually, data-driven testing can result in time and
financial savings for a development effort – time and
resources which can be redirected to other areas of need
within your testing team.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 2
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
4/18
To illustrate how impactful data-driven testing can be, let’s
apply the key concepts to an everyday, practical problem from
outside of the office. Imagine you are about to embark on
a long road trip, but your old car does not appear to be up
for the task – the engine is making noises and your mileagehasn’t been great. Before you hit the road, it might be best
to take your car in for a tune-up. After running a few manual
diagnostic tests and fixing your engine, your mechanic might
do a few other things to help your car perform better.
• Inflating the tires to the proper pressure will help you
maximize your miles per gallon and help you get more
out of every tank of gas. In a sense, data-driven testing
is the same way, helping you squeeze more coverage
into every test.
Replacing an old, dirty air filter will drastically improve
your horsepower. In the same sense, using a tool thatautomatically generates data for your tests will help
turbocharge your automated tests.
• Before leaving the shop, your car might get hooked up
to a diagnostics tool to test each of the systems within
your engine – just to make sure everything
is in working order. Having robust reportingcapabilities within a testing tool serves a similar
function, allowing you to see the results of your
automated tests, and identify areas where
performance is lacking.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 3
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
5/18
Data-driven testing can help you get the most out of your
automated testing, especially when coupled with a tool that
allows you to generate data and report on your results. In the
next section, we’ll demonstrate how this can all be done using
TestComplete.
Although most automated testing tools allow you to perform
data-driven testing, a feature that can really help take your
results to the next level is data generation. TestComplete’s
data generator allows you to quickly build Excel files or internal
tables full of data. It is able to create data in 19 predefined
types, including email addresses, zip code, phone numbers and
more. Additionally, there are customizable strings for users to
build data specific to their needs. Once this data is created, it
can be incorporated into your data-driven test immediately,
saving time associated with building data sets.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 4
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
6/18
Now that we have a basic understanding of what data-driven
testing is and why we use frameworks to build out data-driven
test cases, you can now learn how to build your automated
test within TestComplete. TestComplete allows us to quickly
and easily script out our test automation and grab the
appropriate code from any of our frameworks.
Creating a New Project
First, we will create a new TestComplete project. To do this:
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 5
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
7/18
Recording an Automated Test
In the following steps, we will record a keyword test. The
test will perform actions like opening the tested
application, opening the Order form, populating the form,
creating checkpoints, closing the form and closing the
application.
If TestComplete is not already running, launch it now.
Once it is running, select New | New Project from the
TestComplete File menu. This will call the Create New
Project wizard. In this wizard, specify the project’s name
and location. On the next pages of this wizard, you can
specify tested applications for your test, select the desired
scripting language and so on. However, we will close the
wizard now. We will specify tested applications later.
Click Finish to close the wizard. TestComplete will create
the project and display its contents in the Project Explorer
panel.
Right-click the TestedApps node in the Project
Explorer panel and choose Add | New Item from the
context menu. The Open File dialog will appear.
Select the Orders application executable in the
Add Tested Application dialog and click Open. You can
find the executable in the following folder:
\Open Applications\C#\bin\
Release\Orders.exe.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 6
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
8/18
To record the test:
1. Select Record Keyword Test from TestComplete’s Test
Engine toolbar, or choose Test | Record | Record Keyword
Test from TestComplete’s main menu, or select Record NewTest from the toolbar of the KeywordTest Editor.
2. Add the “Launching the Orders application” comment to a
test. To do this:
Expand the Recording toolbar by clicking the Show
Additional Buttons item and click Add Comment to
Test on the Recording toolbar. This will invoke the Add
Comment to Test dialog:
Enter “Launching the Orders application” and click
Add.
3. Click Run Tested Application on the Recording toolbar and
add the “Opening the Order form” comment to a test as
described above.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 7
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
9/18
4. Choose Orders | New order from the main menu of the
Orders application and add the “Populating the Order form”
comment to a test.
5. Enter the following values into the edit boxes of the Order
form:
6. Add the “Creating the checkpoint” comment to a test.
7. Perform the following actions to create a checkpoint for the
Total value:
Select Create Property Checkpoint from the
Recording toolbar. This calls the Create PropertyCheckpoint wizard.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 8
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
10/18
On the Select Object page of the wizard, specify the
Total edit box of the Order form. After specifying this
edit box, TestComplete displays its name in the Object
field of the page, then click Next.
On the Select Property page, choose the Text property
from the list of the object’s properties and click Next.
On the Comparison Settings page, make sure that
the Equal condition is selected, then click Finish.
8. Add the “Closing the Order form” comment to a test and
click OK on the Order window to close the Order form and
add new record to the Orders application.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 9
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
11/18
9. Add the “Closing the Orders application” comment to a test
and close the orders application and click No on the
Confirmation window to close the application without
changes, then click Stop on the Recording toolbar.
The recorded test looks like this:
Modifying the Data-Driven Test
Now we will modify the recorded data-driven test to make it
use data stored in an external Excel file. The test will iterate
through the rows of the specified file, each time accessing onerow of the DB Table variable and then using that stored data
to populate the Order form’s fields.
You can create the DB Table variable on the Variables edit
page of the KeywordTest editor manually or add this variable
to the test automatically while creating a data loop.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 10
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
12/18
We will create a data-driven loop including the operations
between the RunTestedApp operation and the operation that
closes the Orders application.
To do this:
Select the specified operations in
the Keyword Test editon and right-
click them.The editor’s context-
menu will appear.
Select Make Data Loop from the
ensuing context menu item. The
Data-Driven Loop wizard will ap-
pear.
On the first page of the wizard, select Create New
Variable, type STET in the edit box and click Next.
On the next page of the wizard select Excel
Worksheet from the ensuing table and click Next.
Now, specify the path to the TestBook.xlsx file that is
located in the\Data-Driven
Testing folder. You can enterthe file path and
name, or click the ellipsis button and select the file inthe subsequent Open File dialog. After you’ve specified
the file, click Next.
Select TestSheet from the ensuing table and click Next.
Specify the database records to process. We will need
to process all of the records, so, click Next.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 11
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
13/18
On the next page of the wizard, you can change the
parameters that are used by the selected operations.
In our example, we need to assign the values that are
stored in the Excel file to some of the selected
operations. To do this:Select ProductNames ClickItem | Param1
parameter. Click the corresponding Value cell
and choose DBTable1 -> Product from
the drop-down list.
Similarly, you can specify new values for other
parameters: Quantity, Date and so on.
Click Finish.
TestComplete then adds the Data-Driven Loop operation to
the keyword- driven test, and all the selected operations will be
moved as its child operations.
Also, the selected operations now have the new parameter
values that are specified through the DBTable1 variable. You
can see this in the picture:
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 12
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
14/18
Changing out Operations Based on Data
Note that in the test described above the Master Card option
button is always selected. However, you may want your test
to select the card type against the Credit Card column of theTestBook.xlsx file. In this case you will have to specify the
conditions for this manually. For example, you can do it using
the If … Then operation. To demonstrate this approach, we’ll
change the keyword-driven test in the following way:
delete the MasterCard.ClickButton operation and insert the
segment below:
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 13
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
15/18
Executing a Data-Driven Test and Checking Results
Now we will run the data-driven test. TestComplete opens the
tested application and adds new records until there is no more
data in the TestBook sheet. After the data-driven test is over,
you can find the test results in the Test Log.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 14
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
16/18
If the output value had not matched the appropriate verifying
value, “The property value does not equal the template value”
error messages would have been posted into the log. This
indicates that the application works incorrectly. In this case,
the values should be revised. You can include any informationyou need in error messages. For instance, you can specify the
values of all input parameters that were used when the
error occurred.
The ability to report, analyze and communicate the results of
a test is critical for any tester, particularly when using complex
data-driven tests. TestComplete makes this easy by detailing
every action performed in the test, including screenshots of
these actions, and a summary report displaying a chart of
the test’s success/failure ratio. Additionally, it’s easy to email
reports as an attachment to other members – right out of Test-
Complete. Teams that use issue tracking systems such as Bug-
zilla, Ontime, Jira, TFS and QAComplete can take advantage ofthis integration, allowing bugs to be logged quickly and easily.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 15
http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/
-
8/9/2019 Final Data Driven Testing eBook
17/18
When implemented properly, data-driven testing is an effec-
tive method of extending your automated test coverage while
saving time. The process of taking randomly generated data,
iterating an automated test using this data and reporting
on the results helps you get more mileage out of our tests.Data-driven testing is a method that can be applied using a
variety of automated testing tools (or even to manual testing).
However, in order to supercharge your Data-Driven Testing, be
sure to use a tool that offers both data-generation capabilities
and robust reporting. If you’re interested in learning more, or
want to try your hand at data-driven testing, download a free
trial of TestComplete today.
TestCompleteby SMART BEAR
What is Data-Driven Testing? | Page 16
http://www2.smartbear.com/TC-Free-Trials.htmlhttp://www2.smartbear.com/TC-Free-Trials.htmlhttp://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://smartbear.com/products/qa-tools/automated-testing-tools/http://www2.smartbear.com/TC-Free-Trials.htmlhttp://www2.smartbear.com/TC-Free-Trials.html
-
8/9/2019 Final Data Driven Testing eBook
18/18
In short, TestComplete is an easy-to-use and affordable
testing tool that will turn your redundant software tests
into fast, reliable automated tests without proprietaryscripting.
Start a Free 30-Day
TestComplete Trial
http://www2.smartbear.com/TC-Free-Trials.htmlhttp://www2.smartbear.com/TC-Free-Trials.htmlhttp://www2.smartbear.com/TC-Free-Trials.htmlhttp://www2.smartbear.com/TC-Free-Trials.htmlhttp://smartbear.com/products/qa-tools/automated-testing-tools/http://www2.smartbear.com/TC-Free-Trials.htmlhttp://www2.smartbear.com/TC-Free-Trials.html