AUTOMATED USER ACCEPTANCE TESTING OF WEB APPLICATIONS WITH SELENIUM

Presented by Onur Yaman 2010719258 Bogazici University – Software Engineering

Overview • General Definitions • Problem Statement • Solutions Proposal • Why Selenium? • What is Selenium? • Demo • What’s next? • References

2

General Definitions

• User Acceptance Testing

•  Final testing based on

•  Specifications of the end-users or customer

•  Use by end-users or customer over some limited period of time

•  Completed system vs. requirements

•  Verification testing

•  Validation testing

•  Audit testing

•  Performed by users of the system

1/3

3

General Definitions

•  Test Automation

•  Use of software to control the execution of tests

•  Comparison of actual outcomes to predicted outcomes

•  Setting up

•  Test preconditions

•  Test control and reporting functions

•  Automating a manual process already in place that uses a

formalized testing process

2/3

4

General Definitions

• Web Application

•  Application that is accessed over a network

•  Client-server architecture

•  Ability to update and maintain without distributing and installing on

potentially thousands of clients

•  Cross-platform compatibility

3/3

5

Problem Statement

• User acceptance testing of web applications is

•  Costly

•  Browser (client) compatibility

•  Web applications tend to get updated frequently

•  Many possible inputs

•  Slow

•  Tied to browser performance

•  Clients and servers communicate over network

6

Solution •  Automating user acceptance tests

•  Usage of proprietary software

•  Many tools exist

•  FitNesse

•  HtmlUnit

•  HttpUnit

•  Watir, Vapir, Watij, WatiN, Win32-Watir, Watir for Rails

•  Celerity

•  iMacros

•  Canoo WebTest

•  Selenium

7

Why Selenium? •  Handles JavaScript well

•  Other tools are not reliable in handling in-page JavaScript

•  Provides maintainable tests •  Other tools tend to break after small page changes

•  Works well with TDD

•  Easy-to-write scripts

•  Open source •  Possibility of extending

•  Supports many browsers

•  Supports many programming languages

8

What is Selenium? •  Automated acceptance testing tool •  Developed by ThoughtWorks, now open source •  Portable

•  Deploys on Windows, Linux and MacOSX •  Supports various programming languages including C#, Java, Python,

PHP, Perl, Ruby, Groovy •  Supports most modern web browsers including Firefox, Safari, Google

Chrome, Opera, Internet Explorer •  Supports popular smart phones like Android and iPhone

•  Tool suite •  Selenium 1 (a.k.a. Selenium RC or Remote Control) •  Selenium 2 (a.k.a. Selenium WebDriver) •  Selenium IDE •  Selenium Grid

1/10

9

What is Selenium? •  Selenium 1 (a.k.a. Selenium RC or Remote Control)

•  Was the main Selenium project

•  Still actively supported

•  Supports several languages •  Java, JavaScript, Ruby, PHP, Python, Perl, C#

•  Supports almost every browser out there

•  Composed of components •  Selenium Server

•  Manages browsers, runs Selenese commands, acts as HTTP proxy between the browser and the AUT

•  Client libraries •  Provide interface between any programming language and the Selenium Server

2/10

10

What is Selenium?

• Selenium 1 (a.k.a. Selenium RC or Remote Control)

•  Architecture

3/10

11

What is Selenium? •  Example for Selenium 1 (a.k.a. Selenium RC or Remote Control)

4/10

1- Run the Selenium Server

2- Run the language specific tests using the corresponding client API

12

What is Selenium?

• Selenium 2 (a.k.a. Selenium WebDriver)

•  Provides a simple, concise programming interface

•  Better supports dynamic web pages (AJAX)

•  Supplies a well-designed object-oriented API

•  Makes direct calls to the browser using browser’s native support for

automation (browser dependent)

•  Possibility of not using Selenium-Server

5/10

13

What is Selenium? • Example for Selenium 2 (a.k.a. Selenium WebDriver)

6/10

14

What is Selenium? •  Selenium IDE

•  Tool to develop Selenium test cases

•  Uses Selenese (Selenium script language)

•  Firefox plug-in

•  Allows saving macros (like Microsoft Office macros)

•  Allows exporting saved test cases to any desired language that uses

the Client API

•  Allows grouping test cases as test suites

•  Allows running previously saved test cases/suites

•  Provides results of tests

7/10

15

What is Selenium?

• Example for Selenium IDE

8/10

16

What is Selenium? •  Selenium Grid

•  Builds on the traditional Selenium setup •  More than one machine can be used

•  Selenium RC and the browser are not tied to a specific application (they can be shared by multiple applications and projects)

•  Possibility of a distributed grid system of Selenium RCs •  Shared across builds, applications, projects, organizations

•  Scalable •  Selenium Hub

•  Allocates a Selenium RC to a specific test (transparently)

•  Limits the number of concurrent test runs on each RC

•  Shields the tests from the actual grid infrastructure

9/10

17

What is Selenium?

• Selenium Grid Setup

10/10

18

Demo

•  Live demo

•  Bingo! – Java Web Application

•  Very simple test cases

•  Selenium WebDriver is used

•  Implementation is made in Python 2.7

19

What’s next? •  Writing good tests

•  Keep tests self-contained

•  Write own extensions if needed

•  Write tests before development (TDD)

•  Refactor tests (if possible)

•  Selenium and FitNesse Integration •  Everyone (including the customers) see, track and run tests

•  Continuous Integration •  Running selenium tests is possible (in-memory X window system)

•  Test coverage analysis

20

References •  Selenium Documentation (Release 1.0) / May 10, 2012

<URL: http://seleniumhq.org/docs/book/Selenium_Documentation.pdf>

•  Selenium Grid: How It Works / 2008 <URL: http://selenium-grid.seleniumhq.org/how_it_works.html>

•  Holmes, A.; Kellogg: Automating functional tests using Selenium. M.Agile Conference, 2006.

•  Bruns, A.; Kornstadt, A.; Wichmann, D.: Web Application Tests with Selenium. Software, IEEEVolume: 26 , Issue: 5.

•  Razak, R.A.; Fahrurazi, F.R.: Agile testing with Selenium. Software Engineering (MySEC), 2011 5th Malaysian Conference in Digital

Object Identifier: 10.1109/MySEC.2011.6140672.

21