writing an extensible web testing framework ready for the cloud slide share

48
Writing an Extensible Web Testing Framework ready for the Cloud Presented by Mike Ensor May 22, 2013 Presented at GR8Conf in Copenhagen Denmark

Upload: mike-ensor

Post on 06-May-2015

1.577 views

Category:

Technology


0 download

DESCRIPTION

Presentation given at GR8Conf in Copenhagen, Denmark on May 22nd, 2013

TRANSCRIPT

Page 1: Writing an extensible web testing framework ready for the cloud   slide share

Writing an Extensible Web Testing

Framework ready for the Cloud Presented by Mike Ensor May 22, 2013 – Presented at GR8Conf in Copenhagen Denmark

Page 2: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Introductions

2

• What is this talk about?

• Curious about automated browser testing

• Testing a Reference application and/or SaaS

• Expanding Geb’s inheritance model

• Who is Mike!?

• Developing software for 16+ years

• Started working with Groovy and Grails in 2007

• Attended GR8Conf last year

• Acquity Group

• Blending strategy, creative and technology

• Currently a principal architect; Intershop and Demandware platform

• Introduced Geb and Spock to Demandware practice

Page 3: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Useful Links

3

Demandware testing framework

• http://mikeensor.bitbucket.org/demandware-testing-framework

Spock feature override extension

• https://github.com/mike-ensor/spock-feature-override-extension

Presentation

• http://www.slideshare.net/MikeEnsor/

Twitter @mikeensor

• http://www.twitter.com/mikeensor

Blog

• http://www.ensor.cc

Page 4: Writing an extensible web testing framework ready for the cloud   slide share

Web Testing

Page 5: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Why Test?

5

Raise Confidence

Safety Net

Prove Functionality

Ensure Quality

Lower Costs

Page 6: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Cost Savings

6

$0

$1,000

$2,000

$3,000

$4,000

$5,000

$6,000

Developer Unit Tests IntegrationTests

Environment

Cost per Defect

Avg Cost

Google Study presented at XP Days London 2009 – Mark Streibeck, Google Inc

Page 7: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

SaaS Access

Acceptance

Functional

Integration

Unit tests

Levels of Testing

7

Com

ple

xity Q

ua

ntity

Page 8: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

History of Web Testing

Manual Click Testing

•Time consuming

•Error prone

Screen Scrape

•Custom scripts

•Maintenance nightmare

HTTPUnit

•Does not work with JavaScript

•Tests HTML structure only

Canoo WebTest

•Developing in XML*

•Duplicate code

Selenium RC

•Cut-Copy-Paste scripts

•Duplication of code

•Mouse driven

WebDriver

•Very low level framework

•Easy to create brittle tests

Geb

•Abstraction on top of WebDriver

•Designed for code reuse

8

Page 9: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

What is Geb?

9

• Groovy based abstraction of Selenium Web Driver with

jQuery-like syntax for content selection

• Designed for robust yet simplistic syntax

• Simplified DSL describing Pages and Modules

• Words like

• url

• content

• base

• at

• Created with DRY principal in mind

• Use of highly configurable reusable modules

• Abstraction of WebDriver’s Page Model

Page 10: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Geb Core

10

Page Objects

Modules Geb

Page 11: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Page Objects

11

• What is a “Page”?

• Object representing a generic type of

webpage on a site

• Domain Specific Language

• url - Appended to configurable ‘baseUrl’

• Assists in navigating to page

• at

• closure used to determine if browser

is ‘at’ the page

• content

• Variables representing page content

with CSS selectors

class HomePage extends Page {

static url = "default/Home-Show"

static at = {

title.trim() == "Welcome"

}

static content = {

pageTitle {

$("header h1.title")

}

}

}

...

<header>

<h1 class=“title”>Main Page Title</h1>

</header> ...

Page 12: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Building multiple Pages

12

class HomePage extends Page {

static url = "default/Home-Show"

static at = {

title.trim() == "Welcome"

}

static content = {

companyTitle {

$("header h1.title")

}

footerCopywrite {

$("footer span.copy")

}

}

}

class ProductPage extends Page {

static url = "default/Product-Show"

static at = {

title.trim() == "Products"

}

static content = {

companyTitle {

$("header h1.title")

}

footerCopywrite {

$("footer span.copy")

}

productTitle {

$("#product h2.title")

}

}

}

Page 13: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Code Duplication

13

Page 14: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Dangers of Code Duplication

14

• Small change in underlying structure = many changes

• Long sections of nearly identical code

• Conceal purpose of code

• File size

• Code Maintenance

Page 15: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Modules

15

• What is a ‘Module’?

• Re-usable encapsulation

consisting of content references

• Domain Specific Language

• base

• Optional content closure acting as a

base to start content selection

• content

• Variables representing page content

with CSS selectors, optionally

originating from base closure

class HeaderModule extends Module {

static base = { $("header") }

static content = {

companyTitle {

$("h1.title")

}

login { $("a.login") }

}

}

...

<header>

<h1 class=“title”>Page Title</h1> <a href=“/login”>Login</a>

</header> ...

Page 16: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Geb: Pages + Modules

16

class HomePage extends Page {

static url = "default/Home-Show"

static at = { title.trim() == "Welcome" }

static content = {

footer { module FooterModule }

header { module HeaderModule }

}

}

class ProductPage extends Page {

static url = "default/Product-Show"

static at = { title.trim() == "Products" }

static content = {

productTitle { $("#product h2.title") }

footer { module FooterModule }

header { module HeaderModule }

}

}

class HeaderModule extends Module {

static base = { $("header") }

static content = {

companyTitle { $("h1.title") }

login { $("a.login") }

}

}

Page 17: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Now what?

17

Not Logged In Logged In

Page 18: Writing an extensible web testing framework ready for the cloud   slide share

Extending Geb Objects

Page 19: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Scenario

Your Mission:

You have been asked to create a special Product page for when a customer has logged in

The page looks and functions exactly like the Product page, but has an extra header

19

Page 20: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Extending Page Objects

20

• Extend our page object

• Re-define url, and at

• Inherits content

• Overwrite/add content

elements

class PersonalProductPage extends ProductPage { static url = "default/MyProduct-Show" static at = { title.trim() == "Personalized" } static content = { // header inherited from HomePage // footer inherited from HomePage productTitle { $("section#personal h2.title") } } }

Page 21: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

• Extend module object

• Re-define base

• Inherits content

• Overwrite/add content

elements

Extending Modules Objects

21

class AuthHeaderModule extends HeaderModule {

static base = { $("header section#personal") }

static content = {

// companyTitle inherited from HeaderModule

// login inherited from HeaderModule

logout { $("a.logout") }

}

}

Page 22: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Drawbacks & Best Practices

• Watch out for

• Overriding content segments happen at runtime

• No name spacing

• Possible to unintentionally overwrite content

• Possible solutions

• Document well (use JavaDoc style comments)

• API Groovy Doc

• Write a test to explicitly prove the inheritance

• Tests should fail if the inheritance fails

• Subscribe to [email protected]

22

Page 23: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

With great power

comes great responsibility -- Uncle Ben

23

Page 24: Writing an extensible web testing framework ready for the cloud   slide share

Using Geb with Spock

Page 25: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Spock

• Specification based testing framework designed to describe

the intention of the test in business friendly terms

• Provides keywords for structure

• when

• Action

• then

• Expectation

• Geb provides GebReportingSpec

• Can be extended to change functionality

• Provides extension model

• Control specification lifecycle

25

Page 26: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Creating a Specification

• Extend GebReportingSpec

• Describe a feature

• aka - “Test method”

• Browse to

• Verify at

• Verify content

• Defined in ProductPage

• Verify module content

• Defined in HeaderModule

26

class ProductPageSpec extends GebReportingSpec {

def "user can view the product page"() {

given:

to ProductPage

expect:

at ProductPage

and:

productTitle.text() == "My Cool Product"

and:

header.companyTitle.text() == "Company X"

}

}

Page 27: Writing an extensible web testing framework ready for the cloud   slide share

Extensible Web Testing

Page 28: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Scenario

Your Mission:

You work on a SaaS platform where each implementation is very similar in features and structure.

Design a testing framework where you can leverage common functionality inherent in all

implementations of the SaaS platform.

28

Reference Application Implementation

• Size & Color Filters

• Breadcrumbs

• Category Navigation

• Mini-Cart

• Wish List

• 3-in-row Product Grid

• Size & Color Filters

• Breadcrumbs

• Category Navigation

• Mini-Cart

• Wish List

• 4-in-row Product Grid

Page 29: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Common Functionality: Reference Application

29

Category Filter

Color Refinement

Size Refinement

Mini-Cart

Wish List

Breadcrumbs

Pagination

Price Refinement

Page 30: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Common Functionality: Implementation Application

30

Category Filter

Price

Refinement

Color

Refinement

Mini-Cart

Wish List

Breadcrumbs

Pagination

Page 31: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Inheriting Test Functionality

• Add dependency on

Reference App’s Test JAR

• Extend target specification

• Inherits all features from parent specification

• Foresee problems?

• Overwrite inherited features?

• Use derived Page objects?

• Investigate these one-by-one

31

class HomePgSpec extends GebReportingSpec {

def "user can view the home page"() {

// trimmed for clarity

given:

to HomePage

}

def "user can login"() {

// omitted for simplicity

}

}

class ClientPageSpec extends HomePgSpec {

// Inherit "user can view the home page“ & "user can login"

def "user can open wishlist"() {

// trimmed for clarity

given:

to ClientHomePage

}

}

Page 32: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Inheriting Test Functionality: First Run

32

class HomePgSpec extends GebReportingSpec {

def "user can view the home page"() {

given:

to HomePage

when:

header.login.click()

then:

at MyAccountPage

}

def "user can login"() { /*obmitted for clarity*/ }

}

class ClientPageSpec extends HomePgSpec {

def "user can open wishlist"() {

given:

to ClientHomePage

}

}

• Run ClientPageSpec

• Spock determines list of features

• “user can view the home page”

• “user can login”

• “user can open wishlist”

• Run “user can view the home page”

• Navigate “to” HomePage ????

Page 33: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Plan: overwrite inherited features

• Look for a feature in the parent

class with the same name

• Replace the inherited feature

with the provided feature

• Utilize Spock Extension features

to modify test execution

33

class ClientPageSpec extends HomePgSpec {

def "user can login"() {

given:

to ClientHomePage

when:

header.login.click()

then:

at ClientMyAccountPage

}

def "user can open wishlist"() {

// omitted for clarity

}

}

Page 34: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Spock Extensions

Add runtime functionality

Implements Visitor Pattern • Allow for modifications to the

Specification feature sequence

Ignore inherited features if implemented

in Child

Triggered by Annotation

• @OverrideFeatures

34

Page 35: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Overwrite Inherited Features

• Look for a feature in the Parent

class with the same name

• Create a marker @Annotation

• Tell Spock to use this feature, not

the Parent by using a Spock

Extension

• spock-feature-override-extension

• Open Sourced on github

35

@OverrideFeatures

class ClientPageSpec extends HomePgSpec {

def "user can login"() {

given:

to ClientHomePage

when:

header.login.click()

then:

at ClientMyAccountPage

}

def "user can open wishlist"() {

// omitted for clarity

}

}

Page 36: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Now what?

• Able to override inherited features

• Override every feature?

• Why inherit?

36

class HomePage extends Page {

static url = "default/Home-Show"

static at = {

title.trim() == "Welcome"

}

static content = {

header { module HeaderModule }

footer { module FooterModule }

}

}

class ClientHomePage extends HomePage {

static url = "default/Home-Show"

static at = {

title.trim() == "Company Home"

}

static content = {

nav { module NavigationModule }

}

}

• What do we want?

• Dynamically swap sub-classed Page

objects

Page 37: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Dynamically Swap Page Objects

• Page objects are managed by Browsers in Geb

• Extend existing Browser

• Create a “Browser” that knows about Page object relationships

37

geb.Browser

---------------------------------------------

Browser()

Page at(Class<? extends Page>)

Page to(Class<? extends Page>)

PageSelectingBrowser extends Browser

----------------------------------------------------------

PageSelectingBrowser(Map pageMap)

Page at(Class<? extends Page>)

Page to(Class<? extends Page>)

Page createPage(Class<? extends Page>)

Page 38: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

PageSelectingBrowser

38

class PageSelectingBrowser extends Browser {

private final Map<Class<? extends Page>, Class<? extends Page>> overridePages

/** Other methods omitted **/

@Override

Page createPage(Class<? extends Page> possibleParent) {

Class<? extends Page> type = possibleParent

if (overridePages.containsKey(possibleParent)) {

type = overridePages.get(possibleParent)

if (!possibleParent.isAssignableFrom(type)) {

throw new IllegalArgumentException("${type} not a ${possibleParent}")

}

}

type

}

}

Page 39: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Using PageSelectingBrowser

39

• Browsers are created/used in GebSpec or GebReportingSpec

• Extend GebReportingSpec

• Override createBrowser() and return new PageSelectingBrowser

• Create a default empty Page object Map

• Each specification can overwrite Page object Map

class OverridableGebReportingSpec extends GebReportingSpec {

@Override

Browser createBrowser() {

new PageSelectingBrowser(createConf(), getChildPageMap())

}

Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {

[:]

}

}

Page 40: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

OverridableGebReportingSpec

40

class HomePageSpecification extends OverridableGebReportingSpec {

def "user can view the home page"() {

given:

to HomePage

expect:

at HomePage

}

}

class ClientHomePageSpecification extends HomePageSpecification {

// inherit all features from HomePageSpecification

@Override

Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {

[(HomePage.class): ClientHomePage.class]

}

}

Dynamically replace

ClientHomePage for HomePage

Page 41: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Final Example

41

class HomePageSpec extends OverridableGebReportingSpec {

def "user can view the home page"() { // omitted for clarity }

def "user can login"() { // omitted for clarity }

}

@OverrideFeatures

class ClientHomePageSpecification extends HomePageSpecification {

def "user can login"() {

when:

to ClientHomePage

header.login.click()

then:

at ClientMyAccountPage

}

def "user can open wishlist"() { /* omitted for clarity */ }

@Override

Map<Class<? extends Page>, Class<? extends Page>> getChildPageMap() {

[(HomePage.class): ClientHomePage.class]

}

}

Inherit Features

Override Feature by Name

Create new features

Dynamically replace Page objects

Page 42: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved. 42

Page 43: Writing an extensible web testing framework ready for the cloud   slide share

Testing in the Cloud

Page 44: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Testing on Continuous Integration

44

Page 45: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Running without a Browser?

45

• What is RemoteWebDriver?

• Remotely sends driver commands to Selenium Grid

• Allows for same code to run on multiple browsers

Page 46: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Sauce Labs

46

• SauceLabs – In-cloud based Remote WebDriver

• Removes infrastructure requirements

• Records video of remote run

• Can operate behind firewalls via SSH tunnel

Page 47: Writing an extensible web testing framework ready for the cloud   slide share

©2013 Acquity Group, LLC. All rights reserved.

Parallelization

47

• Initial approach

• Modify GebSpec to create multiple browsers

• Cons

• Spock Extensions are not thread safe

• Browsers run at different speeds

• Remote WebDriver runners difficult to determine completion

• Reports not easy to collate

• Solution*

• Poor man’s Parallel

• Kick off multiple builds on CI with different configuration

• Possible parallelization with Gradle

• Fork per JVM

Page 48: Writing an extensible web testing framework ready for the cloud   slide share

Questions?