agile bodensee - testautomation & continuous delivery workshop

AGILE BODENSEE 2014 TEST AUTOMATION & CONTINUOUS INTEGRATION WORKSHOP

Konstanz 01.10.2014

1

2

WHO AM I? Gridfusion Software Solutions Contact: Michael Palotas Gerbiweg 2 8853 Lachen SWITZERLAND Tel.: +41 79 6690708 Email: [email protected]

Head of Productivity & Test Engineering, eBay

Founder / Principal Consultant Gridfusion Software Solutions

SETTING THE STAGE

Tell me about yourself J

What are your expectations for today?

3

POSSIBLE AGENDA TOPICS

Introduction

CI – what is it, why do we use it, what are we trying to achieve

Automation / Code Quality

Tools

Unit tests

Cobertura

Sonar

E2E Tests

Selenium

Cucumber

Amazon cloud

Pipeline

Integrating mobile

Vagrant - Infrastructure as code

Management / organizational aspect

4

Your setup

How do you build software?

5

WHAT IS SO SPECIAL ABOUT AGILE?

6

WHY CI/CD?

7

WHAT IS IMPORTANT IN AGILE?

8

Agile

=?

Release working software anytime

9

Traditional waterfall model / tools do not support “build and deploy anytime”

10

WHAT IS CI / CD?

CI and CD

=

Automated Build?

Automated Tests?

Automated Quality?

Automated Deployment?

Automated Feedback?

11

WHY CI / CD

Deliver value to the business more frequently

Better Quality

Early Bugs

Bug Prevention instead of late detection

Fast & frequent feedback

12

WHY CI / CD

Automated frequent builds

Automated frequent tests

Automated frequent code quality metrics

(Hopefully) Fewer bugs

Early feedback

Fast feedback

13

WITHOUT CI

Slow / long release cycles

Late testing

Waterfall (WaterScrum)

Bugs

Slow feedback

Complex integration

14

CORE PRINCIPLES

Every build could be a release

Everything should be automated

Stable and trustworthy automated tests

Build pipelines

15

RELEASING IN THE OLD WORLD

16

Coding

Deploy to

QA

QA

Deploy to Production

Production Smoke Tests

Bug Bashes

CI / CD - CORE WORKFLOW

17

Compile

Unit Test

Deploy to QA

Acceptance tests



Code Quality

THE MAIN TASKS

Automated build

Automated code quality

Automated testing

Automated deployment

18

19

Wakaleo.com

CAN YOU MEASURE AUTOMATED CODE QUALITY? DOES THAT MAKE SENSE?

20

AUTOMATED CODE QUALITY?

Sonar gives you information on:

-  Lines of code

-  % of comments

-  Duplications

-  Complexity

-  Rules compliance

-  Unit test coverage

-  Unit test success rate

-  Unit test duration

-  Hotspots

21

AUTOMATE EVERYTHING?

22

WHAT SHOULD YOU AUTOMATE?

23

WHAT ARE BARRIERS TO CI / CD?

24

WHAT IS CONTIUOUS INTEGRATION?

25

Continuous integration (CI) is the practice, in software engineering, of merging all developer working copies with a shared mainline several times a day. It was first named and proposed as part of extreme programming (XP). Its main aim is to prevent integration problems, referred to as "integration hell" in early descriptions of XP. CI can be seen as an intensification of practices of periodic integration advocated by earlier published methods of incremental and iterative software development, such as the Booch method. CI isn't universally accepted as an improvement over frequent integration, so it is important to distinguish between the two as there is disagreement about the virtues of each.

WHAT IS CONTINUOUS DELIVERY?

26

Continuous Delivery (CD) is a design practice used in software development to automate and improve the process of software delivery. Techniques such as automated testing, continuous integration and continuous deployment allow software to be developed to a high standard and easily packaged and deployed to test environments, resulting in the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead. The technique was one of the assumptions of extreme programming but at an enterprise level has developed into a discipline of its own, with job descriptions for roles such as "buildmaster" calling for CD skills as mandatory.

THE MANAGEMENT / ORGANIZATIONAL ASPECT

What are the changes for developers and testers?

What needs to be changed in the organization to enable them to implement CI / CD?

What role has management in creating a devops culture?

27

OUR TOOLS

Version Control System GIT Build Tool MAVEN Unit Test Framework JUNIT / TESTNG End To End Test Framework SELENIUM Build Server / Deployment JENKINS

28

Branching & Merging

Small and Fast

Distributed

Data Assurance

Staging Area

Free and Open Source

VERSION CONTROL: GIT

29

http://git-scm.com/about/

GIT: BRANCHING & MERGING

30

Git-scm.com

GIT: SMALL & FAST

31

Git-scm.com

GIT: THE REST

Distributed

Data Assurance

Staging Area

Free & Open Source

32

GIT

Distributed / local

Download: http://git-scm.com/

Initialize directory: git init

Status: git status

Add files and directories to git: git add file1 dir2

Commit: git commit –am “commit message”

33

SHARE YOUR CODE - GITHUB

Create repository on Github: https://github.com

Create remote: git remote add origin https://…

Push code to Github: git push origin master

Tag your code: git tag –a v0.1 –m “initial version”

Push tag to Github: git push origin v0.1

34

GITHUB

35

CONNECT GIT AND GITHUB

36

Silverpeas.org

MAVEN

37

Apache Maven is a software project management and comprehension tool. Based on the concept of a project object model (POM), Maven can manage a project's build, reporting and documentation from a central piece of information.

http://maven.apache.org/guides/getting-started/maven-in-five-minutes.html

POM.XML

The pom.xml file is the core of a project's configuration in Maven. It is a single

configuration file that contains the majority of information required to build a project in just

the way you want.

38

POM.XML

39

MAVEN TARGETS

validate: validate the project is correct and all necessary information is available

compile: compile the source code of the project

test: test the compiled source code using a suitable unit testing framework. These tests should not require the code be packaged or deployed

package: take the compiled code and package it in its distributable format, such as a JAR.

integration-test: process and deploy the package if necessary into an environment where integration tests can be run

verify: run any checks to verify the package is valid and meets quality criteria

install: install the package into the local repository, for use as a dependency in other projects locally

deploy: done in an integration or release environment, copies the final package to the remote repository for sharing with other developers and projects.

clean: cleans up artifacts created by prior builds

40

EXAMPLES

mvn clean

mvn compile

mvn test

41

OUR APPLICATION TODAY

42

APPLICATION STRUCTURE

It is a super complex application …

2 text fields

1 submit button

1 very complex calculation

43

OUR ENTRY PAGE

44

RESULT PAGE

45

CALCULATOR CLASS

46

CALCULATOR TESTS

47

RUN UNIT TESTS

1.  Run in Eclipse

2.  Run via maven

48

COBERTURA

mvn clean cobertura:cobertura –Dgroup=unit

49

TMFSydney2014/target/site/cobertura/index.html

SONAR

50

http://localhost:9000/dashboard/index/net.gridfusion.dobcalc:tmf2

END TO END TESTS (E2E)

Selenium

Selenium Grid

51

TEST AUTOMATION

Unit Tests

E2E Tests

Manual Tests

Integration Tests

WHAT IS SELENIUM?

Selenium automates browsers

that’s it

…

E2E / UAT AUTOMATION WITH SELENIUM

54

CLIENT

SERVER JSON Wire Protocol

BROWSER

SELENIUM 2 / WEBDRIVER

JSON WIRE PROTOCOL

Client

Java

C#

Ruby

Python

Server

i.e. Selendroid, iOS-Driver

Server

Server

CLIENT

Is seen as „Selenium“ by the users

Generates HTTP requests which are received by the server

Is called by the test framework or the CI server

Supported languages: Java, C#, Python, Ruby, Perl, PHP,

JS

SERVER

Receives HTTP requests

Start and teardown of browser

Translates requests into browser specific commands

Communicates back to the client

SELENIUM GRID

Test 1 Test 2 Test …

Test 4500

Execution Time

Test 3

Parallel Execution

Test Test Test

Execution Time

Test

Test Test Test Test

Test Test Test Test

Par

alle

l Exe

cutio

n

Par

alle

l Exe

cutio

n

SCALING – SELENIUM GRID

CI

DEV

….

SELENIUM GRID HUB

IOS ANDROID

LINUX

WINDOWS

OSX

TEST INFRASTRUCTURE

AUT

DB

API

Browsers Mobiles

CLIENT

A SIMPLE SELENIUM TEST

61

A SIMPLE SELENIUM TEST

62

SELENIUM – MORE ADVANCED TESTS

63

DATA PROVIDER

64

URL FACTORY

65

CONTINUOUS INTEGRATION - JENKINS

Download at http://jenkins-ci.org/

66

RECAP

WHAT DO WE EXPECT

THE CI SYSTEM TO DO?

67

WHAT JENKINS DOES

Jenkins checks out the workspace from Github

Builds and runs tests locally according to POM

Runs maven targets according to POM description

69

A SIMPLE BUILD JOB

70

A SIMPLE BUILD JOB

1.  (Push to Github repository)

2.  Let Jenkins pick up the change

3.  Perform “mvn test –Dgroup=unit”

71

LET’S CREATE A BUILD / DEPLOYMENT PIPELINE

72

THE MASTER JOB

73

Unit Test

Deploy to QA

Acceptance tests



Code Coverage

UNIT TESTS

74

UNIT TESTS

75

UNIT TESTS

76

UNIT TESTS - RESULT

77

DEPLOY TO QA

78

DEPLOY TO QA

79

DEPLOY TO QA - RESULT

80

E2E TESTS

81

E2E TESTS

82

E2E TESTS - RESULT

83

E2E TESTS - RESULT

84

DEPLOY TO PRODUCTION

85

DEPLOY TO PRODUCTION

86

PRODUCTION SMOKE TESTS

87

LET’S GO MOBILE

88

INFRASTRUCTURE AS CODE

Consistent Infrastructure

Efficient Change Management

Simple to rebuild

89

TOOLS

90

VAGRANT EXAMPLE

91

BOOTSTRAP.SH

92

THANK YOU!

94