automation for developers

Automation For Developers

● We are super developers and we don't make mistakes.

● I never do changes to my code.

● I am the life time owner of my code.

● I am the only person who use each component I build.

● We have telepathic.

● My code runs on each platform

○ Its platform vendor’s responsibility to make

platforms unique.

What do you think ??????

Who write the code ??

Who works with your code??

●

●

●

●

●

“Quality is free, but only to thosewho are willing to pay heavily for it.”

- Lister, Demarco: “Peopleware”

●

●

●

●

●

●

Testing Good Bad And Ugly

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

Understanding - Cost of test Wrong way

Understanding - Cost of test Right way

Basic Understanding - Unit Test

● Focused on testing behavior of a

particular class, module or method.

● Smallest part of the application.

● Fast (less than 0.1 second per test)

● No external implementation

dependencies

(filesystem, database, web services, etc.). All

dependencies are faked for the test context.

● Can easily be parallelized, since each test is atomic

Basic Understanding - Integration Test

● Focused on verifying the integration

of one or more components together.

● May have external dependencies.

● A test that takes longer than a unit

Test should

Integration Tests - with in WSO2 ● Executes on product pack created by build.

● Not depending on other product environments.

● Independent.

● Lightweight.

● Covers only scenarios related to features on individual product.

●●

○○○○○○○

●M = E − N + 2P,

where

E = the number of edges of the graph.

N = the number of nodes of the graph.

P = the number of Connected Components

●fi

●

●

●●●●

Jacoco /EclEmma coverage report

Emma coverage report

● Platform scenarios.

● Integration scenarios.

● UI scenarios.

● Real world use cases.

● Performance and Security scenarios.

● Patches provided.

How tests really looks like

Platform - Platform Tests

Cross Platform Tests

Product - Integration Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Product - Integration Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Product - Integration Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Product - Integration Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

Component - Unit Tests

What got better and bigger

● Lightweight test execution engine.

● More focus on platform wide scenarios.

● More user friendly configuration management.

● Ability to extend TestNG based listener implementation

○ Ability to plugin custom servers like Tomcat, CXF ,ActiveMQ,

Apache FTP and more.

● Ability to come up with your automation story.

Organization Test Automation Framework modules

Engine

Extensions Admin Service API

Core - Utilities

Automation Test

Core Servers External Tools

Extensibility Modules

Configuration

Overall Architecture

Configuration

Support Modules Test Automation Framework

Test

Build platform -Maven

Test Trigger- Surefire

Testing Framework - TestNg

Test Suite

platform

Automation.xml

Coverage

Instrumentation.txt

Pluggable Extensions

TestNg Listener Extension Modules

Test Tooling Extensions

Server Management

Tenant Management

Engine

TestNg Listener Extensions

Configuration Management

Context Management

Extensible module handling

Admin Service API

Reusable core utils

filter.txt

Test Configurations

TestNg.xmlTestNg.xmlTestNg.xml

Flexibilities Provided

● Manage several products inside single test.

● Manage users in several product domains.

● Deployment of artifacts for different products under

different users.

● Configured admin service clients as test oriented API.

● Retrieving test environment configurations easily.

● Running same test in both local and Stratos environments.

● Reporting TestNg and surefire reports for all your tests.

● Tooling support of Selenium and Jmeter.

Technologies for Automation

Test Framework Build Systems

Code Coverage

Extensibility Support

UI Automation

● Like JUnit but not JUnit

● Annotation based Architecture

● Test Grouping

● Extensibility features

○ Listeners

○ Bean Shell

○ Annotation Transformers

● Easiness of managing Test cases.

● Failed test execution (Straight and easy)

●

●

●

○ Trigger out TestNg.

○ Control TestNG flow (Through Surefire TestNg Utils)

○ Generate reports. (HTML, XML and emailable report)

Execution Started@BeforeSuite

@BeforeTest

@AfterTest

@AfterSuite

Execution Closed

@BeforeClass

@BeforeGroups

@BeforeMethod

@AfterMethod

@AfterGroups

@AfterClass

● Listeners are interfaces allows to modify TestNG's

behavior.

● Listeners are binned to a testNg execution.

● Listeners implement the interface org.testng.ITestListener○ IExecutionListener

■ Triggers at start and end of any execution.■ ISuiteListener.

○ ISuiteListener■ Triggers at suite start and end.ITestListener■ Triggers at Test start, Finish, Failure, Skip and partial failure.

● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run

●

○ Offline instrumentation

○ On fly instrumentation

●

●

○ LOC coverage

○ Class coverage

○ cyclomatic complexity

●

●

Common Extensions

Automation.xml

Test Suite

Test Suite Extensions

Test Suite Utilities

Test classes

Automation Framework Engine

●

●

●

●

● Includes all utility classes that can play supportive role inside a test.

○ Ex :-

■ Axis2 client.

■ Wire message monitor.

■ Custom server startup scenarios (Axis2, Tomcat, ActiveMQ).

■ Concurrency test scenarios.

■ Rest Clients.

● Provides common methodology for all development

teams and Automation team to maintain a set

of supportive classes without changing the Core.

Questions?

Happy Automation !!

Contact us !