Download - Introduction to clarity
Introduction To Clarity
What is Clarity?
Clarity is a Test Automation Framework to
make automation
● Eazy
● Organized
● Relevant
● Optimized
Why Clarity?
● To automate platform-wide scenarios.
● To execute tests against stratos and private clouds.
● To adopt tooling support to automation
○ Selenium
○ SoapUI
○ Jmeter
○ Ravana
● Reporting and keeping historical records.
● Coverage.
WSO2 Platform Scenario
More On Clarity?Automate scenarios with multiple products with multiple
users/ tenants.
● Execute tests using different DBs, OS, browsers and
app servers.
● Support for webcontext root, different port changes,
host name bindings.
● Testing Clustered environments.
● Run tests against up and running server.
Unit Vs Integration TestsUnit Tests:
● Focused on defining (not testing) the behavior of a particular class, and that class only.
● Fast (less than 0.1 second per test)● No external implementation dependencies (filesystem, database, web
services, etc.). All dependencies are faked for the test context.● Can easily be parallelized, since each test is atomic
Integration Tests:● Focused on verifying the integration of one or more components
together.● May have external dependencies. (in fact, it is likely testing the
integration with this dependency)● A test that takes longer than a unit Test should (longer than 0.1
seconds per test)
Integration Tests Vs Platform TestsIntegration Tests - with in wso2 product
● Executes on product pack created by build.● Not depending on other product environments.● Independent.● Lightweight.● Covers only scenarios related to features on individual product.
Platform Tests:● Can run on a configured ( With mounts , Clusters ...) product packs
and already up and running server. ● Can generate scenarios based on multiple product environments.● Can execute load and UI tests
Architecture Overview
Reference:- http://www.ibm.com/developerworks/java/library/j-testng/
● Follows TestNG Annotations bases
Execution Structure.
● Operations based on TestNG Listeners.
○ Server startup/Shutdown.
○ Emma instrumentation, Report Generation And Cleaning.
○ Artifact Deployment /Artifact cleaning.
○ Tennent Creation and Registering.
○ Report Generation.
● Test Optimized API.
● Utility Classes to Support test cases.
Execution Template for ClarityExecution Started
@BeforeSuite@BeforeTest
@AfterTest
@AfterSuite
Execution Closed
@BeforeClass
@BeforeGroups
@BeforeMethod
@AfterMethod
@AfterGroups
@AfterClass
TestNg Listeners● Listeners are interfaces allows to modify TestNG's
behavior.
● Listeners are binned to a testNg execution.
● Listeners implement the interface org.testng.ITestListener○ IExecutionListener
■ Triggers at start and end of any execution.■ ISuiteListener.
○■ Triggers at suite start and end.ITestListener■ Triggers at Test start, Finish, Failure, Skip and partial failure.
● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run
Usage of Listeners in Clarity● PlatformExecutionManager (IExecutionListener)
○ onExecutionStart()
■ Emma instrumentations
■ Server start
○ onExecutionFinish()
■ Server Sutdown
■ Emma report generation
● PlatformSuiteManager (ISuiteListener)
○ Set environment properties ex- Key Store Paths
○ Populate Users.
Usage of Listeners in Clarity cont.d
● PlatformTestManager (ITestListener)
○ On Start
■ Artifact Deployment
○ On Finish
■ Artifact Clean up
● PlatformReportManager (IReporter)
○ Generate TestNg Report
○ Generate sure-fire report
○ Export data for Dashboard
Clarity Architecture
Artifact Repository|-- AS| |-- aar| | |-- Axis2 Service.aar| |-- car| | `-- AxisCApp-1.0.0.car|-- BPS| `-- bpel| |-- CleanUpTest1.zip| |-- BRS| |-- aar| |-- rsl| `-- ruleservice| `-- GreetingService.aar|-- DSS| |-- dbs| | |-- csv|-- ESB| |-- car| | |-- DemoProxy-1.0.0.car| |-- endpointlookup.xml|-- GREG| |-- artistRegistry.gar|-- GS| `-- gadget| `-- HelloWorldGadget.xml|-- IS| |-- IIA001Policy.xml`-- MS `-- jszip `-- schemaTest1.zip
● Location for keeping all artifacts
needs for your Test
● Refers by automated Artifact
deployment process
● Manages my file structure
depending on product and artifact
type.
Clarity Properties
# Testing server details#---------------------------------------------------------------------------------------------------stratos.test=falseport.enable=truecarbon.web.context.enable=falseservice.deployment.delay=30000cluster.enable=falseproduct.list=GREGdeployment.framework.enable=falsedeployment.framework.home=/home/krishantha/automation/deployment-framework
coverage.enable=falsegreg.carbon.home=/home/dharshana/latestPack/gregPack2/wso2greg-4.0.0-SNAPSHOT
as.host.name=localhostas.http.port=9764as.https.port=9444#as.webContext.root
● File Location <FRAMEWORK_HOME>/core/org.wso2.automation.platform.core/src/main/resources/framework.properties
● Contains all information
about your environment.
● To configure WSO2 clarity
for your environment and
configure package details
Manage Tenants
"0","admin","admin123""1","[email protected]","admin123""2","[email protected]","admin123""3","[email protected]","admin123""4","[email protected]","admin123""5","[email protected]","admin123""6","[email protected]","admin123""7","[email protected]","admin123""8","[email protected]","admin123""9","[email protected]","admin123""10","[email protected]","admin123""11","[email protected]","admin123""12","[email protected]","admin123"
● File Location● <FRAMEWORK_HOME>/core/org.wso2.automation.platform.
core/src/main/resources/tenantList.csv● <FRAMEWORK_HOME>/core/org.wso2.automation.platform.
core/src/main/resources/usertList.csv
Automated Artifact Deployment
<?xml version="1.0"?><scenarios> <scenario testName="org.wso2.automation.common.test.as.service.AARServiceSpaceInFileNameTest"> <products> <product name="AS"> <artifacts userId="1"> <artifact name="Axis2 Service.aar" type="aar"/> </artifacts> </product> </products> </scenario>
Write First Clarity Test - Demo
Reporting● Includes TestNg reports ,Junit
Reports ( Sure Fire report), And
Emma Coverage report
● Directories containing separate
results for each suite ( Directory
name is as same as the suite name).
● Junit Reports ( Surefire reports).
● index.html (Dashboard for executed
test).
|-- reports | |-- BPSStructuredSuite | | |-- classes.html | | |-- FlowClient.html | | |-- FlowClient.properties | | |-- FlowClient.xml | | |-- ForEachClient.html | | |-- ForEachClient.properties | | |-- ForEachClient.xml | |-- index.html | |-- junitreports | | |-- TEST-org.wso2.automation.common.test.bps.mgts.BpelStructAct_FlowClient.xml | | |-- TEST-org.wso2.automation.common.test.bps.mgtst.BpelStructAct_forEachClient.xml
Reference
OT article
http://wso2.org/library/knowledge-base/2012/06/configure-execute-tests-clarity
TestNg documenation
http://testng.org/doc/documentation-main.html
SVN Location https://svn.wso2.org/repos/wso2/carbon/platform/trunk/platform-integration/system-test-framework
Questions?