test report andrei turcanu, 27. november 2015. 2 introduction 1
TRANSCRIPT
Test Report
Andrei Turcanu, 27. November 2015
2
IntroductionIntroduction
1
3
What is Test Report?
Test Report is a web-platform that
aggregates test results (as well as other
test-specific data) from Jenkins and
generates reports from the collected
data.
What is Test Report
TEST REPORT
4
Motivation
Visibility (Pfeil)
Making our testing efforts more visible
Quantifiability (ggf)
Deducing interesting/important numbers
Accessibility (ggf)
Have the reports always up to date & accessible for everyone
Jenkins Integration (Jenkins)
Integrate with our Build Management software
Documentation (ggf)
TODO
Motivation
TEST REPORT
5
Evaluation of Existing Solutions
Results
Four established web-based TM solutions
Klaros, Squash TM, Tarantula, TestLink
Only Klaros & TestLink with Jenkins-Integration
Generally old/outdated technology
TestLink: No out of the box support for Unit Tests
TestLink: Poor reporting & not easily extensible
Evaluation of Existing Solutions
TEST REPORT
6
Drupal
Jenkins provides XML-API
Established Drupal modules for all requirements
Feeds & Rules for data-fetching
Forena Reports for reporting
Configuring instead of developing
Easily extensible where necessary (Solr Search, …)
Flexibility (adjusting panels, themes, …)
Several years of experience with Drupal
Drupal
TEST REPORT
7
Test ReportTest Report
2
8
Prototype Phase
Data Model (Content Types)
Very fast, no coding effort
Data Import (Feeds, Rules)
Basic setup fast, one script necessary for triggering Feed Import
Data Processing (Charts/Charts & Graphs)
Most charting solutions don’t offer enough aggregation methods
Data Processing (Forena Reports & Custom Views)
Forena allows for complex SQL-Syntax (SQL effort)
Custom Visualizations (coding effort)
TEST REPORT
9
Problem 1: Rules/Feeds integration
No out-of-the-box integration of Rules with Feeds
Only one abandoned sandbox project
First solution via custom PHP-code
“Rules Feed Import”-module (hopefully)
10
Problem 2: Complex Queries for Reports
Most of Drupal‘s graphing modules don’t support “complex” queries
Drupal “Content Types” not suited for Forena Reports
Custom (SQL-)Views simplify queries
D3-integration on the horizon (sandbox & comments)
11
Test Report Install Profile
TODO
Requirements: Jenkins with xUnit-Plugin & accessible by Drupal
12
Usage Example/Demonstration
Configure Taxonomy
Create a project in Drupal
Create a build in Drupal
Configure CRON
Wait for CRON to fetch data
Analyze/Configure your reports
13
Planned features
Test Case Design feature + Manual Test Cases
D3-integration
Code Coverage (SonarQube)
Test Case import feature
