joomla! testing - j!dd germany 2016

Joomla! TestingYves Hoppe

@yveshoppe

@compojoom

Why?

Deliver better and reliable application

Validate there is no regression

Make sure your application runs as expected

Trust your own application

Makes debugging much easier

TestsIn Joomla!

Unit testsIntegration testsSystem testsJavaScript testsCodeStyle (phpcs)

Joomla! Automated Testing Team

Unit testing

In Unit testing ()

we test a small and specific amount of code

Sample method

public function isTheAnswer($number){

if ($number === 42) {

return true; } return false;}

Sample test

class TestCase extends \PHPUnit\Framework\TestCase{

public function testIsTheAnswer(){

$this->assertTrue(isTheAnswer(42));$this->assertFalse(isTheAnswer(1));

// … Best practice: One assertion per test}}

The issue / reality

public function getTheAnswer($number){ if (is_array($number)) throw new Exception('Arrays not accepted');

if (!is_numeric($number)) // Check

// Decimal, Binary, Hex - @todo add lower / upper case check if ($number === 42 || $number === '00101010' || $number == '2A' || $number == '0x2A') return '<span class="answer-right">The number is the answer to

everything</span>'; elseif ($number == 13) return '<div class="answer-partial">What do you get when you multiple 6 x

9=</div>'; elseif ($number == 666) return 'WTF?!';

return false;}

Benefits of Unit testing

Issues can be discovered early

Acts as documentation

Forces you to write simpler and more readable code

Verifies the design

Fastest and no GUI necessary

Drawbacks

Does not show the absence of errors

Hard to set up useful and realistic tests

Writing unit tests after the development can be tough

Does only cover code

Unit tests in Joomla!

https://github.com/joomla/joomla-cms/tree/staging/tests/unit

PHPUnit 4.8.27

Within today 6119 different tests with 10526 assertions

We need you

Running the Joomla! unit testsClone the current staging repo from GitHub

git clone --depth 1 https://github.com/joomla/joomla-cms

Navigate with the command line to the cloned joomla instancecd joomla-cms

Install dependencies with composer (get it before)composer install

Run the tests with:‘libraries/vendor/bin/phpunit’

Value for Joomla! Extension Developers

Downsides

Complicated to set up with Joomla!

You have to mock and bootstrap everything

Lot of work

System Tests

In System Testing (Browser testing)

we test against the installed and running application

Benefits

Tests the whole application

Real world testsTest more code

with less lines

System tests vs Unit tests

Unit tests are mainly used to test framework classes

System tests make sure the application works correctly from a user point

System tests does not require you to understand the code (or even the programming language)

System tests test the whole system not only a small part

They also cover JavaScript, CSS and markup changes

It’s not System or Unit testing - they complement each other!

System tests in Joomla!

https://www.youtube.com/watch?v=ZHo0TuA5Qjs

Google Summer of Code 2016

Prital Patel

● Setup of browser automated tests for Joomla!

● Powered by Gherkin● For com_users● And com_content

Software used

https://github.com/joomla-projects/gsoc16_browser-automated-tests

Sample Codeception testpublic function administratorCreateWeblink(\Step\Acceptance\weblink $I) { $I->wantToTest('Weblink creation in /administrator/'); $I->doAdministratorLogin();

$I->amOnPage('administrator/index.php?option=com_weblinks'); $I->waitForText('Web Links', '30', ['css' => 'h1']); $I->checkForPhpNoticesOrWarnings();

$I->clickToolbarButton('New'); $I->waitForText('Web Link: New', '30', ['css' => 'h1']); $I->fillField(['id' => 'jform_title'], $this->title); $I->fillField(['id' => 'jform_url'], $this->url); $I->clickToolbarButton('Save & Close');

$I->waitForText('Web Links', '30', ['css' => 'h1']); $I->see('Web link successfully saved', ['id' => 'system-message-container']);

$I->see($this->title, ['id' => 'weblinkList']);}

Running the Joomla! Browser testsClone the current staging repo from GitHub

git clone --depth 1 https://github.com/joomla-projects/gsoc16_browser-automated-tests

Navigate with the command line to the cloned instance codeception foldercd gsoc16_browser-automated tests/tests/codeception

Install dependencies with composer (get it before)composer install

Configure ‘tests/codeception/acceptance.suite.yml’ and run the tests with:‘tests/codeception/vendor/bin/robo run:tests’

Value for Joomla! extension developers

BenefitsFast to setup and you don’t need to mock Joomla!

JoomlaBrowser has common Steps for Joomla, like:

Installation, uninstallation of extensions

Navigation, Checking for PHP notices

Creating menu items

Fast to write and easy to learn

JavaScript Tests

In JavaScript Testing (Browser based)

We cover the JavaScript files of our application.

JavaScript tests in Joomla!

Google Summer of Code 2016

Ruchiranga Wickramasinghe

● Setup of an automated test system

● Coverage reporting● For /media/system/js● Tests are merged in Joomla!

Core now

Software used

Jasmine 2.4.1, Jasmine-jQuery 2.1.1, Karma 0.13.22 and Firefox 48

Within today we have 196 different tests

Running the JavaScript! testsClone the current staging repo from GitHub

git clone --depth 1 https://github.com/joomla/joomla-cms

Navigate with the command line to your joomla instance javascript tests foldercd joomla-cms/tests/javascript

Install dependencies with npm (get it before)npm install

Run the tests with:‘tests/javascript/node_modules/karma/bin/karma start karma.conf.js’

Value for Joomla! Extension Developers

Benefits

No need for mocking Joomla!

No running Joomla! Installation needed

Easy to write and setup

CodeStyle tests

Code Styleand Metrics tests(Static Analysis)

check .

Why Code Style matters

Communication is the Key

Code looks consistent

Clear rules for developers

Make Errors obvious

Code Style in Joomla!

http://joomla.github.io/coding-standards/

PHP_CodeSniffer (phpcs)

and PHPQA Tools

PHP_CodeSniffer 1.5.6 (working on upgrade)

Running the PHP_CodeSniffer checksClone the current staging repo from GitHub

git clone --depth 1 https://github.com/joomla/joomla-cms

Navigate with the command line to your joomla instance cd joomla-cms

Install dependencies with composer (get it before)composer install

Run the tests with:‘libraries/vendor/bin/phpcs --extensions=php --standard=build/phpcs/Joomla .’

Automated testing

Putting the teststogether to a bigautomatic safety net!

Why?

Repeatable

Detect issues for every commit / pull request

Saves you time and money

Automated tests in Joomla!

Software used

Directly see issues at your Pull Request

Runs for every change and pull request against coreTravis:

Multiple PHP Versions:

- PHP 5.3, 5.4, 5.5, 5.6, 7.0- 7.1, HHVM (allowed to fail)

JavaScript tests (Node.js)

PHP_CodeSniffer (PHP 5.6)

Conclusion

Call for volunteers!

The Joomla! Automated Testing Working Group is looking for new members, which want to make

Joomla! better

https://volunteers.joomla.org/teams/automated-tests-working-group

Questions?Thank you for listening!

@yveshoppe