automated testing in wordpress, really?!

Automated Testing in

WordPress,Really?!

Rate this talk: https://joind.in/10115

#dc4d - Automated Testing in WordPress with @ptahdunbar

https://joind.in/10115


Ptah (Pirate) Dunbar

● Started with WordPress and PHP in ‘05

● Contributing developer toWordPress, BuddyPress, bbPress

● Full stack Web Developer

● Architect at LiveNinja.com

● WPMIA co-organizer and SoFloPHP member

☠ Became Pirate Dunbar


● Started with WordPress and PHP in ‘05

● Contributing developer toWordPress, BuddyPress, bbPress

● Full stack Web Developer

● Architect at LiveNinja.com

● WPMIA co-organizer and SoFloPHP member

☠ Became Pirate Dunbar

Ptah (Pirate) Dunbar



● Understand automated testing concepts,ideas and best practices.

● Learn PHPUnit basics and the WordPress testsuite.

● Resources and homework

Agenda

In one hour


WordPress powers

1 in 5 websites

source: http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites

http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites


WordPress community

28,5102,177

source: http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites

http://w3techs.com/blog/entry/wordpress_powers_1_in_5_websites


“The result is that a lot of the plugins are written in poor code and turn out to be poorly compatible with other plugins”

— Yoast

http://yoast.com/plugin-future/




● WP_DEBUG

● var_dump();

● print_r();

● error_log();

● debug_backtrace();

Manual Testing

Pull out the tools


● WP_DEBUG

● var_dump();

● print_r();

● error_log();


Manual Testing

Pull out the tools

Ad-hoc & Temporary


● WP_DEBUG

● var_dump();

● print_r();

● error_log();


Manual Testing

SLOW & Error Prone

Pull out the tools


● WP_DEBUG

● var_dump();

● print_r();

● error_log();


Pull out the tools

Manual Testing

Doesn’t scale


A scripted process that invokes your app to test

features and compares the outcome with expected

results.

Automated Testing


Persistent var_dumps();

Automated Testing


Better than checking the logs

Automated Testing


The Bigger Picture

Automated Testing

Continuous Integration

Continuous Delivery

TDD

BDD

Agile

Scrum

Continuous InspectionReleasing early, releasing often

Phingvagrant


Getting started

Automate Testing


There are so many

Frameworks

CHOOSE YOUR FRAMEWORK


CHOOSE YOUR FRAMEWORK

PHPUnithttp://phpunit.de/manual/

Sebastian Bergmann

http://phpunit.de/manual/

http://phpunit.de/manual/


vim composer.json && composer update

{ "require-dev": {

"phpunit/phpunit": "3.7.*",

"phpunit/phpunit-selenium" : "*",

}

}

http://getcomposer.org




$>./vendor/bin/phpunit

PHPUnit



PHPUnit

Terminology


● Test CaseA set of conditions that you set up in order to assert expected outcome.

PHPUnit

Terminology


PHPUnit

Terminology


● Test ClassA collection of test cases, extends PHPUnit


PHPUnit

Terminology


● Test ClassA collection of test cases, extends PHPUnit

● Test SuiteA collection of test classes


<?php// test classclass CalTest extends PHPUnit_Framework_TestCase{

// test case public function testAddReturnsSumOfTwoPositiveIntegers() { // assert stuff. }}

PHPUnit

TEST CLASS


plugin/loader.phpincludes/

admin.phpapi.php…

phpunit.xmltests/

adminTest.phpApiTest.php…

…

PHPUnit


plugin/loader.phpincludes/

admin.phpfunctions.php…

phpunit.xmltests/

integration/…

acceptance/…

…

PHPUnit


<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="vendor/autoload.php"> <testsuites> <testsuite name="tests"> <directory suffix="Test.php">tests/</directory> </testsuite> </testsuites></phpunit>

PHPUnit

phpunit.xml - configuration file for PHPUnit



PHPUnit

phpunit.xml

Configure your test suite location


<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="vendor/autoload.php"> <testsuites> <testsuite name="integration"> <directory suffix="Test.php">tests/integration</directory> </testsuite> <testsuite name="acceptance"> <directory suffix="Test.php">tests/acceptance</directory> </testsuite> </testsuites></phpunit>

PHPUnit

phpunit.xml

Configure your test suite location



PHPUnit

phpunit.xml

Bootstrap file is included before any tests run


Assertions

PHPUnit

Explicitly check expected outcomeagaisnt actual outcome.


Assertions

PHPUnit

Explicitly check expected outcomeagaisnt actual outcome.

$this->assertTrue(condition);


Arrange, Act, Assert

PHPUnit


function testThatItsTestingTime(){

}

1. A

2. A

3. A

PHPUnit



}

1. A

2. A

3. Assert (check for the expected behavior)

PHPUnit



}

1. A

2. Act (call the method/trigger the action)


PHPUnit



}

1. Arrange (the context/dependencies)

2. Act (call the method/trigger the action)


PHPUnit


Example

PHPUnit


<?phpclass CalTest extends PHPUnit_Framework_TestCase{ public function testAddReturnsSumOfTwoPositiveIntegers() { // Arrange

// Act

// Assert

}}

PHPUnit

plugin/tests/unit/calTest.php



// Act

// Assert $this->assertEquals(3, $result);

}}

PHPUnit




// Act $result = $calculator->add(1,2);


}}

PHPUnit




$calculator = new Calculator();



}}

PHPUnit







}}

PHPUnit


./vendor/bin/phpunit






}}

PHPUnit


Time: 148ms, Memory: 2.75Mb

OK: (1 test, 1 assertions)







}}

PHPUnit







}}

PHPUnit



Failed asserting that 4 equals 3


<?phpclass UserServiceTest extends PHPUnit_Framework_TestCase{ public function testPersistWithValidUserdataReturnsUserObject() {

// Arrange

// Act

// Assert

}

//…}

PHPUnit

plugin/tests/unit/LiveNinja/User/ServiceTest.php



// Arrange

// Act

// Assert$this->assertTrue($user instanceof ‘\LiveNinja\User\Entity’);

}

//…}

PHPUnit




// Arrange

// Act$user = $service->persist($validUserdata);


}

//…}

PHPUnit




// Arrange$service = new \LiveNinja\User\Service;$validUserdata = [...];



}

//…}

PHPUnit







}

//…}

PHPUnit







// Assert$this->assertTrue($user instanceof ‘\LiveRacoons\User\Entity’);

}

//…}

PHPUnit







}

//…}

PHPUnit








}

//…}

PHPUnit


Failed asserting that false equals true



ASSERTIONSAppendix: http://phpunit.de/manual/3.7/en/appendixes.assertions.html

PHPUnit

Use the most specific assertion possible

● assertTrue();

● assertEquals();

● assertContains();

● assertGreaterThan();

● assertNotNull();

● assertFalse();

● assertNotEquals();

● assertContainsOnly();

● assertLessThan();

● assertSame();

http://phpunit.de/manual/3.7/en/appendixes.assertions.html


PHPUnit

There was 1 failure:

1) Tests_Basic::test_readmereadme.html's version needs to be updated to 3.9.Failed asserting that '3.8' matches expected '3.9'.

/private/tmp/wordpress-tests/tests/phpunit/tests/basic.php:29

FAIL


PHPUnit

There was 1 failure:

1) Tests_User_Author::test_get_the_authorFailed asserting that two objects are equal.--- Expected+++ Actual@@ @@ WP_User Object ( 'data' => stdClass Object (- 'ID' => '3'- 'user_login' => 'User 1'- 'user_pass' => '$P$BpIqOzMNRGZNy9qxKL/3cCDCMe85o2.'- 'user_nicename' => 'user-1'- 'user_email' => '[email protected]'+ 'ID' => '2'+ 'user_login' => 'test_author'+ 'user_pass' => '$P$BUdMebxEjJ23.6LbH9ujvVUFBsUuZv/'+ 'user_nicename' => 'test_author'+ 'user_email' => '[email protected]' 'user_url' => '' 'user_registered' => '2013-12-20 15:31:01' 'user_activation_key' => '' 'user_status' => '0'- 'display_name' => 'User 1'+ 'display_name' => 'test_author' )- 'ID' => 3+ 'ID' => 2 'caps' => Array (- 'subscriber' => true+ 'author' => true ) 'cap_key' => 'wptests_capabilities' 'roles' => Array (- 0 => 'subscriber'+ 0 => 'author' ) 'allcaps' => Array ( 'read' => true 'level_0' => true- 'subscriber' => true+ 'upload_files' => true+ 'edit_posts' => true+ 'edit_published_posts' => true+ 'publish_posts' => true+ 'level_2' => true+ 'level_1' => true+ 'delete_posts' => true+ 'delete_published_posts' => true+ 'author' => true ) 'filter' => null )

/private/tmp/wordpress-tests/tests/phpunit/tests/user/author.php:50

FAIL





// Assert$this->assertTrue($user instanceof ‘\LiveRacoon\User\Entity’);

}

//…}

PHPUnit






// Assert$this->assertInstanceOf(‘\LiveNinja\User\Entity’, $user);

}

//…}

PHPUnit







}

//…}

PHPUnit




<?phpclass UserServiceTest extends PHPUnit_Framework_TestCase{ public function testPersistWithInvalidUserdataReturnsWPError() {

// Arrange$service = new \LiveNinja\User\Service;$invalidUserdata = [];

// Act$user = $service->persist($invalidUserdata);

// Assert$this->assertInstanceOf(‘WP_Error’, $user);

}

//…}

PHPUnit


WordPress Testsuite


WordPress with Testshttp://develop.svn.wordpress.org/trunk/

1858 Tests, 8611 Assertions, 2.59 minutes

WordPress Testsuite

http://develop.svn.wordpress.org/trunk/

http://develop.svn.wordpress.org/trunk/


WordPress Testsuite


Getting started

WordPress Testsuite


WordPress Testsuite

<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="tests/bootstrap-wp.php"> <testsuites> <testsuite name="tests"> <directory suffix="Test.php">tests/</directory> </testsuite> </testsuites></phpunit>


WordPress Testsuite

<?xml version="1.0" encoding="UTF-8"?><phpunit bootstrap="tests/bootstrap-wp.php"> <testsuites> <testsuite name="tests"> <directory suffix="Test.php">tests/</directory> </testsuite>

<testsuite name="integration"> <directory suffix="Test.php">integration/</directory> </testsuite> </testsuites></phpunit>


plugin/tests/integration/PluginTest.php

<?phpclass PluginTest extends PHPUnit_Framework_TestCase{ // test cases...}

WordPress Testsuite


plugin/tests/integration/PluginTest.php

<?phpclass PluginTest extends WP_UnitTestCase{ // test cases...}

WordPress Testsuite


$>./vendor/bin/phpunit

WordPress Testsuite



Run Testsinside of an isolated

WordPress Environment

WordPress Testsuite


ConfigureWordPressOptions

$GLOBALS['wp_tests_options'] = ['active_plugins' => [

'hello.php',...

],'current_theme' => 'kubrick',...

];

bootstrap.php

WordPress Testsuite


ConfigureWordPress

Includesfunction __muplugins_loaded(){

// code and stuff.require_once 'env-debug.php';

}tests_add_filter('muplugins_loaded', '__muplugins_loaded');

bootstrap.php

WordPress Testsuite


● Navigate to site URL (Updates globals)$this->get_url($url);

● Test WP_Query for Conditionals (is_page, is_single, is_404)$this->assertQueryTrue($arg1, $arg2, ...);

● Test for Errors$this->assertWPError($thing);

● Genereate WordPress data fixtures$this->factory->post->create_and_get();

$this->factory->comment->create_post_comments($pid, 100);

$this->factory->user->create_many(5);

$this->factory->blog->create();

and more…

WordPress Testsuite


plugin/tests/integration/WPCustomizationTest.php

WordPress Testsuite

<?phpclass WPCustomizationTest extends WP_UnitTestCase{

// test casesfunction testRedirectForDateBasedPermalinks(){

// Arrange

// Act

// Assert

}}



WordPress Testsuite



// Arrange

// Act

// Assert$this->assertQueryTrue( 'is_404' );

}}



WordPress Testsuite



// Arrange$customWP = new WPCustomization;$this->factory->post->create(['post_date' => '2007-09-04 00:00:00']);

// Act$customWP->deprecate_unused_pages();$this->go_to('/2007/');


}}



WordPress Testsuite






}}




WordPress Testsuite


/** * @dataProvider getRequiredPlugins */function testAllRequiredPluginsAreActive($plugin){

// Assert$this->assertTrue( is_plugin_active($plugin),

sprintf('%s is not activated.', $plugin) );}

function getRequiredPlugins(){

return [[‘hello.php’],

];}

}



WordPress Testsuite







];}

}




WordPress Testsuite


/** * @dataProvider getWPOptions */function testWPOptionSettingsAreConfigured($option_name, $option_value){

// Assert$this->assertSame($option_value, get_option($option_name));

}

function getWPOptions(){

return [[‘home’, ‘http://example.org/wp/’],[‘siteurl’, ‘http://example.org/’],

];}

}



WordPress Testsuite




}



];}

}

http://example.org



WordPress Testsuite




}



];}

}


http://example.org



WordPress Testsuite




}



];}

}




http://example.org

http://www.seleniumhq.org/



plugin/tests/acceptance/ConnectTest.php

Acceptance Testing

<?phpclass ConnectTest extends PHPUnit_Framework_TestCase{ protected function setUp() { }

public function testUserCanLogInViaTwitter() { }}



Acceptance Testing

<?phpclass ConnectTest extends PHPUnit_Extensions_Selenium2TestCase{ protected function setUp() { }




Acceptance Testing

<?phpclass ConnectTest extends PHPUnit_Extensions_Selenium2TestCase{ protected function setUp() { $this->setBrowser("*chrome"); $this->setBrowserUrl("https://wpss.dev/"); }




Acceptance Testing


public function testUserCanLogInViaTwitter() { $this->open("/");

}}



Acceptance Testing


public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000");

}}



Acceptance Testing


public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000"); $this->click("css=img[alt=\"Twitter\"]"); $this->waitForPageToLoad("30000");

}}



Acceptance Testing


public function testUserCanLogInViaTwitter() { $this->open("/"); $this->click("link=Log in"); $this->waitForPageToLoad("30000"); $this->click("css=img[alt=\"Twitter\"]"); $this->waitForPageToLoad("30000"); $this->assertContains( ‘dashboard’, $this->title() ); }}


Acceptance

Selenium IDE Plugin

● Visually navigate throughout your site and generate a PHPUnittest case.

● Download Extension○ http://www.seleniumhq.

org/projects/ide/

● Download PHPUnit Formatter○ https://addons.mozilla.org/en-

US/firefox/addon/selenium-ide-php-formatters/

http://www.seleniumhq.org/projects/ide/



https://addons.mozilla.org/en-US/firefox/addon/selenium-ide-php-formatters/





How can we be confident that our tests

cover everything?


Testing boundaries


● (User) Acceptance Testing

○ Verify that all features are done done.

○ Black-box testing, no knowledge of internals.

Testing boundaries





● Integration Testing

○ Test WordPress settings/configuration;

○ Compatibility between plugins and themes.

Testing boundaries






○ Test WordPress settings/configuration,

○ Compatibility between plugins and themes

● Unit Testing

○ Test class methods and functions in isolation, zero dependencies

○ Does one “behavoir”

Testing boundaries



Verify that all features are done done,

black-box testing, no knowledge of

internals


Test WordPress settings/configuration,

compatibility between plugins and

themes

● Unit Testing

Test class methods and functions in

isolation, zero dependencies,

does one “behavoir”.

Testing boundaries

AcceptanceTesting

Integration Testing

Unit Testing



Verify that all features are done done,

black-box testing, no knowledge of

internals


Test WordPress settings/configuration,

compatibility between plugins and

themes

● Unit Testing

Test class methods and functions in

isolation, zero dependencies,

does one “behavoir”.

Testing boundaries

Integration Testing

Unit Testing

AcceptanceTesting

#ATWP // Automated Testing in WordPress // @ptahdunbar

What to tests?

● Test plugin works in various WordPress setups

○ Does it work under multisite?

○ What about a custom content directory?

● Test all code paths in functions and methods

● Test compatiblity between most popular plugins

● Test that default pages exists


What to tests?

● Test for theme support

● Test that post formats contain property elements

● Test any required assets that need to be loaded in

templates

● Test for required elements on a page

● Verify search results template displays search term

● Verify SEO meta tags

1. WordPress APIs


What to not tests?


What to not tests?

1. WordPress APIs

2. PHP language features


What to not tests?

1. WordPress APIs

2. PHP language features

3. Third party vendor code

Getting into the groove




● Build out templates

● Build out templates○ Create HTML/CSS





● Build out templates○ Create HTML/CSS○ Identify dynamic elements and their data

structure



● Build out templates○ Create HTML/CSS○ Identify dynamic elements and their data

structure○ Label them and fill them with dummy data



○ Verbally state your trying to do




○ Verbally explain what the code does




○ Verbally explain what the code does

○ Do this alone or with a fellow dev :)

What’s Next?


● Download WP Skeleton Family

○ https://github.com/ptahdunbar/wp-skeleton-site

○ https://github.com/ptahdunbar/wp-skeleton-plugin

○ https://github.com/ptahdunbar/wp-skeleton-theme

Get started

“A Walking Skeleton is a tiny implementation of the thinnest possible slice of real functionality that we can automatically

build, deploy and test end-to-end.”

https://github.com/ptahdunbar/wp-skeleton-site

https://github.com/ptahdunbar/wp-skeleton-site

https://github.com/ptahdunbar/wp-skeleton-plugin

https://github.com/ptahdunbar/wp-skeleton-plugin

https://github.com/ptahdunbar/wp-skeleton-theme

https://github.com/ptahdunbar/wp-skeleton-theme


● Art of Unit Testing (.NET)

○ https://leanpub.com/u/royosherove

○ Udemy Five day course

● #GOOS Book (Java)

● XUnit Test Patterns (Java)

● Grumpy Books (PHP)

○ https://leanpub.com/u/chartjes

● Misko Hevery

Resources

https://leanpub.com/u/royosherove

https://leanpub.com/u/royosherove

https://www.udemy.com/tdd-in-net-with-roy-osherove/

https://www.udemy.com/tdd-in-net-with-roy-osherove/

https://leanpub.com/u/chartjes

https://leanpub.com/u/chartjes


● Learn moar PHPUnit features

○ data providers,

○ mocks and stubs

○ wordpress testsuite

● Goal: Write at least 100 assertions!

Homework!

TODO


Automated Testing


Automated Testingincreases your productivity


Automated Testingfacilitates more shipping


Automated Testingscales with you


Automated Testingis your professional duty

as a developer


Thank youAutomated Testing in WordPress

Pirate Dunbar@[email protected]

Rate this talk:https://joind.in/10115



automated testing in wordpress, really?!

Technology