РЕТРОСПЕКТИВА ТЕСТОВЫХ ИСТРУМЕНТОВ

ДЛЯ JAVASCRIPT СЕРГЕЙ СЕЛЕЦКИЙ

@SSELETSKYY

ПРОВЕРКА МИКРОФОНА

О ЧЕМ МЫ ПОГОВОРИМ• КАКИЕ ИНСТРУМЕНТЫ БЫЛИ ЗА ПОСЛЕДНИЕ 12 ЛЕТ

• СРАВНЕНИЕ СТИЛЕЙ ОПИСАНИЯ ТЕСТОВ

• ВЫБОР МЕЖДУ ПРОСТОЙ БИБЛИОТЕКОЙ И МУЛЬТИВАРКОЙ или КРАТКИЕ ОБЗОР

• КАКИЕ ТЕНДЕНЦИИ СВЕТЛОГО БУДУЩЕГО? (вопросы к аудитории)

А КАКОГО ЭТО НУЖНО?

”

“

Я ВИТРАТИВ ЦІЛИЙ ТИЖДЕНЬ ЩОБ НАПИСАТИ ЦЕЙ ТЕСТ ДЛЯ НАШОГО ПРОДУКТУ

А Я НАПИСАВ ТЕСТ ЩОБ ПЕРЕВІРИТИ ТЕСТ ДМИТРА

ТВІЙ ТЕСТ БУВ МАЙЖЕ ДОСКОНАЛИМ

ОТОЖ ПРОДОВЖУЙ ГАРНО ПРАЦЮВАТИ, ДМИТРИКУ

ИНСТРУМЕНТЫ

• 2004 - ?

• 2004 - SELENIUM -> WEB DRIVER

• 2004 - SELENIUM -> WEB DRIVER • 2007 - JSSpec (BDD framework)

• 2004 - SELENIUM -> WEB DRIVER • 2007 - JSSpec • 2008 - Screw.Unit

• 2004 - SELENIUM -> WEB DRIVER • 2007 - JSSpec • 2008 - Screw.Unit, QUnit

• 2004 - SELENIUM -> WEB DRIVER • 2007 - JSSpec • 2008 - Screw.Unit, QUnit,

FireUnit, Blue Ridge

• 2004 - SELENIUM -> WEB DRIVER • 2007 - JSSpec • 2008 - Screw.Unit, QUnit,

FireUnit, Blue Ridge, Sugar Test, jShoulda

• 2004 - SELENIUM -> WEB DRIVER • 2007 - JSSpec • 2008 - Screw.Unit, QUnit,

FireUnit, Blue Ridge, Sugar Test, jShoulda JSpec, Jasmine

• 2009 - TestSwarm, Inspec, YUI Test

• 2009 - TestSwarm, Inspec, YUI Test • 2010 - Sinon.js, JSLint, JSLitmus, JSHint

• 2009 - TestSwarm, Inspec, YUI Test • 2010 - Sinon.js, JSLint, JSLitmus, JSHint • 2011 - Mocha.js, Karma, Buster

• 2009 - TestSwarm, Inspec, YUI Test • 2010 - Sinon.js, JSLint, JSLitmus, JSHint • 2011 - Mocha.js, Karma, Buster • 2012 - Intern, Nightwatch.js

• 2009 - TestSwarm, Inspec, YUI Test • 2010 - Sinon.js, JSLint, JSLitmus, JSHint • 2011 - Mocha.js, Karma, Buster • 2012 - Intern, Nightwatch.js • 2013 - Protractor, ReactTestUtils

• 2009 - TestSwarm, Inspec, YUI Test • 2010 - Sinon.js, JSLint, JSLitmus, JSHint • 2011 - Mocha.js, Karma, Buster • 2012 - Intern, Nightwatch.js • 2013 - Protractor, ReactTestUtils • 2014 - Jest (React), Machiatto, Ava

• 2009 - TestSwarm, Inspec, YUI Test • 2010 - Sinon.js, JSLint, JSLitmus, JSHint • 2011 - Mocha.js, Karma, Buster • 2012 - Intern, Nightwatch.js • 2013 - Protractor, ReactTestUtils • 2014 - Jest (React), Machiatto, Ava • 2015 - Yolpo

СТИЛИ ТЕСТОВ

ASSERT SHOULD EXPECT

🙀 😺 😻

СТИЛИ ПРОВЕРОК

DESCRIBE { IT { EXPECT() } }

TEST { ASSERT() }

CONTEXT { SHOULD() }

🙀

СТИЛИ БЛОКОВ

SPEC.WHEN() .AND() .SHOULD()

TEST('foo', t => { t.pass() })new Y.Test.Case({testName: function() {…}})

😻

😺

{блоки}

{блоки}или .цепочки.

КРАТКИЙ ОБЗОР

Intern Software testing for humans

Features

• Write tests in JavaScript using any style you like (TDD, BDD, whatever!).

• Run tests straight from your browser, right in Node.js, or against many browsers & platforms at the same time with the Intern test runner.

• Get full code coverage reporting so you know what’s left to test.

• Seamlessly run tests on Sauce Labs, BrowserStack, TestingBot, or your own Selenium Grid.

• Ready for continuous integration? Intern fully integrates with Travis CI, Jenkins, and TeamCity out of the box.

https://theintern.github.io/

Intern What can Intern do that others can’t?

Intern What can Intern do that others can’t?

😺😺

Jasmine

Jasmine is a Behavior Driven Development testing framework for JavaScript. It does not rely on browsers, DOM, or any JavaScript framework. Thus it's suited for websites, Node.js projects, or anywhere that JavaScript can run.Features • Simple setup for node through jasmine-node • Headless running out of the box • Nice fluent syntax for assertions built-in, and does play pretty well with other assertion libraries

• Supported by many CI servers (TeamCity, Codeship, etc.) • Descriptive syntax for BDD paradigm

Nov 23, 2008 – Mar, 2016

https://github.com/jasmine/jasmine

Jasmine Nov 23, 2008 – Mar, 2016

describe("A suite", function() { it("contains spec with an expectation", function() { expect(true).toBe(true); }); });

Mocha.js

Mocha is a simple, flexible, fun JavaScript test framework for node.js and the browser. Features • Simple setup • Asynchronous testing is a breeze • Headless running out of the box

• Allows use of any assertion library that will throw exceptions on failure, such as Chai • Supported by some CI servers and plugins for others (jenkins has a maven plugin) • Has aliases for functions to be more BDD-oriented or TDD-oriented • Highly extensible

Aug 28, 2011 – Mar, 2016

https://github.com/mochajs/mocha

Mocha.js

Assertion Libraries • should.js - BDD style shown throughout these docs • chai - expect(), assert() and should style assertions • expect.js - expect() style assertions • expectations - Jasmine-style expect() • unit.js - simple, fluent assertions • unexpected - extensible BDD assertion toolkit

Aug 28, 2011 – Mar, 2016

machiattobehavior driven test framework

Features • BDD friendly syntax • Context re-use between tests and files • Better structure for same context tests • Sync and async code testing • Browser and node.js

May 18, 2014 – Nov, 2014

https://github.com/alexbeletsky/machiatto

machiatto

second argument is missing a context re-use

Context re-use between tests and files

AvaFuturistic test runner

Even though JavaScript is single-threaded, IO in Node.js can happen in parallel due to its async nature. AVA takes advantage of this and runs your tests concurrently, which is especially beneficial for IO heavy tests. In addition, test files are run in parallel as separate processes, giving you even better performance and an isolated environment for each test file.

Switching from Mocha to AVA in Pageres brought the test time down from 31 sec to 11 sec.

Having tests run concurrently forces you to write atomic tests, meaning tests don't depend on global state or the state of other tests, which is a great thing!

Nov 16, 2014 – Mar, 2016

https://github.com/sindresorhus/ava

AvaFuturistic test runner

Nov 16, 2014 – Mar, 2016

test(t => { t.plan(1);

return Promise.resolve(3).then(n => { t.is(n, 3); }); });

test.cb(t => { t.plan(1);

someAsyncFunction(() => { t.pass(); t.end(); }); });

Nightwatch.jsUI automated testing framework

Nightwatch.js is an easy to use Node.js based End-to-End (E2E) testing solution for browser based apps and websites. It uses the powerful Selenium WebDriver API to perform commands and assertions on DOM elements.

Main Features • Clean syntax • Selenium server • CSS & Xpath support • Easy to extend • Built-in test runner • Cloud services support • Continuous integration support

Mar 11, 2012 – Feb, 2016

https://github.com/nightwatchjs/nightwatchhttp://nightwatchjs.org/

Nightwatch.jsUI automated testing framework

Mar 11, 2012 – Feb, 2016

module.exports = { 'Demo test Google' : function (browser) { browser .url('http://www.google.com') .waitForElementVisible('body', 1000) .setValue('input[type=text]', 'nightwatch') .waitForElementVisible('button[name=btnG]', 1000) .click('button[name=btnG]') .pause(1000) .assert.containsText('#main', 'Night Watch') .end(); } };

YolpoA tool to see the execution of plain and simple javascript code

May 3, 2015 – Jul, 2015

• Tests do not rely upon any other library except the one they are testing • Tests are 100% javascript. • Tests are running on the browser • Use Gist GitHub to store code; then get yolpo to run it.

https://github.com/lorenzoongithub/yolpohttp://www.yolpo.com/

http://www.yolpo.com/player/v.html?gist=d2d19a00747991285307&autoplay=2

JSLitmusJavaScript benchmark testing made easy

Aug 22, 2010 - Aug 27, 2010

JSLitmus is a lightweight tool for creating ad-hoc JavaScript benchmark tests. Features • Single-file install (JSLitmus.js) • Works on Firefox, Opera, Safari, IE, Google Chrome, and iPhone browsers • Works in node.js • Adaptive test cycles - tests take ~1-2 seconds, regardless of the operation • Google Chart + TinyUrl integration • Open Source MIT-style license (see source)

JSLitmus determines the performance of your code by running it repeatedly until a sufficient amount of time has passed to obtain a valid result. This requires an iteration loop, of course. But for fast operations - operations that can be run more than ~100K/second or more - the overhead of the iteration loop can overshadow the performance of your test code. JSLitmus has two features that address this issue.

http://www.broofa.com/Tools/JSLitmus/ https://github.com/broofa/jslitmus

JSLitmus <script src="jslitmus.js"></script> <script> // Create a test to see how fast Array.join() is on 10K elements var a = new Array(10000); jslitmus.test('Join 10K elements', function() { a.join(' '); });

// Log the test results jslitmus.on('complete', function(test) { console.log(test); });

// Run it! jslitmus.runAll(); </script>

In browser

JSLitmus var sys = require('sys'); var jslitmus = require('./jslitmus.js');

var a = new Array(10000); jslitmus.test('Join 10K elements', function() { a.join(' '); });

// Log the test results jslitmus.on('complete', function(test) { sys.log(test); });

// Run it! jslitmus.runAll();

In node.js

JSLitmus// Many developers are unaware how much influence the scope of a variable has // on script performance. To demonstrate this, we create the following // JSLitmus tests to measure how the "increment" operator performs when applied // to variables defined in different scopes

// First, test a variable in the global scope var global_var = 1; JSLitmus.test('global', function(count) { while (count--) global_var++;} );

// Now test one that's in a function's local scope JSLitmus.test('local', function(count) { var local_var = 1; while (count--) local_var++; });

// Try a variable bound to a closure function. Prototype and JQuery developers // should find this particularly interesting. JSLitmus.test('closure', (function() { var closure_var = 1; return function(count) {while (count--) closure_var++;} })() );

// Closure binding again, but this time with the variable bound through nested // closures. JSLitmus.test('multi-closure', (function() { var multi_var = 1; return (function() { return function(count) {while (count--) multi_var++;} })() })() );

BusterA powerful suite of automated test tools for JavaScript

A JavaScript test-runner built with Node.js. Very modular and flexible. It comes with it's own assertion library, but you can add your own if you like.

Main Features • Run tests from browser or headless with PhantomJS • Run on multiple clients at once • Don't need to run server/clients on development computer (no need for IE) • Run tests from command line (can be integrated in ant/maven) • Write tests xUnit or BDD style • Supports multiple JavaScript test frameworks • Defer tests instead of commenting them out • SinonJS built in • Auto-run tests on save • Proxies requests cross-domain • Possible to customize assertions, reporters • TextMate and Emacs integration

Mar 13, 2011 – Jan, 2016

https://github.com/busterjs/buster

Buster// This assertion does not exist! assert.notEquals(foo, bar);

// Instead: refute.equals(two, objects);

// And so on refute(false); refute.same(two, objects); refute.defined(something);

var buster = require("buster"); var assert = buster.referee.assert; var myLib = require("../lib/my-lib");

buster.testCase("A test case", { "test it": function () { assert(myLib.doIt()); } });

Buster$ buster-server buster-server running on http://localhost:1111

Open the browsers you want to run tests in and click the capture button.

First, start the server

Buster

$ buster-static Starting server on http://localhost:8282

Buster.JS also has a static browser runner that runs tests by opening a web page in a browser. This is similar to QUnit, Mocha

JestPainless JavaScript Unit Testing

May 11, 2014 – Mar, 2016

• Familiar Approach: Built on top of the Jasmine test framework, using familiar expect(value).toBe(other) assertions

• Mock by Default: Automatically mocks CommonJS modules returned by require(), making most existing code testable

• Short Feedback Loop (!): DOM APIs are mocked and tests run in parallel via a small node.js command line utility

Jestjest.unmock('../CheckboxWithLabel');

import React from 'react'; import ReactDOM from 'react-dom'; import TestUtils from 'react-addons-test-utils'; import CheckboxWithLabel from '../CheckboxWithLabel';

describe('CheckboxWithLabel', () => { it('changes the text after click', () => { // Render a checkbox with label in the document const checkbox = TestUtils.renderIntoDocument( <CheckboxWithLabel labelOn="On" labelOff="Off" /> );

const checkboxNode = ReactDOM.findDOMNode(checkbox);

// Verify that it's Off by default expect(checkboxNode.textContent).toEqual('Off');

// ... }); });

КАКИЕ ТЕНДЕНЦИИ

(ВОПРОСЫ К АУДИТОРИИ)

PRODUCTION TESTING

A/B TESTING

ГЕНЕРАЦИЯ ТЕСТОВ

RECORD/PLAY CODELESS TESTS

UI ДЛЯ СОЗДАНИЯ ТЕСТОВ

TIME TRAVEL TESTING

BIG DATA TESTING

INTERNET OF THINGSSO WHAT?

INTERNET OF THINGSSO WHAT? ——>API

ПРОВЕРКА АПЛОДИСМЕНТОВ

СПАСИБО

@SSELETSKYY