07.04.2008 @ vienna.pm aldo “dada” calpini 1/22dada.perl.it/talks/tyfs.pdf · 07.04.2008 @...
TRANSCRIPT
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 11/22/22
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 22/22/22
Thank you for smokingThank you for smoking
……wtf?wtf?
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 33/22/22
Thank you for smokingThank you for smoking
no, actually not.no, actually not.
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 44/22/22
Thank you for smokingThank you for smoking
but…but…
SMOKE TESTINGSMOKE TESTING
can be healthy!can be healthy!
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 55/22/22
Thank you for smokingThank you for smoking
what is…what is…
SMOKE TESTINGSMOKE TESTING
““...a collection of tests [..] sometimes performed by the ...a collection of tests [..] sometimes performed by the automated system that builds the final software”automated system that builds the final software”
““...a collection of written tests that are performed on a ...a collection of written tests that are performed on a system prior to being accepted for further testing. This system prior to being accepted for further testing. This is also known as a build verification test”is also known as a build verification test”
(Wikipedia)(Wikipedia)
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 66/22/22
Thank you for smokingThank you for smoking
...in Perl words:...in Perl words:
make testmake test
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 77/22/22
Thank you for smokingThank you for smoking
...in Perl words:...in Perl words:
make testmake test
(automated)(automated)
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 88/22/22
Thank you for smokingThank you for smoking
SMOKE TESTINGSMOKE TESTINGin software development cycle:
REPOSITORY
COMMIT
HACK HACK HACK
DEVELOPER
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 99/22/22
Thank you for smokingThank you for smoking
SMOKE TESTINGSMOKE TESTINGin software development cycle:
DEVELOPER
REPOSITORY
UPDATE
DEVELOPER
DEVELOPER
DEVELOPER
UPDATE
UPD
ATE UPD
ATE
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1010/22/22
SMOKE SERVERTEST R
EPOR
T
TEST REPORT
TEST REPO
RTDEVELOPER
DEVELOPER
DEVELOPER
DEVELOPER
TEST REPORT
Thank you for smokingThank you for smoking
SMOKE TESTINGSMOKE TESTINGin software development cycle:
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1111/22/22
Thank you for smokingThank you for smoking
who's using…who's using…
SMOKE TESTINGSMOKE TESTING PerlPerl
• 187.867 tests187.867 tests
• 17 architectures, 15 major platforms17 architectures, 15 major platforms
ParrotParrot• ~11.000 tests~11.000 tests
• 7 architectures, 8 major platforms7 architectures, 8 major platforms
Pugs Pugs (currently down)(currently down)
• ~16.000 tests~16.000 tests
• 6 architectures, 5 major platforms (eg. where GHC runs)6 architectures, 5 major platforms (eg. where GHC runs)
and then there's CPAN Testersand then there's CPAN Testers
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1212/22/22
Thank you for smokingThank you for smoking
the benefits ofthe benefits of
SMOKE TESTINGSMOKE TESTING
catches hard to spot errorscatches hard to spot errors• eg. feature X breaks Y on platform Zeg. feature X breaks Y on platform Z
• or simply, breaks something with a different setup than the developer's oneor simply, breaks something with a different setup than the developer's one
......before before going to releasegoing to release
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1313/22/22
Thank you for smokingThank you for smoking
the benefits ofthe benefits of
SMOKE TESTINGSMOKE TESTING
catches hard to spot errorscatches hard to spot errors• eg. feature X breaks Y on platform Zeg. feature X breaks Y on platform Z
• or simply, breaks something with a different setup than the developer's oneor simply, breaks something with a different setup than the developer's one
......before before going to releasegoing to release
and test reports are cool :-)and test reports are cool :-)
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1414/22/22
Thank you for smokingThank you for smoking
do do youyou need need
SMOKE TESTINGSMOKE TESTING
??
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1515/22/22
Thank you for smokingThank you for smoking
do do youyou need need
SMOKE TESTINGSMOKE TESTING
?? probably notprobably not ...but if you really do, try Smolder:...but if you really do, try Smolder:
• http://www.sourceforge.net/projects/smolderhttp://www.sourceforge.net/projects/smolder
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1616/22/22
Thank you for smokingThank you for smoking
but still...but still...
test reports are cooltest reports are cool
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1717/22/22
Thank you for smokingThank you for smoking
enterenter
TAPTAPTest Anything ProtocolTest Anything Protocol
example:example:1..61..6
ok 1ok 1
ok 2ok 2
ok 3 – this is a commentok 3 – this is a comment
ok 4ok 4
ok 5ok 5
not ok 6 damn!not ok 6 damn!
# Failed test 'damn!'# Failed test 'damn!'
# at t/foobar.t line 10.# at t/foobar.t line 10.
# got: 'foo'# got: 'foo'
# expected: 'bar'# expected: 'bar'
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1818/22/22
Thank you for smokingThank you for smoking
enterenter
Test::TAP::ModelTest::TAP::Model
based on Test::Harness::Strapsbased on Test::Harness::Straps• (actually old, replaced by TAP::Parser, but we'll have to live with it)(actually old, replaced by TAP::Parser, but we'll have to live with it)
can run tests (as can run tests (as make testmake test does) and collect results does) and collect results in a Perl data structurein a Perl data structure
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 1919/22/22
Thank you for smokingThank you for smoking
enterenter
Test::TAP::HTMLMatrixTest::TAP::HTMLMatrix
uses Test::TAP::Model to produce super cool HTML test uses Test::TAP::Model to produce super cool HTML test reportsreports
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 2020/22/22
Thank you for smokingThank you for smoking
how to use it:how to use it:
~/bin/smoke~/bin/smoke#!/usr/bin/perl
use strict;
use warnings;
use Test::TAP::Model;
use Test::TAP::HTMLMatrix;
my $title = $ARGV[0] ? "$ARGV[0] test report" : undef;
eval "use blib;"; # nevermind if we're not testing a distribution
my $tap = Test::TAP::Model::Visual>new_with_tests(glob('t/*.t'));
$tap>run_tests();
my $v = Test::TAP::HTMLMatrix>new($tap, $title);
print $v>html;
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 2121/22/22
Thank you for smokingThank you for smoking
how to use it:how to use it:
chdir to a folder with testschdir to a folder with tests• usually your module/distribution folderusually your module/distribution folder
run: run: smoke > report.htmlsmoke > report.html
now you're ready to upload report.html to your website, now you're ready to upload report.html to your website, email it to friends, print it and use it to impress girls.email it to friends, print it and use it to impress girls.
automationautomation is left as an exercise to the reader :-) is left as an exercise to the reader :-)
07.04.2008 @ Vienna.pm07.04.2008 @ Vienna.pm Aldo “dada” Calpini <[email protected]>Aldo “dada” Calpini <[email protected]> 2222/22/22
Thank you for smokingThank you for smoking
that's allthat's all some reference:some reference:
• Perl smoke tests: Perl smoke tests: http://www.nntp.perl.org/group/perl.daily-build.reports/http://www.nntp.perl.org/group/perl.daily-build.reports/
• Parrot smoke tests: Parrot smoke tests: http://smoke.parrotcode.org/smoke/http://smoke.parrotcode.org/smoke/
• Pugs: Pugs: http://www.pugscode.orghttp://www.pugscode.org
• in particular: in particular: http://svn.pugscode.org/pugs/util/testgraph.plhttp://svn.pugscode.org/pugs/util/testgraph.pl
• TAP: TAP: http://testanything.orghttp://testanything.org
• Smolder: Smolder: http://www.sourceforge.net/projects/smolderhttp://www.sourceforge.net/projects/smolder
thank you (for smoking)thank you (for smoking)