real world cross-platform testing

04/10/23Real Life Cross-Platform Testing1

Real Life Cross-Platform TestingReal Life Cross-Platform Testing

Peter Edwards

MiltonKeynes.pmPerl Technical Talk8th July 2008

[email protected]


ContentsContents Background aka "Real Life" Cross-Platform Testing

Add Windows Testing Under Unix Test::MockObject Test::MockModule Running Unix unit tests under Windows Future Plans For Testing Summary and Links


Background aka "Real Life"Background aka "Real Life"

Content Management System used at BBC to enter XML documentsthat are later transformed to make public websites

Client-side– GUI using WxPerl (WxWidgets)– WYSIWYG editing– Talks SOAP over HTTP to server– Runs under ActiveState Perl

Server-side– Handles SOAP requests– Stores document blobs in filesystem– Stores indexes, metadata in Oracle database– Runs under Solaris Perl

Usage– 100s of users– Time critical publishing : failure during release is not an option


Cross-PlatformCross-Platform

CMS code running on Windows and Solaris

Solaris perl 5.8.8 $ perl -V Summary of my perl5 (revision 5 version 8 subversion 8) configuration: Platform: osname=solaris, osvers=2.10, archname=sun4-solaris

Windows ASPerl 5.8C:\WINNT>perl –VSummary of my perl5 (revision 5 version 8 subversion 6) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread


TestingTesting Unit tests for dev Automated overnight smoke testing of unit tests Dev / Staging Test / Live environments Manual release test on staging test area using

Windows appProblems Lots of tests for server side code, very few for

client side because difficult to run 'use Wx' code on Unix in batch

Existing tests run on Unix, fail on Windows


Add Windows Testing Under UnixAdd Windows Testing Under UnixNeed to write lots of client-side tests for1) GUI

WxPerl -> Gtk+ under Solaris‘Use Wx’ was failing because no X displayProblems with font sizing and window alignmentWindows-specific components, e.g. ActiveX Altova editor

InstallationShortcuts, registry Win32::OLE, unzipping archives to Windows Apps dir etc.

Solutions1) Use Xvfb

$ alias runxvfb='Xvfb :10 -dev vfb screen 0 1152x900x8 > /dev/null 2>&1 &'Lets you check code compile and call many routinesBut how do you test UI rendered properly - interpreting the virtual screen

bitmaps is too hard!2) Sandboxing and mocking

Mock required Win32 functionsMake them do file I/O to a sandbox areaTest::MockObject - Perl extension for emulating troublesome interfacesTest::MockModule - Override subroutines in a module for unit testing


Test::MockObject 1Test::MockObject 1 Helpers

sub make_mock_obj_in_class { my $class = shift; my $obj = Test::MockObject->new; $obj->fake_module($class); $obj->fake_new($class); return $obj; }

sub dump_mock_calls { my $mockobj = shift; my $i = 1; while ( my $name = $mockobj->call_pos($i) ) { diag " call $i: $name"; my @args = $mockobj->call_args($i); for (0 .. $#args) { diag ' arg '.($_ +1).': '; diag Dumper($args[$_]); } $i++; } }


Test::MockObject 2Test::MockObject 2 Mocking

my $wx = make_mock_obj_in_class( 'Wx' ); my $mock_WxPerlSplashProgress = make_mock_obj_in_class( 'Wx::Perl::SplashProgress' ); $mock_WxPerlSplashProgress->set_true(qw( SetLabelColour SetIcon Show SetValue Update Destroy )); $mock_WxPerlSplashProgress->mock( SetLabel => sub { diag ' SetLabel: '.$_[1] } );

$mock_Win32OLE = make_mock_obj_in_class( 'Win32::OLE' ); $mock_Win32OLE->mock( 'SpecialFolders', sub { shift } ); $mock_Win32OLE->mock( 'AppData', sub { return catdir(qw(data win32), 'Application Data') } ); $mock_Win32OLE->mock( 'StartMenu', sub { catdir(qw(data win32 startmenu)) } ); $mock_Win32OLE->mock( 'Desktop', sub { catdir(qw(data win32 desktop)) } ); $mock_Win32Shortcut = make_mock_obj_in_class( 'Win32::Shortcut' ); $mock_Win32Shortcut->mock( 'Load', sub { my ($self, $filename) = @_; $self->{content} = read_file($filename); return 1; } ); $mock_Win32Shortcut->mock( 'Path', sub { my ($self, $path) = @_; $self->{content} = $path; } ); $mock_Win32Shortcut->mock( 'Arguments', sub { my ($self, $args) = @_; $self->{content} .= ' '.$args . "\r\n"; } ); $mock_Win32Shortcut->mock( 'Save', sub { my ($self, $filename) = @_; write_file($filename, $self->{content} . "writetime ". gmtime() . "\r\n"); return 1; } ); $mock_Win32Shortcut->set_true(qw( ShowCmd Description IconLocation Close )); { no strict 'refs'; *{'Win32::Shortcut::SW_SHOWMINNOACTIVE'} = sub {}; }


Test::MockObject 3Test::MockObject 3 Testing$mock_WxPerlSplashProgress->clear();is( $i->_install_loginscript, 1, '$i->_install_loginscript' );dump_mock_calls($mock_IFLDesktopLoginScript);$mock_IFLDesktopLoginScript->called_pos_ok( 3, 'install', 'called

IFL::Desktop::LoginScript->install' );dump_mock_calls($mock_WxPerlSplashProgress);$mock_WxPerlSplashProgress->called_pos_ok( 4, 'SetLabel', 'called

Wx::Perl::SplashProgress->SetLabel' );$mock_WxPerlSplashProgress->called_args_pos_is( 4, 2, 'Checking

login script' );$mock_WxPerlSplashProgress->called_pos_ok( 7, 'SetLabel', 'called

Wx::Perl::SplashProgress->SetLabel' );$mock_WxPerlSplashProgress->called_args_pos_is( 7, 2, 'Installing

login script...' );


Test::MockModule 1Test::MockModule 1 Helper

sub mock_module { my ($module,$options,@functions) = @_; my $no_auto = defined($options->{no_auto}) ? $options->{no_auto} : 1; my $create_new = defined($options->{create_new}) ? $options->{create_new} : 1; my $testmockmodule = new Test::MockModule($module, no_auto => $no_auto); my $object; if ($create_new) { $object = bless {}, $module; $testmockmodule->mock('new',sub { $logger->log($module,'new',@_); return $object }); } for my $function (@functions) { $testmockmodule->mock($function,sub { $logger->log($module,$function,@_) }); } no strict 'refs'; push @{$module . "::ISA"},'Exporter'; my $module_path = $module; $module_path =~ s{::}{/}xmsg; $module_path .= '.pm'; $INC{$module_path} = "1 (Inserted by mock_module())"; return $testmockmodule, $object;}


Test::MockModule 2Test::MockModule 2 Mocking

my ($mock_wx_activex_ie, $mock_wx_activex_ie_object) = mock_module('Wx::ActiveX::IE',{});my ($mock_wx_activex_event, $mock_wx_activex_event_object) = mock_module('Wx::ActiveX::Event',{},@Wx::Event::EXPORT_OK);my ($mock_wx_panel,$mock_wx_panel_object) = mock_module('Wx::Panel',{}, qw( SetSizer ));my ($mock_wx_boxsizer,$mock_wx_boxsizer_object) = mock_module('Wx::BoxSizer',{}, qw( Add ));

Tests - use your objects as normal… then check call sequencemy @mf_calls = $logger->filter({'FLIPClient::UI::MicroForms' => []});my $call = shift(@mf_calls);is($call->{function},'set_template','position_change (' . $test->{name} . ') calls set_template');ok($call->{args}->[1] =~ $test->{template},'position_change (' . $test->{name} . ') sets

template');

$call = shift(@mf_calls);is($call->{function},'set_data','position_change (' . $test->{name} . ') calls set_data');is_deeply($call->{args}->[1],$test->{data},'position_change (' . $test->{name} . ') sets data');


Running Unix unit tests under Windows 1Running Unix unit tests under Windows 1

Some libraries shared between Unix and Windows;not being tested properly client-side

Perl Portability– "perldoc perlport“ http://perldoc.perl.org/5.8.8/perlport.html

"When the code will run on only two or three operating systems, you may need to consider only the differences of those particular systems. The important thing is to decide where the code will run and to be deliberate in your decision.“

– Only worrying about Windows and Unix; OpenVMS support is hard

binmode and chomp - binmode saves headaches on Windows like EOF ^Z; watch out for CR-LF

use File::Spec::Functions rather than Unix pathsYES : my $path = rel2abs( catdir(qw( data local cache file.txt ));NO : my $path = './data/local/cache/file.txt';


Running Unix unit tests under Windows 2Running Unix unit tests under Windows 2

Generic configuration interface with platform-specific subclassesSystem.pm |-- System/Win32.pm |-- System/Unix.pmusing File::Spec::Functions for paths

Change tests from path strings to regexes using a quote path separator my $script = $i->startup('remote');NO : is( $script, 'scripts/FLIP_real.PL', '$i->startup("remote") script’YES : $ps = ($^O eq 'MSWin32') ? "\\" : '/'; $qps = quotemeta $ps; like( $script, qr{ scripts [$qps] FLIP_real.pl \z }xms, '$i-

>startup("remote") script' );Note PBP style regex

Actually run the tests on multiple platforms


Future Plans For TestingFuture Plans For Testing

Automate application release test under Windows– Win32::GuiTest (or pay for WinRunner)


Summary and LinksSummary and Links Summary

– "perldoc perlport“– Write cross-platform tests from the outset; convert old ones– Mock platform-specific GUI or system library calls– Automate tests (life is short) and get as much coverage as

possible

Links– WxPerl http://wxperl.sourceforge.net/– WxWidgets http://docs.wxwidgets.org/trunk/– "Perl Testing: A Developer's Notebook" Ian Langworth &

chromatic, O'Reilly Media, Inc., 2005 http://preview.tinyurl.com/5k6wnc

Thank you. Any Questions?

real world cross-platform testing

Technology