on testing

testing is good for your modules. even the ones you know work as intended. case in point: i wanted to refactor curses::toolkit::object::coordinates to use moose.however, refactoring such a core component of curses::toolkit without a test suite did not sound such a good idea...

therefore i wrote some tests for this module - and i discovered 2 bugs in this class. a small one that made an error message totally useless, and a serious one that affected the semantics of some methods.

so, not counting the fact that refactoring was quite easy once the test suite was in place, i also found some real bugs in the code. those tests were not very funny to write, but at least i know that the roi was very good! :-)

migration to moose - step 3

continuing ^[0] to port mpd modules to moose - this time it's poe::component::client::mpd turn. in order to do that, i had to fix the tests (using the shiny new test::corpus::audio::mpd) in order to be sure not to introduce regressions.

migration was quite easy, but a bit long. especially since i used a lazy_builder attribute for the connection, leading to commands sent before actual connection took place - and thus the tests failed with a cryptic message from poe::component::client::tcp ^[1]

the excellent moosex::poe was used too, but not in the connection, since it reuses various components having a more classic way of defining their events.

and now that the base code is moose-ified, i think i'll be able to gain more clarity while leveraging some moose levers.


[0] see this previous post and this one
[1] saying that {server} doesn't have a put method... as opposed as what's documented. but that's because at that time it has not (yet) morphed into a poe::wheel::readwrite. even if i'm at fault for not waiting the connected event, couldn't it provide a better error message?

test::corpus::* namespace

both audio::mpd and poe::component::client::mpd share the same files for their regression test suite. this includes some boring ogg files - weighing around 150 kb. although not that much, those files are duplicated between those 2 modules.

moreover, it means that everytime i upload a fix for one of those modules, i'm taking some more space on cpan for those exact same files - they never ever change...

something had to be done. i therefore wanted to split this raw test data, and put it in a module of its own. the other 2 modules would then get rid of the test data, and require the new one during the tests.

the question was of course how to name this module. i asked module-authors for advices, and david nicol proposed the test::corpus::* namespace.

since the idea is quite sound, i then released test::corpus::audio::mpd with the data (with some fixes to bring it up to mpd 0.15.x). audio::mpd has already been migrated to use this. i still need to do it for pococm, though.

anyway, this post is merely about the announce of the test::corpus:: namespace, to hold your test data instead of putting them in your dists. let us know what you think of it.

dist-zilla: compile tests can skip some modules

the compilation test plugin for dist-zilla now accepts a parameter to skip some modules. this can be handy to prevent testing some known-to-fail modules - yes, this can happen if your module does stuff assuming a given environment not currently in place (eg: test modules...).

just use the following in your dist.ini file:

[CompileTests]
skip = Test$

the skip parameter is interpreted as a regex, matched against the module name (not the file name of the module). in this example, all modules with a name ending with Test will not be tested for compilation.

external event loops no more part of poe

for those (such as me) that did not spotted it, external event loops for poe are now released as their own distribution. this means they aren't part of the poe distribution anymore, and you should add them to your prereqs...

of course, there's no way that dzil's autoprereq plugin could extract poe::loop::tk from code such as:


so, i had to use the regular prereq plugin to list it as a manual dependency. since dzil is using moose roles, it means more than one plugin can provide the prereqs, so i can continue using autoprereq beside this exception - woohoo!

in conclusion, games::pandemic 1.092660 is on its way, fixing a problem reported by a cpantester (thank you guys - you rock!)

more dzil goodness

i'm happy to announce yet another plugin for dist-zilla: critictests.

in your dist.ini, just add:

[CriticTests]

and dist-zilla will automatically create for you an xt/author/critic.t file that will criticize your code during author tests.

of course, since your code will be munged by dist-zilla, code that passes critics in your repo may fail after being rewritten... case in point: stricture use, since dist-zilla-plugin-pkgversion will insert a version just after the package statement - and thus before any use statements. in order to make your code compliant again, just move the use strict / warnings before the package. indeed, use is lexically scoped, and thus the scope of the use statement will be the whole file.

your code will now look like:

use strict;
use warnings;
package Foo::Bar;

and of course, perlcritic this time will now be happy.

shipping compile tests with dist-zilla

perl community places some emphasis on testing. which is great, since it enhances the overall quality of perl ecosystem.

the first, easiest test that one should provide is basic: making sure the modules compile. we have 2 options here:

• either create a test and add some use_ok('Foo::Bar'). but we then need to update it whenever we add more modules to our distribution - without forgetting to update test count.
• or, to get rid of this burden, ship a test that automatically finds the module and tries to compile them.

but shipping the exact same generic test is boring, and could be automated. which is now the case for dist::zilla with the new module dist::zilla::plugin::compiletests. just add the following in your dist.ini file:

[CompileTests]

and you will get a brand new t/00-compile.t that will test your modules and your scripts one by one. this my take some time if your dist ships a lot of modules having lots of prereqs, especially if they are a bit slow to compile such as Moose. but this will potentially find more compile errors in your modules.

polyglot: introducing our test environment

reminder: this article is part of a serie.

before we start, we need to set up a working environment. this environment should provide us with two things:

• a version control mechanism
• a test environment with some non-regression tests

the version control is needed to be sure that we get some points where we can go back in time. since we're going to mess our program quite hard, we'll be happy to check out a version that was working at a given time, and start messing in another direction.

the easiest way to go is of course to go with git. the following simple steps will create a git repository, and start tracking our polyglot program:

$ git init
$ vi fibonacci.txt
[fill in the perl program mentioned in the introduction]
$ git add fibonacci.txt
$ git ci -m 'first language: perl'

note: for easier reference, i will publish the changes to an online git repository. of course, i will only push the parts needed for a given blog entry, so don't expect to see all commits supporting all languages! :-)

that being done, we'll now create a test suite to make sure that our modifications won't break the languages that were previously working.

perl is well-known for its test automation. it has a strong history in regression testing: perl itself comes with over 200.000 internal tests, and the various distributions available on cpan almost all come each with a comprehensive test suite. i'll therefore use the standard test modules to come up with the polyglot test suite.

the simplest way to build our test suite is to use prove. each covered language will have a test file, emitting some tap. prove will harness those test files and report their status. each test will do, in order:

1. create a clean directory where the test will take place
2. copy the program in this directory. since some languages are picky on the extension, the destination will have a file extension matching the language.
3. run the appropriate command, capturing stdout / stderr
4. compare the output with what was expected
5. remove the test directory

since we're lazy, this will be in a module that will export a single function fibo_check() doing everything mentioned above. the complete module, located in a lib directory, will look like:

                                                                                           
package FiboTest;                                                              

use File::Copy;
use File::Path qw{ mkpath rmtree };
use FindBin    qw{ $Bin };
use Test::More tests => 1;

use parent qw{ Exporter };
our @EXPORT = qw{ fibo_check };

# fibo_check( $lang, $ext, $cmd )
sub fibo_check {
    my ($lang, $ext, $cmd) = @_;

    # clean room for the test
    my $dir = "$Bin/tmp/$lang";
    rmtree($dir);
    mkpath($dir);

    # get expected result
    my $want = do { local $/; <DATA> };

    # run the test
    copy( "$Bin/../fibonacci.txt", "$dir/fibonacci.$ext");
    my $command = sprintf "cd $dir; $cmd 2>&1", "fibonacci.$ext";
    my $have = qx{ $command };

    is( $have, $want, "language $langtest");

    # clean after ourselve
    rmtree($dir) unless $ENV{POLYGLOT_DEBUG};
}

1;
__DATA__
1
1
2
3
5
8
13
21
34
55

this fibo_check() function really implements the steps described above. the only things worth noting are:

• the fact that we're storing the expected output in the data section of the module
• the last line, that will _not_ remove the test environment if the environment variable POLYGLOT_DEBUG is set.

with this helper module, each test (one language per file) in the t/ directory
is really dumbed-down, and will look like the following:

use strict;
use warnings;
use FiboTest;
fibo_check('perl', 'pl', "perl %s");

running the test suite is as simple as:

$ prove -l t
t/perl....ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.03 cusr  0.00 csys =  0.05 CPU)
Result: PASS

running a single language test will be achieved with:

$ prove -l t/perl.t
t/perl....ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.03 cusr  0.00 csys =  0.06 CPU)
Result: PASS

of course, let's not forget to commit this in git:

$ git add lib
$ git ci -m 'test helper module'
$ git add t
$ git ci -m 'test file for perl'

and for even better checkpoint, let's add a git tag, named after the implemented language:

$ git tag perl

our working environment is now ready, and we can now go wild - knowing that we have a safety belt and a control mechanism. let the fun begin!