polyglot: introducing our test environment

reminder: this article is part of a serie.

before we start, we need to set up a working environment. this environment should provide us with two things:

• a version control mechanism
• a test environment with some non-regression tests

the version control is needed to be sure that we get some points where we can go back in time. since we're going to mess our program quite hard, we'll be happy to check out a version that was working at a given time, and start messing in another direction.

the easiest way to go is of course to go with git. the following simple steps will create a git repository, and start tracking our polyglot program:

$ git init
$ vi fibonacci.txt
[fill in the perl program mentioned in the introduction]
$ git add fibonacci.txt
$ git ci -m 'first language: perl'

note: for easier reference, i will publish the changes to an online git repository. of course, i will only push the parts needed for a given blog entry, so don't expect to see all commits supporting all languages! :-)

that being done, we'll now create a test suite to make sure that our modifications won't break the languages that were previously working.

perl is well-known for its test automation. it has a strong history in regression testing: perl itself comes with over 200.000 internal tests, and the various distributions available on cpan almost all come each with a comprehensive test suite. i'll therefore use the standard test modules to come up with the polyglot test suite.

the simplest way to build our test suite is to use prove. each covered language will have a test file, emitting some tap. prove will harness those test files and report their status. each test will do, in order:

1. create a clean directory where the test will take place
2. copy the program in this directory. since some languages are picky on the extension, the destination will have a file extension matching the language.
3. run the appropriate command, capturing stdout / stderr
4. compare the output with what was expected
5. remove the test directory

since we're lazy, this will be in a module that will export a single function fibo_check() doing everything mentioned above. the complete module, located in a lib directory, will look like:

                                                                                           
package FiboTest;                                                              

use File::Copy;
use File::Path qw{ mkpath rmtree };
use FindBin    qw{ $Bin };
use Test::More tests => 1;

use parent qw{ Exporter };
our @EXPORT = qw{ fibo_check };

# fibo_check( $lang, $ext, $cmd )
sub fibo_check {
    my ($lang, $ext, $cmd) = @_;

    # clean room for the test
    my $dir = "$Bin/tmp/$lang";
    rmtree($dir);
    mkpath($dir);

    # get expected result
    my $want = do { local $/; <DATA> };

    # run the test
    copy( "$Bin/../fibonacci.txt", "$dir/fibonacci.$ext");
    my $command = sprintf "cd $dir; $cmd 2>&1", "fibonacci.$ext";
    my $have = qx{ $command };

    is( $have, $want, "language $langtest");

    # clean after ourselve
    rmtree($dir) unless $ENV{POLYGLOT_DEBUG};
}

1;
__DATA__
1
1
2
3
5
8
13
21
34
55

this fibo_check() function really implements the steps described above. the only things worth noting are:

• the fact that we're storing the expected output in the data section of the module
• the last line, that will _not_ remove the test environment if the environment variable POLYGLOT_DEBUG is set.

with this helper module, each test (one language per file) in the t/ directory
is really dumbed-down, and will look like the following:

use strict;
use warnings;
use FiboTest;
fibo_check('perl', 'pl', "perl %s");

running the test suite is as simple as:

$ prove -l t
t/perl....ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.03 cusr  0.00 csys =  0.05 CPU)
Result: PASS

running a single language test will be achieved with:

$ prove -l t/perl.t
t/perl....ok
All tests successful.
Files=1, Tests=1,  0 wallclock secs ( 0.02 usr  0.01 sys +  0.03 cusr  0.00 csys =  0.06 CPU)
Result: PASS

of course, let's not forget to commit this in git:

$ git add lib
$ git ci -m 'test helper module'
$ git add t
$ git ci -m 'test file for perl'

and for even better checkpoint, let's add a git tag, named after the implemented language:

$ git tag perl

our working environment is now ready, and we can now go wild - knowing that we have a safety belt and a control mechanism. let the fun begin!