the great renaming: Dist::Zilla::Plugin::CriticTests

oops! kentnl did it again. :-)

this time, dist::zilla::plugin::critictests has been renamed to dist::zilla::plugin::test::perl::critic. once again, change will be trivial for authors [0]. previous module will continue working, for at least one year counting from today.

github repository has also changed and is available here: https://github.com/jquelin/dist-zilla-plugin-test-perl-critic

[0] and this time, i did not fubar-ed the code! :-|

the great renaming: Dist::Zilla::Plugin::CompileTests

in a move to rationalize the plugin namespace of dist::zilla, kentnl started a  great renaming: dzp:compiletests has been renamed to dzp:test::compile

previous module is deprecated, and may be removed later on (but not before one year, that is 2012-08-27). in the meantime, it will continue working (although with a warning).

what does it mean for dzil users? well, nothing much, since old module will continue working as is. however, they should better migrate their dist.ini from:

[CompileTests]

to:

[Test::Compile]

nothing else is needed... for now, since i guess kentnl will continue proposing patches / pullreqs to dzil:*tests plugin authors! ;-)

also, please note that the github repository has been renamed and is now available at a different url. ditto for cpan, since the dist is now named dist-zilla-plugin-test-compile.

new dist::zilla command: pot

i just released dist::zilla::app::command::pot, providing dist::zilla with a new pot command (not the plant, you hippy!).

this command allows to (re-)generate a messages.pot file holding all strings to be translated from your module. If a messages.pot file is found, it will update it. otherwise, you will be prompted for a file location, with a default matching Locale::TextDomain settings:

$dzil pot
[...]
[DZ] Trying to find a messages.pot file...
[DZ] No messages.pot found - enter your own.
messages.pot to use [lib/LocaleData/Foo-Bar-messages.pot]:
[DZ] Running xgettext...

yup, underneath it just runs xgettext. and currently, it only finds calls to T() - but i'll add an option to provide your own convention such as _(), gettext(), etc.

supply your own perlcriticrc to dzil test plugin

if you wanted the ability to supply your own perlcriticrc while using dist-zilla-plugin-critictests, then stephen scaffidi is your hero of the day. indeed, he just implemented this - and i had to do the tedious work of typing "dzil release". oh well, i can say that life is difficult. :-)

roll your own dzil tutorial

rjbs created a roll-your-own dzil tutorial. this brings back memories, maybe there should be monsters in it. otoh, since it mentions module::install, maybe the monster quota is already met. :-)

anyway, the tutorial is a nice read for those wanting to learn dist-zilla. there are some glitches here and there (eg, it still mentions AllFiles plugin instead of GatherDir, and other v1 to v2 misses - i'll send patches), but is otherwise wicked cool.

and some of my plugins are even mentioned! :-)

github power

i started using github somewhere in 2009 iirc. i was a bit reluctant to use it back at that time, especially because of its speed (or lack of, actually)... and because i already hosted my projects of repo.or.cz.

however, i'm now really glad i started using it. speed is no longer a problem, and it provides really nice features. cia integration, network graphs, comments on commits, watching repos (even if i'm not totally happy with it right now), and of course... pull requests!

i was away this week-end, and found 4 pull requests in my mailbox for my modules when i came back. integrating other's work has never been easier with git and github which allows people to fork at will. that's the new way of writing code: create the basic stub of your module, and wait for others to enhance it! :-) git plugin for dist-zilla now supports pushing to a different branch, supports the new dist-zilla-tester framework and the bundle @git now accepts multi-valued args.

dvcs are a bit harder to grasp at first, but once you understand the concepts, they're really a killer application - especially when coupled with a cooperative platform such as github.

dzil activity

of course, you are aware that dzil v2 is out, with lots of exciting new features, brought to you by rjbs and the perl foundation.

but it's only one side of the activity floating around dist-zilla... indeed, as dagolden, aevar and others try dzil, they (of course) like it, and start contributing their own plugins.

here's a list of new plugins landed on cpan... first some additional author tests:

• CheckChangesTests - check changelog
  
• CheckExtraTests - runs xt/ tests, but don't copy them to t/
  
• DistManifestTests - check manifest
  
• HasVersionTests - check all your modules have a version
  
• KwaliteeTests - check your kwalitee
  
• MinimumVersionTests - check minimum perl version needed
  
• PodSpellingTests - check pod spelling
  
• PortabilityTests - check if your code is portable
  
• SynopsisTests - check if code in your synopsis compiles
  
• UnusedVarsTests - detects unused vars
  
• ReportVersions - additional test to report version
  

some plugins to complete meta-data:

• Bugtracker - http://rt.cpan.org/Public/Dist/Display.html?Name=xxxx
• HomePage - http://search.cpan.org/dist/xxxx
  

some plugins to customize your build process:

• ModuleBuild::Custom - for mb
• OverridableMakeMaker - for eumm

some plugins to generate additional files:

• FatPacker - creates a script with all dependencies packed
• InstallGuide - creates an INSTALL file
• ReadmeMarkdownFromPod - README.mkdn file

some plugins to compute your next version number:

• BumpVersionFromGit - version taken from last git tag
• VersionFromPrev - classic perl versions, 1.00 to 1.99

and finally some bundles:

• @AVAR - ævar arnfjörð bjarmason's take on it
  
• @MARCEL - marcel grünauer
• @PDONELAN - patrick donelan

you can install all of them in one go via task-dist-zilla.

they are also available in mandriva, and suggested when installing dist-zilla...

how to obsolete a cpan dist?

my autoprereq plugin for dist-zilla is now going core. however, it's currently existing as a dist of its own within cpan... since the module does not change name, what is needed to make sure rjbs can upload dzil v2 when it's ready? is giving co-maint rights on pause enough? will i have to remove previous tarballs?

the same kind of question can be asked for obsoleting a given module. there's (afaik) no way to tell cpan / cpanplus that foo::bar is replacing bar::foo... those problems are tackled for linux distribution packages, could we reuse some of their logic here?

activity in git plugin for dist-zilla

it's been a busy week for dist-zilla-plugin-git...

first, there was some test fixes to work with latest dist-zilla (thanks to ricardo), and also to support git 1.7.

the git plugin then saw support for annotated tags which is now the default - but christopher madsen restored the possibility to use lightweight tags. he then refactored commit message generation, in order to allow a custom commit message. not only that, but he also brought to ability to specify which files are allowed to be dirty, and automatically committed.

then david golden jumped in, and scratched his itches: support for empty lines in changelog and possibility to specify which branch(es!) to push to. all of that with new tests - and some existing test fixes! \o/

git support in dist-zilla is now in a pretty good shape. report bugs if you miss your pet-peeve... or even better: fork the code and send me pull requests once the code is doing what you want!

which module to extract perl prereqs?

in dzil plugin autoprereq, i'm extracting prereqs from the dist modules. i want this extract to be fast, based on the actual code (not makefile.pl or meta.yml, since the goal is to generate them), and as accurate as possible. it should also find base classes, moose roles and other "hidden" dependencies. finally, it should extract the minimum version needed for a given module, including minimum perl version.

my first version was regex-based. i can already see your horrified face - but really it wasn't so bad, since it only needed to find some specific statements such as uses and requires. current version is using ppi, which is better suited for corner cases.

however, long-term makes me think that it would be better to rely on an external module. so, what are the alternatives out there on cpan, and can i use them in autoprereq?

• b::perlreq - parses the file, but reports file (File/Basename.pm) instead of modules, and is generally more suited for rpm
  
• module::extract::use - using ppi to parse a file, but extracts only use & require statements (no inheritance, moose roles, etc). also, it reports no minimum version extraction, only a list of modules.
  
• test::dependencies - using either b::perlreq (see above) or a regex scheme underneath
  
• module::scandeps - runs the file (which is slow), and finds all modules included - and sometimes a bit more (eg: file::homeDir::darwin is found for a module using file::homedir, even on a unix platform). can also run as a static analyser, but calls cpanplus (?!) which is slow.
  
• module::info - regex based
  
• module::cpants::generator::prereq - parses makefile.pl, where i want sthg that parses actual code
  
• module::cpants::kwalitee::prereq - parse meta.yml, makefile.pl or build.pl

so, no module was doing exactly what i wanted... since i am using ppi and that module::extract::use does the same, i contacted brian d foy to see whether he would be interested in additional extractions (moose roles, base classes, etc.) for this module. he was, so those enhancements are now pushed on my github clone.

i'm now waiting for a new release of this module with my enhancements, meaning that i can get rid of this part of the code in dzil autoprereq. which was, if you recall, the original goal! :-)

next padre version will (finally) recognize dist-zilla projects

padre 0.55, due tomorrow, will finally recognize dist-zilla projects correctly. among other things, it means that it will set @INC accordingly for syntax checking, keep the directory browser at the project root, etc.

it took a bit of time since the "installer" detection is spread out in different places in padre... this needs refactoring, for those interested.

some dzil goodness coming

i scratched some itches in dist-zilla, and hopefully the result will be useful to you too... some are rather trivial, while some are more feature-ful.

on the it-s-the-details-that-count front, dzil clean will now remove *~ files lingering in your local copy.

dzil command now accepts a -I argument that adds a directory to perl @INC (same as perl's -I option). which means dzil plugin author can now do:

$ dzil -Ilib release

and release their distribution using the latest version of their plugin...

a new run subcommand has been added, which is doing more or less the following:

$ dzil build
$ rsync -avp My-Project-version/ .build/
$ cd .build
$ perl Makefile.PL     # or perl Build.PL
$ make                 # or ./Build     
$ export PERL5LIB=$PWD/blib/lib:$PWD/blib/arch
$ launch command defined by rest of your args

which means you can now do:

$ dzil run ./bin/myscript
$ dzil run prove -l t/unit.t
$ dzil run bash

the first one is specially useful if you're using file::sharedir and don't want to add extra hooks detecting whether it runs in a dev checkout or not.

speaking of sharedir, i crafted a quick'n'dirty implementation that made modulebuild plugin automatically install an existing share directory (which is possible since module::build 0.36). rjbs then re-factored it to use the installdirs plugin. nperez then finalized the work for the makemaker plugin. so if you have a share directory, it will now be automatically recognized as such by dzil and installed accordingly.

rjbs just released a new version of dist-zilla with those enhancements (and other stuff).

cpan task for dist-zilla

dist-zilla has a lot of prereqs. and even when you have installed it, you may need to install other plugins manually. therefore, i just uploaded task::dist::zilla that pulls in all dzil plugins and plugin bundles.

it still takes time to install all those modules, but at least you can do it in one go with cpan or cpanplus: no need to install them one by one manually.

of course, if you're on mandriva, running:

urpmi perl-Dist-Zilla

will install the package perl-Dist-Zilla and all its deps while also suggesting to install all the dzil plugins...

retrospective 2009

2009's over, but the year was quite a busy one for me on the perl front:

• i finally took the time to investigate moose, and decided to port some of my code to use it.
• i discovered dist::zilla, and definitely plan to use it in all my dists.
• i discovered some other cool new perl modules (thanks guys).
  
• i uploaded some new dists on cpan, including games::pandemic (which i'm quite proud of, even if i need to continue hacking on it).
• i updated & released some of my existing dists.
  
• i contributed to other projects such as padre, dist-zilla, etc. sending patches is a good way to thank the author for their work.
• it's even easier to contribute with github, which i'm using more and more for my projects (and it's no more slow by now, woohoo!). of course, git is the master piece allowing this easy sharing.
  
• on mandriva's front, i resurrected parrot rpm, and finally shipped rakudo.
  
• 2009 was also the year of the great migration to %perl_convert_version
• ... with lots of new perl modules available as rpm (i am managing 450 of them).
  
• ... and finally i took ownership of perl rpm. (thank you perl5 porters btw for 5.10.1 and the push for next stable version!)

so, busy indeed 2009 was... i just hope that 2010 will be as fruitful on the perl front for me, and for the perl ecosystem by large.

helping perl packagers package perl modules (for real this time)

chromatic posted a long rant (who would have guessed? :-) ) about perl modules shipped by linux distributions. however, he doesn't have all the answers... nor the experience needed for this rant. since i'm a perl author and mandriva packager for perl and lots of perl modules, i think i have more enlightened information about this topic.

first, let me state that using system perl is fine, but i really discourage it for your enterprise application. the perl version will change from time to time, ditto for the perl modules you are relying upon. so if you want to be in control your software foundation for your app (and you should) - compile your perl and your modules yourself.

second thing: i also discourage mixing using perl modules installed by your package system and by running cpan as root. you'll end up with a mix of files in /usr/perl5 that either belong or not to a system package, which sucks. installing packages in a local lib of yours is fine, which is made quite easy with local::lib by now.

if you're comfortable with those rules, then you're welcome to using the system perl and the modules shipped by your distribution. after all, we packagers are going through this work in the hope of being useful to others - that is, you!

but back to chromatic post. if you want to install cpan modules as system packages, there's already a tool that does it for you: it's called cpan2dist, and is part of cpanplus. it works as long as a backend for your distribution exists. there's currently one for debian, mandriva (that i wrote), fedora and gentoo. it's not that difficult to write, and allows you to write:

# cpan2dist --format CPANPLUS::Dist::Mdv --install Foo::Bar

this will automatically download foo::bar, check its dependencies (and build & install them if needed recursively), build the module as a mandriva package and install it. what else exactly do you want/need? (as a cpanplus backend writer, i do have some things that i'd like cpan2dist to have, but none as a regular user). and if you're a packager and want to integrate cpan2dist with your linux distribution build system, cpan2pkg can help you (even if it's currently stalled due to sthg missing in cpanplus).

however, if you want to help perl packagers package your modules for a distribution, here's a list of thing that you should not do. this is a list of real, practical things to do as a module author - not some generic hand-waving towards the perl community out there. this comes from my experience as packager for mandriva of more than 400 modules, and makes me curse the module author everytime i'm encountering one of those problems...

• test your dist before shipping. really, i'm not kidding. lots of dists just fail their tests. and not just on linux, on all the platforms. so if you make an update that "just can't fail" (yeah, right) to your dist just before shipping, please run your test suite nevertheless. just in case, you know, it might fail.

• if you're shipping pod tests that are skipped depending on the presence of test::pod and test::pod::coverage, make sure you have those modules installed, so you are running those tests, too. even better: skip those tests unless RELEASE_TESTING or AUTHOR_TESTING is set. after all, it's nice for you to know you still have some documentation work to do, but i don't care as a packager... and, you know, it's now the standard & recommended way of shipping those tests.

• those 2 items lead me to another easy thing for module authors to do to help us: check the cpantester status of your dist. investigate all the fails that you have. if you see a fail that is your fault, fix it and upload a new version. it helps us because this prevents us from having to report a bug against your dist. i generally wait 3 or 4 days before reporting a bug on a dist that has some failure reports, hoping (what a fool) that the author will notice by herself that sthg is going wrong.
  

• speaking of bug reports, if we take the time to open a report for your dist (very often with a patch attached)... please read it. and act. or at least answer us. either apply the patch, or explain why you don't want to apply it like that... and ship a new version of your dist, with the fix included.

• but of course, before reporting a bug, we should find the bug tracker. so, by using rt.cpan.org, you really help us to have a single unified point of contact. i know that rt is kind of slow, not very intuitive, has some problems and could be cleaned out a bit... but it is here, bestpractical is providing & administering it for us for free, and has this nice feature of having a queue for every perl dist on cpan. if you don't want to use it, there are some more polite ways of saying it... and giving the url of your tracker helps, too. oh, and if i took the pain to play by your rules and report a bug to your non-standard bug tracker, i would greatly appreciate that you act on my ticket. or at least, you know, just acknowledge the fact that you received the report.

• if you want to really piss off a packager, a simple but effective way is to change your versioning scheme every now and then, by (ab-)using your knowledge of perl way of understanding versions. in the same major version, of course. going from version 1.470 to 1.50 is not funny. if you want to change your versioning scheme, you can change the major number to. after all, i'm pretty sure that you're not paying any extra money per major number used in your dist. this is what caused us to mangle the version of perl modules shipped in mandriva.
  

• speaking of regular changes, it's irritating to have to follow you through your use of makefile.pl to build.pl to makefile.pl to build.pl to... well, you understand what i mean. even to use this shiny replacement that is module::build, or this oh-so-marvelous module::install, oh no finally module::build way of working fits me better in retrospective... it's ok for you to change from time to time, but changing at every version of your dist - just make up your mind dammit!

• speaking of it, i hate module::install. and especially its feature that prompts and tries to handle the deps itself coz-it's-so-cool-it-can-do-it-for-real. sorry, but that's not your job in the tool chain. just report that you miss some deps. i know that there's a flag to make this feature go away while launching makefile.pl. but i don't want to bother and would rather expect that the whole stuff has sane defaults...

• oh, and in case you're wondering - every prompting in the configure phase (makefile.pl or build.pl) sucks and should be banned.

• having clear and up-to-date dependencies would be fine, too. i know it's not always easy to have them correctly, but you can change your tools and adopt one that extract your prereqs for you.

• try to avoid dependency on modules that are known to fail. even if it works in your setup, trust cpantesters if they tell you that it fails 95% of its reports: it might not be a good idea to depend on it.

• trying to support old perl versions and old releases of modules is fine, but update your modules and see if your code work. some functions may become deprecated, or you were relying on a buggy behaviour, or whatever. we update the perl packages as we see new versions, not only your modules. so a linux dist will usually have latest & greatest version of all the modules - you'd better be sure that your code work with them, hmm?
  

• finally, if you're developping under macosx, make sure that you don't ship resource files, or textmate temp files. having ._Foo.pm in the dist is not fun: automatic compile tests will fail, unless that's your manicheck or signature check. and even if everything in perl dist is fine, things may bork in the repackaging of the system package due to a file not listed. so, be extra careful when shipping your dist - or change platform and burn your shiny toy that calls itself a computer (careful & clever readers may have guessed from previous sentence that i don't like macosx - but that's not a reason to ditch this post and not to follow the advices i'm reporting).
  

there, i think that's a pretty good start. i've encountered each and every item of this list at least once (i stopped counting exactly how much a long time ago). don't take it personally if you made some of those mistakes - i have done almost all of them by myself as module author (except of course using module::install and using a mac, but you could have guessed for at least the last part). what's important is to realize that those behaviours are annoying for packagers, and that changing those habits is quite easy to do (except for the burning your mac part, because i agree that it's not very environmental-friendly - see, i'm not that stubborn! ;-) ).

if you're following all those advices, packagers of your modules will love you. (or at least, not hate you - which is still a win :-) ). i know i will...

announcing git plugin for dist-zilla

pushing lazyness even further, i released a git plugin for dist-zilla. in fact, there are 4 plugins within the distribution. they are kicking in during dzil release, since i've hacked dzil to provide beforerelease and afterrelease roles.

dzp:git:check - this plugin checks that git is in a clean state before releasing. the following checks are performed before releasing:

• there should be no files in the index (staged copy)
• there should be no untracked files in the working copy
• the working copy should be clean. the changelog and dist.ini can be modified locally, though.

if those conditions are not met, the plugin will die, and the release will thus be aborted. this lets you fix the problems before continuing.

dzp:git:commit - once the release is done, this plugin will commit in git if needed changelog and dist.ini (which is a valid target if you happen to set the version manually in it). the commit message will be taken from the changelog for this release.

dzp:git:tag - once the release is done, this plugin will record this fact in git by creating a tag named v$VERSION.

dzp:git:push - once the release is done, this plugin will push current git branch to remote end, with the associated tags.

if you want to use all those neat plugins, you may want to use the git plugin bundle by writing in your dist.ini:

[@Git]

there, one more thing that dzil is doing for me... no more time lost doing those tedious steps - if you're lazy, you should really try it! as usual, comments and feature requests are welcome.

the state of perl in mandriva 2010

mandriva linux 2010 is out... this is thus a good time to have a look at the state of perl in this release.

adelie (mdv 2010 codename) ships perl 5.10.1 with threads^[0], which allowed me to drop half the patches carried within mandriva^[1]. and since perl 5 porters hope to make perl 5 releases more often, it means that (hopefully) i'll be able to trim down those patches even more!

on the modules front, mandriva is shipping plenty of them. padre 0.47 (and lots of plugins), moose, poe framework (and lots of components), devel modules, dbi drivers, gui (tk, wx, gtk) and various lib bindings, you name it... using book's pearl of wisdom, this gives us^[2]:

12117 installed modules
including 11016 from cpan (15.65% of cpan)
that is 1685 distributions (8.88% of cpan)

that's not half of cpan, but a good deal nevertheless. chances are that you will find the module you need - and if you don't, drop me a mail or open a request and i'll package it for you.

but the biggest change that you will see in mandriva 2010 regarding perl (well, maybe you won't even notice it), is the new standardized version scheme forced upon all perl modules. for example, dist-zilla version 1.092680 is shipped for mandriva 2010 in rpm perl-Dist-Zilla-1.92.680. yes, i know, it means that the mandriva version might differ from upstream version. yes, i am aware that this sucks. yes, i am also aware that there's no real way to get upstream version back from mandriva rpm version. and no, i don't plan to roll back this change - unless you have another miracle solution that deals with rpm not understanding cpan modules changing their versioning scheme without bumping their major (grrr!).

however, note that previous mandriva version (and fedora too) already mangled some modules version, but not all of them - which is even worse imo. since i agree with david that version numbers should be boring, all modules now have their version mangled. no ifs, no buts, we're mangling it. and it makes life easier for us packagers...

of course, the modules' code is not affected by this change. it's only the packaging that changed a bit. to check which version is installed, just check the $VERSION of this package. that's always been, and remains, the only safe way to be sure. eg:

$ perl -E 'for (@ARGV){eval "require $_"; say "$_\t",$_->VERSION if !$@}' Dist::Zilla
Dist::Zilla     1.092680

on the parrot front, we're shipping 1.6.0 (1.7.0 was too late to be included). and rakudo is still not present, since it cannot use an installed parrot.

all in all, that's a quite a good release on the perl front. if you add all the work that went in all other areas, mandriva 2010 will be quite a solid release. i invite you to test it out - who knows, you might even like it so much that you'll switch to it permanently! :-)

[0] no need to complain, rgs - i know your feelings about it.
[1] yup, i took the ownership of the perl package - silly me...
[2] why, yes, i install all perl modules available as rpm on my box :-)

right trimming for dzil autoprereq plugin

autoprereq plugin for dzil used to trim all modules under the dist namespace from the list of prereqs. eg, for dist-zilla dist, it would trim all dist::zilla::* modules.

although good for a first try, it had two problems:

• modules outside of your dist can be trimmed. eg, audio::mpd is requiring some audio::mpd::common modules which are outside of audio-mpd dist.
• if your dist is shipping 2 namespaces, only the modules inside the first namespace are trimmed, and you can end up with a dist with circular deps.
  

the first one is easy to solve with a fixed [Prereq] section in your dist.ini (this is what i've done for audio-mpd). and the second is not really common, so i thought that i could leave dzpa as is...

but dams now uses dzil for the excellent curses-toolkit - which ships curses::toolkit::* and poe::component::curses::* modules. so he complained (rightly) and i had to fix autoprereq plugin.

so, dzpa will only trim modules that are part of the dist, instead of blindly removing the modules under the dist namespace. and this only change fixes at once the 2 problems outlined above. where dzpa used to trim too much and not enough, it now trims just what it has to...

migration to moose - step 2

another module converted to moose (and dist-zilla): audio::mpd. migrating the attributes is quite easy, but this time the constructor migration was a bit trickier to do. fortunately, moose lazy building proved handy.

since the module is quite stable by now, i bumped version to 1.x (1.yyyddn even).

one more module with a simplified code-base... next migration will be pococm, and should prove more difficult, but for even more reward (thanks to moosex::poe).

dist-zilla: compile tests can skip some modules

the compilation test plugin for dist-zilla now accepts a parameter to skip some modules. this can be handy to prevent testing some known-to-fail modules - yes, this can happen if your module does stuff assuming a given environment not currently in place (eg: test modules...).

just use the following in your dist.ini file:

[CompileTests]
skip = Test$

the skip parameter is interpreted as a regex, matched against the module name (not the file name of the module). in this example, all modules with a name ending with Test will not be tested for compilation.