the great renaming: Dist::Zilla::Plugin::CriticTests

oops! kentnl did it again. :-)

this time, dist::zilla::plugin::critictests has been renamed to dist::zilla::plugin::test::perl::critic. once again, change will be trivial for authors [0]. previous module will continue working, for at least one year counting from today.

github repository has also changed and is available here: https://github.com/jquelin/dist-zilla-plugin-test-perl-critic

[0] and this time, i did not fubar-ed the code! :-|

the great renaming: Dist::Zilla::Plugin::CompileTests

in a move to rationalize the plugin namespace of dist::zilla, kentnl started a  great renaming: dzp:compiletests has been renamed to dzp:test::compile

previous module is deprecated, and may be removed later on (but not before one year, that is 2012-08-27). in the meantime, it will continue working (although with a warning).

what does it mean for dzil users? well, nothing much, since old module will continue working as is. however, they should better migrate their dist.ini from:

[CompileTests]

to:

[Test::Compile]

nothing else is needed... for now, since i guess kentnl will continue proposing patches / pullreqs to dzil:*tests plugin authors! ;-)

also, please note that the github repository has been renamed and is now available at a different url. ditto for cpan, since the dist is now named dist-zilla-plugin-test-compile.

cpan2pkg: prettier gui, ready for mageia!

remember cpan2pkg? this is the tool built around cpan2dist to create a native linux package, taking dependencies into account, and integrated with linux distribution repository + buildsystem.

it used to be mandriva only, and curses-based. now that mageia is in full swing mode, i had to port cpan2pkg to support it. and it was a perfect opportunity to clean up the module, and prettify the interface.

here's the result:

on the left, you can see the modules currently being processed, with their status both locally and on the build-system. green means available, and here you can see that moosex::alwayscoerece is being built locally (orange) and is not yet available on mageia (yellow).

note that mageia is fully supported: cpan2pkg will poll the buildsystem status page to follow a module build, and make sure a module is ready before submitting those depending on it.

it is still using poe underneath, but this time with tk. adding support for a platform can be done quite easily - patches welcome!

"magpie fix" also tigthens spec files

following cpanplus backend cleaning, "magpie fixspec" also gained the ability to clean up spec files with trivial %clean section and %defattr definition.

this means that updating a perl package on mageia (with either "magpie up" or "magpie dwim") will also trim the rpm spec file accordingly. neat!

more tightened spec file produced by mageia cpanplus backend

cpanplus::dist::mageia 1.111720 has just been released, with 2 changes:

• no more %defattr definition
• no more default %clean section

indeed, rpm will handle those automatically, as dexter mentioned (cf here and here). this way, we'll have spec files a bit more tightened, which is always better from a maintainer pov.

supply your own perlcriticrc to dzil test plugin

if you wanted the ability to supply your own perlcriticrc while using dist-zilla-plugin-critictests, then stephen scaffidi is your hero of the day. indeed, he just implemented this - and i had to do the tedious work of typing "dzil release". oh well, i can say that life is difficult. :-)

announcing file::sharedir::pathclass

following last week's announce of file::homedir::pathclas, i also committed file::sharedir::pathclass on the same principle. therefore, if you always wrap file::sharedir results in path::class objects for greater convenience, just use file::sharedir::pathclass instead, which is doing that automatically for you.

random file-homedir bits

no need to present file::homedir, the module to use to retrieve various information such as where is my home, where is my desktop, where are my documents, etc.

since my needs were not all covered, i contacted alias with some of my ideas. he kindly gave me a commit bit so i can scratch my itches at will...

i therefore implemented xdg support for recent unix desktops, with daxim's help. this means that unix platforms with gnome, kde or any recent desktops won't report $HOME as the one and unique answer to all those questions. this feature is available in released file-homedir 0.91 - towards a cleaning of our homedirs, yay!

still in the work (not yet released), i've also checked in a my_dist_data($dist) function, to standardize the directory where the application will store its internal data, such as database or cache. it is located in:

my_data()/Perl/dist/$dist

following the now traditional data/vendor/application scheme (on legacy unix desktops, the directory will be $HOME/.perl/dist/$dist/var, to be sure that it's a hidden directory). to be consistent with file::sharedir, i guess i'll also implement my_module_data($module), following the same reasoning.

this will likely be released in version 0.92, before working on the last remaining bit on my plate: my_dist_config($dist), returning a directory where an application will be able to store its configuration. it's a bit different of the data directory (even if config is some kind of data): data is supposed to be transient, or can be removed without harming the app - while the config should not be erased. however, it's not really as straightforward as the data directory, since not all platforms support this: on windows, users are not supposed to update config by hand, so it's often stored in the registry... so how to preserve the cross-distribution nature of file-homedir for this very feature? this will require some thinking...

finally, it's with great pleasure that i'm announcing file-homedir-pathclass, which is a convenient wrapper around file-homedir returning path::class objects. alias did not want to introduce it in file-homedir to preserve compatibility, and suggested to release it as a new dist... so i released this new module, allowing to write for example:

perl -MFile::HomeDir::PathClass=-all -E 'say $_ for my_home()->children'

roll your own dzil tutorial

rjbs created a roll-your-own dzil tutorial. this brings back memories, maybe there should be monsters in it. otoh, since it mentions module::install, maybe the monster quota is already met. :-)

anyway, the tutorial is a nice read for those wanting to learn dist-zilla. there are some glitches here and there (eg, it still mentions AllFiles plugin instead of GatherDir, and other v1 to v2 misses - i'll send patches), but is otherwise wicked cool.

and some of my plugins are even mentioned! :-)

github power

i started using github somewhere in 2009 iirc. i was a bit reluctant to use it back at that time, especially because of its speed (or lack of, actually)... and because i already hosted my projects of repo.or.cz.

however, i'm now really glad i started using it. speed is no longer a problem, and it provides really nice features. cia integration, network graphs, comments on commits, watching repos (even if i'm not totally happy with it right now), and of course... pull requests!

i was away this week-end, and found 4 pull requests in my mailbox for my modules when i came back. integrating other's work has never been easier with git and github which allows people to fork at will. that's the new way of writing code: create the basic stub of your module, and wait for others to enhance it! :-) git plugin for dist-zilla now supports pushing to a different branch, supports the new dist-zilla-tester framework and the bundle @git now accepts multi-valued args.

dvcs are a bit harder to grasp at first, but once you understand the concepts, they're really a killer application - especially when coupled with a cooperative platform such as github.

dzil activity

of course, you are aware that dzil v2 is out, with lots of exciting new features, brought to you by rjbs and the perl foundation.

but it's only one side of the activity floating around dist-zilla... indeed, as dagolden, aevar and others try dzil, they (of course) like it, and start contributing their own plugins.

here's a list of new plugins landed on cpan... first some additional author tests:

• CheckChangesTests - check changelog
  
• CheckExtraTests - runs xt/ tests, but don't copy them to t/
  
• DistManifestTests - check manifest
  
• HasVersionTests - check all your modules have a version
  
• KwaliteeTests - check your kwalitee
  
• MinimumVersionTests - check minimum perl version needed
  
• PodSpellingTests - check pod spelling
  
• PortabilityTests - check if your code is portable
  
• SynopsisTests - check if code in your synopsis compiles
  
• UnusedVarsTests - detects unused vars
  
• ReportVersions - additional test to report version
  

some plugins to complete meta-data:

• Bugtracker - http://rt.cpan.org/Public/Dist/Display.html?Name=xxxx
• HomePage - http://search.cpan.org/dist/xxxx
  

some plugins to customize your build process:

• ModuleBuild::Custom - for mb
• OverridableMakeMaker - for eumm

some plugins to generate additional files:

• FatPacker - creates a script with all dependencies packed
• InstallGuide - creates an INSTALL file
• ReadmeMarkdownFromPod - README.mkdn file

some plugins to compute your next version number:

• BumpVersionFromGit - version taken from last git tag
• VersionFromPrev - classic perl versions, 1.00 to 1.99

and finally some bundles:

• @AVAR - ævar arnfjörð bjarmason's take on it
  
• @MARCEL - marcel grünauer
• @PDONELAN - patrick donelan

you can install all of them in one go via task-dist-zilla.

they are also available in mandriva, and suggested when installing dist-zilla...

how to obsolete a cpan dist?

my autoprereq plugin for dist-zilla is now going core. however, it's currently existing as a dist of its own within cpan... since the module does not change name, what is needed to make sure rjbs can upload dzil v2 when it's ready? is giving co-maint rights on pause enough? will i have to remove previous tarballs?

the same kind of question can be asked for obsoleting a given module. there's (afaik) no way to tell cpan / cpanplus that foo::bar is replacing bar::foo... those problems are tackled for linux distribution packages, could we reuse some of their logic here?

activity in git plugin for dist-zilla

it's been a busy week for dist-zilla-plugin-git...

first, there was some test fixes to work with latest dist-zilla (thanks to ricardo), and also to support git 1.7.

the git plugin then saw support for annotated tags which is now the default - but christopher madsen restored the possibility to use lightweight tags. he then refactored commit message generation, in order to allow a custom commit message. not only that, but he also brought to ability to specify which files are allowed to be dirty, and automatically committed.

then david golden jumped in, and scratched his itches: support for empty lines in changelog and possibility to specify which branch(es!) to push to. all of that with new tests - and some existing test fixes! \o/

git support in dist-zilla is now in a pretty good shape. report bugs if you miss your pet-peeve... or even better: fork the code and send me pull requests once the code is doing what you want!

which module to extract perl prereqs?

in dzil plugin autoprereq, i'm extracting prereqs from the dist modules. i want this extract to be fast, based on the actual code (not makefile.pl or meta.yml, since the goal is to generate them), and as accurate as possible. it should also find base classes, moose roles and other "hidden" dependencies. finally, it should extract the minimum version needed for a given module, including minimum perl version.

my first version was regex-based. i can already see your horrified face - but really it wasn't so bad, since it only needed to find some specific statements such as uses and requires. current version is using ppi, which is better suited for corner cases.

however, long-term makes me think that it would be better to rely on an external module. so, what are the alternatives out there on cpan, and can i use them in autoprereq?

• b::perlreq - parses the file, but reports file (File/Basename.pm) instead of modules, and is generally more suited for rpm
  
• module::extract::use - using ppi to parse a file, but extracts only use & require statements (no inheritance, moose roles, etc). also, it reports no minimum version extraction, only a list of modules.
  
• test::dependencies - using either b::perlreq (see above) or a regex scheme underneath
  
• module::scandeps - runs the file (which is slow), and finds all modules included - and sometimes a bit more (eg: file::homeDir::darwin is found for a module using file::homedir, even on a unix platform). can also run as a static analyser, but calls cpanplus (?!) which is slow.
  
• module::info - regex based
  
• module::cpants::generator::prereq - parses makefile.pl, where i want sthg that parses actual code
  
• module::cpants::kwalitee::prereq - parse meta.yml, makefile.pl or build.pl

so, no module was doing exactly what i wanted... since i am using ppi and that module::extract::use does the same, i contacted brian d foy to see whether he would be interested in additional extractions (moose roles, base classes, etc.) for this module. he was, so those enhancements are now pushed on my github clone.

i'm now waiting for a new release of this module with my enhancements, meaning that i can get rid of this part of the code in dzil autoprereq. which was, if you recall, the original goal! :-)

stats about cpan modules shipped by mandriva

adam and gabor asked me about some information on what cpan modules are shipped as rpm packages by mandriva... i finally found some time to work on it and produced ordb::cpan::mandriva.

this module is basically a sqlite database replicated automatically by orlite::mirror. using it is very simple - for example, couting the cpan dists shipped by mandriva is achieved by the following snippet:

(fyi, the result is 1899 as of writing, +1 since i just imported the module itself)

at module use time, it will automatically download (and cache for a week) the sqlite database. and one can then use the module as an ordb for this db... the module does not however produces the stats for you - so alias & gabor, now is your turn to work! :-) i'm waiting for your top-100 most wanted mandriva or whatever crazy stats you'll want to produce.

the database itself is generated by module::packaged::generator. this module uses different drivers depending on the current linux distribution (but nothing prevents *bsd to also have a driver) it runs on. only mandriva is supported currently, but ryan will work on a debian driver. all other contributions are most welcome - the code is on github: fork it and send me pull requests!

next padre version will (finally) recognize dist-zilla projects

padre 0.55, due tomorrow, will finally recognize dist-zilla projects correctly. among other things, it means that it will set @INC accordingly for syntax checking, keep the directory browser at the project root, etc.

it took a bit of time since the "installer" detection is spread out in different places in padre... this needs refactoring, for those interested.

some dzil goodness coming

i scratched some itches in dist-zilla, and hopefully the result will be useful to you too... some are rather trivial, while some are more feature-ful.

on the it-s-the-details-that-count front, dzil clean will now remove *~ files lingering in your local copy.

dzil command now accepts a -I argument that adds a directory to perl @INC (same as perl's -I option). which means dzil plugin author can now do:

$ dzil -Ilib release

and release their distribution using the latest version of their plugin...

a new run subcommand has been added, which is doing more or less the following:

$ dzil build
$ rsync -avp My-Project-version/ .build/
$ cd .build
$ perl Makefile.PL     # or perl Build.PL
$ make                 # or ./Build     
$ export PERL5LIB=$PWD/blib/lib:$PWD/blib/arch
$ launch command defined by rest of your args

which means you can now do:

$ dzil run ./bin/myscript
$ dzil run prove -l t/unit.t
$ dzil run bash

the first one is specially useful if you're using file::sharedir and don't want to add extra hooks detecting whether it runs in a dev checkout or not.

speaking of sharedir, i crafted a quick'n'dirty implementation that made modulebuild plugin automatically install an existing share directory (which is possible since module::build 0.36). rjbs then re-factored it to use the installdirs plugin. nperez then finalized the work for the makemaker plugin. so if you have a share directory, it will now be automatically recognized as such by dzil and installed accordingly.

rjbs just released a new version of dist-zilla with those enhancements (and other stuff).

cpan task for dist-zilla

dist-zilla has a lot of prereqs. and even when you have installed it, you may need to install other plugins manually. therefore, i just uploaded task::dist::zilla that pulls in all dzil plugins and plugin bundles.

it still takes time to install all those modules, but at least you can do it in one go with cpan or cpanplus: no need to install them one by one manually.

of course, if you're on mandriva, running:

urpmi perl-Dist-Zilla

will install the package perl-Dist-Zilla and all its deps while also suggesting to install all the dzil plugins...

retrospective 2009

2009's over, but the year was quite a busy one for me on the perl front:

• i finally took the time to investigate moose, and decided to port some of my code to use it.
• i discovered dist::zilla, and definitely plan to use it in all my dists.
• i discovered some other cool new perl modules (thanks guys).
  
• i uploaded some new dists on cpan, including games::pandemic (which i'm quite proud of, even if i need to continue hacking on it).
• i updated & released some of my existing dists.
  
• i contributed to other projects such as padre, dist-zilla, etc. sending patches is a good way to thank the author for their work.
• it's even easier to contribute with github, which i'm using more and more for my projects (and it's no more slow by now, woohoo!). of course, git is the master piece allowing this easy sharing.
  
• on mandriva's front, i resurrected parrot rpm, and finally shipped rakudo.
  
• 2009 was also the year of the great migration to %perl_convert_version
• ... with lots of new perl modules available as rpm (i am managing 450 of them).
  
• ... and finally i took ownership of perl rpm. (thank you perl5 porters btw for 5.10.1 and the push for next stable version!)

so, busy indeed 2009 was... i just hope that 2010 will be as fruitful on the perl front for me, and for the perl ecosystem by large.

helping perl packagers package perl modules (for real this time)

chromatic posted a long rant (who would have guessed? :-) ) about perl modules shipped by linux distributions. however, he doesn't have all the answers... nor the experience needed for this rant. since i'm a perl author and mandriva packager for perl and lots of perl modules, i think i have more enlightened information about this topic.

first, let me state that using system perl is fine, but i really discourage it for your enterprise application. the perl version will change from time to time, ditto for the perl modules you are relying upon. so if you want to be in control your software foundation for your app (and you should) - compile your perl and your modules yourself.

second thing: i also discourage mixing using perl modules installed by your package system and by running cpan as root. you'll end up with a mix of files in /usr/perl5 that either belong or not to a system package, which sucks. installing packages in a local lib of yours is fine, which is made quite easy with local::lib by now.

if you're comfortable with those rules, then you're welcome to using the system perl and the modules shipped by your distribution. after all, we packagers are going through this work in the hope of being useful to others - that is, you!

but back to chromatic post. if you want to install cpan modules as system packages, there's already a tool that does it for you: it's called cpan2dist, and is part of cpanplus. it works as long as a backend for your distribution exists. there's currently one for debian, mandriva (that i wrote), fedora and gentoo. it's not that difficult to write, and allows you to write:

# cpan2dist --format CPANPLUS::Dist::Mdv --install Foo::Bar

this will automatically download foo::bar, check its dependencies (and build & install them if needed recursively), build the module as a mandriva package and install it. what else exactly do you want/need? (as a cpanplus backend writer, i do have some things that i'd like cpan2dist to have, but none as a regular user). and if you're a packager and want to integrate cpan2dist with your linux distribution build system, cpan2pkg can help you (even if it's currently stalled due to sthg missing in cpanplus).

however, if you want to help perl packagers package your modules for a distribution, here's a list of thing that you should not do. this is a list of real, practical things to do as a module author - not some generic hand-waving towards the perl community out there. this comes from my experience as packager for mandriva of more than 400 modules, and makes me curse the module author everytime i'm encountering one of those problems...

• test your dist before shipping. really, i'm not kidding. lots of dists just fail their tests. and not just on linux, on all the platforms. so if you make an update that "just can't fail" (yeah, right) to your dist just before shipping, please run your test suite nevertheless. just in case, you know, it might fail.

• if you're shipping pod tests that are skipped depending on the presence of test::pod and test::pod::coverage, make sure you have those modules installed, so you are running those tests, too. even better: skip those tests unless RELEASE_TESTING or AUTHOR_TESTING is set. after all, it's nice for you to know you still have some documentation work to do, but i don't care as a packager... and, you know, it's now the standard & recommended way of shipping those tests.

• those 2 items lead me to another easy thing for module authors to do to help us: check the cpantester status of your dist. investigate all the fails that you have. if you see a fail that is your fault, fix it and upload a new version. it helps us because this prevents us from having to report a bug against your dist. i generally wait 3 or 4 days before reporting a bug on a dist that has some failure reports, hoping (what a fool) that the author will notice by herself that sthg is going wrong.
  

• speaking of bug reports, if we take the time to open a report for your dist (very often with a patch attached)... please read it. and act. or at least answer us. either apply the patch, or explain why you don't want to apply it like that... and ship a new version of your dist, with the fix included.

• but of course, before reporting a bug, we should find the bug tracker. so, by using rt.cpan.org, you really help us to have a single unified point of contact. i know that rt is kind of slow, not very intuitive, has some problems and could be cleaned out a bit... but it is here, bestpractical is providing & administering it for us for free, and has this nice feature of having a queue for every perl dist on cpan. if you don't want to use it, there are some more polite ways of saying it... and giving the url of your tracker helps, too. oh, and if i took the pain to play by your rules and report a bug to your non-standard bug tracker, i would greatly appreciate that you act on my ticket. or at least, you know, just acknowledge the fact that you received the report.

• if you want to really piss off a packager, a simple but effective way is to change your versioning scheme every now and then, by (ab-)using your knowledge of perl way of understanding versions. in the same major version, of course. going from version 1.470 to 1.50 is not funny. if you want to change your versioning scheme, you can change the major number to. after all, i'm pretty sure that you're not paying any extra money per major number used in your dist. this is what caused us to mangle the version of perl modules shipped in mandriva.
  

• speaking of regular changes, it's irritating to have to follow you through your use of makefile.pl to build.pl to makefile.pl to build.pl to... well, you understand what i mean. even to use this shiny replacement that is module::build, or this oh-so-marvelous module::install, oh no finally module::build way of working fits me better in retrospective... it's ok for you to change from time to time, but changing at every version of your dist - just make up your mind dammit!

• speaking of it, i hate module::install. and especially its feature that prompts and tries to handle the deps itself coz-it's-so-cool-it-can-do-it-for-real. sorry, but that's not your job in the tool chain. just report that you miss some deps. i know that there's a flag to make this feature go away while launching makefile.pl. but i don't want to bother and would rather expect that the whole stuff has sane defaults...

• oh, and in case you're wondering - every prompting in the configure phase (makefile.pl or build.pl) sucks and should be banned.

• having clear and up-to-date dependencies would be fine, too. i know it's not always easy to have them correctly, but you can change your tools and adopt one that extract your prereqs for you.

• try to avoid dependency on modules that are known to fail. even if it works in your setup, trust cpantesters if they tell you that it fails 95% of its reports: it might not be a good idea to depend on it.

• trying to support old perl versions and old releases of modules is fine, but update your modules and see if your code work. some functions may become deprecated, or you were relying on a buggy behaviour, or whatever. we update the perl packages as we see new versions, not only your modules. so a linux dist will usually have latest & greatest version of all the modules - you'd better be sure that your code work with them, hmm?
  

• finally, if you're developping under macosx, make sure that you don't ship resource files, or textmate temp files. having ._Foo.pm in the dist is not fun: automatic compile tests will fail, unless that's your manicheck or signature check. and even if everything in perl dist is fine, things may bork in the repackaging of the system package due to a file not listed. so, be extra careful when shipping your dist - or change platform and burn your shiny toy that calls itself a computer (careful & clever readers may have guessed from previous sentence that i don't like macosx - but that's not a reason to ditch this post and not to follow the advices i'm reporting).
  

there, i think that's a pretty good start. i've encountered each and every item of this list at least once (i stopped counting exactly how much a long time ago). don't take it personally if you made some of those mistakes - i have done almost all of them by myself as module author (except of course using module::install and using a mac, but you could have guessed for at least the last part). what's important is to realize that those behaviours are annoying for packagers, and that changing those habits is quite easy to do (except for the burning your mac part, because i agree that it's not very environmental-friendly - see, i'm not that stubborn! ;-) ).

if you're following all those advices, packagers of your modules will love you. (or at least, not hate you - which is still a win :-) ). i know i will...