cpan2pkg now available in text mode!

cpan2pkg is the tool used to automatically package a module from cpan to mageia as a rpm, along with all its dependencies. it's built on shoulders of giants (cpan2dist, mgarepo, poe, etc)

it used to feature a tk window with one tab per module being investigated - but this was troublesome for people wanting to have a cli. so i just released a new version of cpan2pkg which brings a --text flag (or -T for the lazy ones) which works on the tty. of course, we loose the ability to interact with the application, but that's still convenient sometimes. and of course, the tk version is still available!

more than 3000 perl packages in mageia!

long time no post... but now that mageia perl stats are up & running (daily update), i can as well present the site and brag about the new record we achieved: mageia is now shipping with more than 3000 perl packages, for a total of 26500+ real perl modules. not that bad, uh? :-)

cpan2pkg: prettier gui, ready for mageia!

remember cpan2pkg? this is the tool built around cpan2dist to create a native linux package, taking dependencies into account, and integrated with linux distribution repository + buildsystem.

it used to be mandriva only, and curses-based. now that mageia is in full swing mode, i had to port cpan2pkg to support it. and it was a perfect opportunity to clean up the module, and prettify the interface.

here's the result:

on the left, you can see the modules currently being processed, with their status both locally and on the build-system. green means available, and here you can see that moosex::alwayscoerece is being built locally (orange) and is not yet available on mageia (yellow).

note that mageia is fully supported: cpan2pkg will poll the buildsystem status page to follow a module build, and make sure a module is ready before submitting those depending on it.

it is still using poe underneath, but this time with tk. adding support for a platform can be done quite easily - patches welcome!

"magpie fix" also tigthens spec files

following cpanplus backend cleaning, "magpie fixspec" also gained the ability to clean up spec files with trivial %clean section and %defattr definition.

this means that updating a perl package on mageia (with either "magpie up" or "magpie dwim") will also trim the rpm spec file accordingly. neat!

more tightened spec file produced by mageia cpanplus backend

cpanplus::dist::mageia 1.111720 has just been released, with 2 changes:

• no more %defattr definition
• no more default %clean section

indeed, rpm will handle those automatically, as dexter mentioned (cf here and here). this way, we'll have spec files a bit more tightened, which is always better from a maintainer pov.

a bit of dwimery in magpie

all the pieces were available, binding them together was just a smop. therefore magpie just got a new subcommand: "magpie dwim" which does exactly what i mean...

in the case of magpie, what i mean is of course:

• check which perl packages are not up to date wrt cpan
• check them out
• tighten spec file
• update the package to latest version
• commit result
• wait for build system if needed
• submit result

and all of this is done in parallel, with errors reported at the end.
you too can have some fun:

$ sudo urpmi magpie
$ magpie dwim

maintaining the ~2500 perl module rpm packages in mageia has never been so easy!

magpie: how to list non up-to-date perl modules

the final stone is now built for magpie: it's possible to list on your mageia system the perl modules that have up to date versions upstream. it allows to see at a glance which perl packages need to be updated. it's based on cpanplus, and only tackles perl modules installed on your system.

the new command is:

$ magpie old
** core packages: 11

Devel::DProf                             20080331.00 20110228.00
Devel::SelfStubber                              1.03        1.05
[...]
if                                              0.05      0.0601

** normal packages: 57

DateTime::TimeZone                              1.28        1.29     perl-DateTime-TimeZone                                 1.280.0
Digest::SHA                                     5.50        5.60     perl-Digest-SHA                                        5.500.0
[...]
parent                                         0.224       0.225     perl-parent                                            0.224.0

** orphan packages: 7

KENTNL                                      0.010173Dist::Zilla…
[...]
inc::latest                                   0.3624        0.38

** strange packages: 3

App::cpanminus                                1.2001      1.4002     cpanminus(1.200.100),perl-App-cpanminus(1.200.100)
HTTP::Response                                  6.00        6.01     perl-libwww-perl(5.837.0),perl-Test-Mock-LWP(0.50.0),perl-HTTP-Message(6.0.0)
Perl::MinimumVersion                            1.27        1.28     perl-Perl-MinimumVersion(1.270.0),perl-Perl-Critic-Pulp(47.0.0)

** ignored modules: 12

Boulder::Unigene(28051999.00), File::MimeInfo::Rox(0.20), Getopt::Lucid(0.19), HTML::Table(2.08), Imager::Font::Type1(1.011), Inline::Python(0.38), Lingua::Features(0.30), Parse::RecDescent::FAQ::Original(6.00), Switch(2.16), Tie::Watch(1.301), WebFetch(0.13), XML::Grove(0.46)

the columns are: module, old version, new version, mageia package name, mageia package version.

as you can see, the modules are sorted in various categories:

• core - modules shipped with perl (or perl-base)
• dual-lifed (not shown above) - modules that are both shipped with perl (or perl-base) and as a stand-alone package
• normal - regular modules with their own package
• orphan - modules that do not belong to a mageia package (either inherited from mandriva, or not yet pushed on buildsystem).
• strange - modules that belong to more than one magiea package
• ignore - those are modules that either cannot be built (with an upstream bug) or that confuse cpanplus when comparing versions

with this information, it's therefore very straightforward to update the packages, launching the following command as needed:

$ eval $( magpie co -s $pkg ) && magpie update

of course, since all the building blocks are now available, that's what the last command to be implemented (magpie dwim) will do!

svk cannot be built anymore - EDIT: svk will build with reduced features

while rebuilding perl packages for mageia, i found out that svn::mirror cannot be built anymore with subversion 1.6.x. said like this, one may think "wtf, i don't give a damn to svn::mirror". and lots of other cpan modules are bit-rotting, so what's the point? well, the thing is, svn::mirror is a prereq for svk, the decentralized subversion...

granted, svk has been end-of-lifed in 2009, yet it's still widely in use. but as long as svn-mirror won't get fixed, mageia won't include svk support.

EDIT (20110301): in fact, svn::mirror is an optional prereq for svk - mageia will therefore ship with svk, albeit with a reduced feature set

magpie update now waaaaaaay faster!

"magpie update", used to update automatically a perl module rpm to its latest version, was a bit slow. the culprit was parse::cpan::packages, taking a whole 10 seconds to parse 02packages.details.txt.gz

fortunately, i found parse::cpan::packages::fast (from slaven++), which does exactly the same job in less than a second...

so, with a 16-line patch (-2/+2), magpie update now is almost instant. cpan is definitely the home of nice gems, and *the* advantage of perl.

new magpie command: update

i'm happy to report yet another magpie release, version 1.110410

this release brings a new subcommand: update (aka refresh)

it will automatically update a perl module package being checked out to the latest version, update buildrequires (with fixspec), try to build it locally, commit if successful, wait according to build-system hints, then submit it.

it's easy to use:

$ eval $( magpie co -s perl-Foo-Bar )
$ magpie update -v

note: it requires a minicpan installation on your computer...

now, you too can join the fun of updating perl modules packages for mageia! but it's not yet over, automation will go one step further - stay tuned...

new magpie command: fixspec

i just released magpie 1.110390, which brings a new subcommand: magpie fixspec

as you can guess, this command will try to sanitize a rpm spec file a bit. to use it, you must be in a package checkout, and run it without any argument.

$ eval $( magpie co perl-Foo-Bar )
$ magpie fixspec -v

# to see the list of options
$ magpie help fix

among the things that fixspec does:

• it updates %doc depending on the existing files, including meta files
• it splits multiple bundled build-/requires to have only one per line
• it lines up vertically the summary / etc
• it extracts perl buildrequires from meta.json/meta.yml if present
• it removes buildroot definition
• it removes mdv macros

note that the code is not really clean, it might gain from being in its
own module, with real rpm parsing instead of big regexes. also, it
assumes that we're cleaning a perl module spec file, so you're on your
own if you're using it on other spec file: it might remove the whole
svn, eat your babies, or even convert your rpm database to rpm5. you've
been warned. :-)

magpie - MAGeia Perl Integration Easy

in my mandriva to mageia switch, i promised to myself that i'd collect the various scripts that i'm using on a day to day basis to maintain the myriad of perl rpm packages.

therefore, i've created magpie (MAGeia Perl Integration Easy) and uploaded it to cpan. the tool is not yet complete, but i'm incorporating new commands now and then.

it currently supports 2 commands:

• magpie bswait - this command pauses according to the recommendation of mageia build-system. it indeed provides some recommendation on how much time to pause between 2 packages submission to not overload it - (this is known as throttling).
• magpie co - this brand new command (in version 1.110320) allows to check-out a given package from mageia repository.

some details about magpie checkout: it is not meant to replace "mgarepo co", it does in fact use mgarepo underneath. so what does this command add?

• the possibility to check out in a given directory
  

  $ magpie co -d ~/rpm/cauldron perl

• the fact that it will either check out *or* update the local check-out if it already exists:
  

  $ magpie co -d ~/rpm/cauldron perl
  # check-out if 1st run, update otherwise

• finally, the possibility to dump a shell command to execute to change directory in the fresh check-out:
  

  # add this in your ~/.bashrc
  function cco() { eval $(magpie co -d ~/rpm/cauldron -q -s $*); }
  # then, one can do a fresh checkout/update + cd by issuing:
  $ cco perl

nothing revolutionary, but it saves some keystrokes here and there.

mageia warming up - rebuilding perl packages

it's been a long time without posting - but real life kicked in.

one of the things that keep me busy those days is the rebuilding of perl packages for mageia. around 2000 have been done, with ~400 still waiting. it's been a good opportunity to clean up the mess: remove old packages, clean spec files, etc.

all in all, mageia will be quite in a good shape regarding perl packages when mirroring will start - which should happen quite soon!

good-bye mandriva, hello mageia!

word is spreading fast: mandriva developers & contributors are forking mandriva. the new distribution is named mageia - read the full announce on http://mageia.org. my name being on the announce, no need to say that i fully support this move... :-) more news will be available in the coming days.