cpan2pkg now available in text mode!

cpan2pkg is the tool used to automatically package a module from cpan to mageia as a rpm, along with all its dependencies. it's built on shoulders of giants (cpan2dist, mgarepo, poe, etc)

it used to feature a tk window with one tab per module being investigated - but this was troublesome for people wanting to have a cli. so i just released a new version of cpan2pkg which brings a --text flag (or -T for the lazy ones) which works on the tty. of course, we loose the ability to interact with the application, but that's still convenient sometimes. and of course, the tk version is still available!

cpan2pkg: prettier gui, ready for mageia!

remember cpan2pkg? this is the tool built around cpan2dist to create a native linux package, taking dependencies into account, and integrated with linux distribution repository + buildsystem.

it used to be mandriva only, and curses-based. now that mageia is in full swing mode, i had to port cpan2pkg to support it. and it was a perfect opportunity to clean up the module, and prettify the interface.

here's the result:

on the left, you can see the modules currently being processed, with their status both locally and on the build-system. green means available, and here you can see that moosex::alwayscoerece is being built locally (orange) and is not yet available on mageia (yellow).

note that mageia is fully supported: cpan2pkg will poll the buildsystem status page to follow a module build, and make sure a module is ready before submitting those depending on it.

it is still using poe underneath, but this time with tk. adding support for a platform can be done quite easily - patches welcome!

cpan2dist oddity

cpanplus is great - really, i mean it. i don't like its internals, and having to adapt to it for cpanplus::dist::mdv is not my fondest memory. however, it gets the job done, and that's what counts in the end.

but cpanplus has some oddities in the user-space also. cpan2dist, which is a tool allowing you to create a native package for your platform, sometimes makes me cry.

it insists on building prereqs for the module you want. that's fine, except for one thing: it will also build packages for modules that are not up to date... and this is where the fun starts.

you can think that --nobuildprereqs would prevent this, but nope, it's the default and really means: "don't rebuild prereqs if we're at latest version, but rebuild anyway if we're not at latest version". the --buildprereqs indeed means "i really really want to loose my time, please rebuild a package for all the prereqs even if i already have them installed and working". like this option is of any use... erm.

so in app::cpan2pkg, in order not to build the prereqs, i ended up using --ignore flags (flags as in plural form, since using a negative look-ahead regex such as /(?<!$name)$/ do not work). it took me quite some time to come up with this solution, since --ban was not working at that time.

so life was good, i've used cpan2pkg quite some time to create rpms for mandriva, and updated it till v1.0.0. however, since some time, i can see strange things in the packages submitted on upstream buildsystem: the build prereqs were not set. and i had to update the spec file manually, adding all those missing prereqs. i got tired of that, and investigated what happened.

after some experiments, it appears that cpan2dist seems to have somehow changed the way it handles its --ignore flags. ignored modules now are removed from the prereqs completely, and thus the cpanplus::dist backend do not even see them any more. and thus, buildrequires end up empty. of course, --nobuildprereqs still retained its old silly behaviour...

fortunately, it seems that --ban now works correctly (at least on 2 tests that i made). so i've just updated cpan2pkg, and version 1.1.0 should be available soon on cpan, this time really adding the needed buildrequires.

till the next cpan2dist change, of course... :-|

note: i don't want to sound too harsh to jos in this post... i totally understand that cpanplus is still 0.x software - moreover, this is a bug fix in my mind, since --ban was not working.

cpan2pkg reaches maturity

i finally took some time to work on app::cpan2pkg - and i'm happy to report that it now chains upstream builds, which means that all basic features are in.

therefore, i just released app::cpan2pkg 1.0.0 on cpan. detailed changelog:

• updating prereqs + moving on when module installed from upstream
• splitted module in poe session + worker
• moved info from app:cpan2pkg to module attributes
• don't submit on upstream build-system if missing prereqs on it
• cpan2pkg documentation written

yes, cpan2pkg can build, install, import and automatically build packages from cpan for linux, while following dependencies. does it work? sure, i already used it successfully.

note however that there are some caveats...

the first one of course is that it only supports mandriva currently. this is not a fatality, i expect other distributions (and why not other platforms such as *bsd or solaris) to be handled later on if they want. after all, i'm using cpan2dist underneath to generate the packages, and there exist some backends besides cpanplus::dist::mdv. if you're interested, here's the git repository - and i'm ready to give push rights. heck, if you insist i might even agree to move to github which seems to have all the hype those days.

the second one is that the interface is rather ugly and has a lot of rough edges. i'm waiting for dams to continue his work on curses::toolkit, so i can drop curses::ui::poe. i might also provide a tk or wx version at some point. in the meantime, you have to know the application a bit to understand what's going on. i intend to polish it later on.

finally, the code can be cleaned - but you don't care as a user.

so, that's it. cpan2pkg is live... enjoy!

dependencies done right

dave has a point regarding cpan module dependencies: it can be troublesome to install everything correctly. however, we can get the best of both worlds... if we create packages from cpan modules for linux distributions. indeed, we can then use all the tools that the distributions have worked on and polished since so long a time. not counting the fact that's definitely the goal of a linux distribution to simplify things, and integrate software ready to be used.

to do so, we're standing once again on the shoulders of giants: cpanplus comes with various backends. of course, the most used is the one that really installs the modules. but with cpanplus::dist, it's easy to create modules for your distribution: debian, mandriva or fedora backends exist. and thus, creating a package for your distribution is as easy as (for mandriva in this case):

$ cpan2dist --format CPANPLUS::Dist::Mdv Foo::Bar
$ sudo rpm -Uvh ~/rpm/RPMS/noarch/perl-Foo-Bar.rpm

of course, this is only half of the story. after all, we don't want the users to do this - we're indeed trying to save them the hassle of building the modules.

so, we need to create the rpms (or debs) and provide them in the distribution. which is a bit more difficult, since we need to create the packages for the wanted module, and all the prereqs. and for each of them, we need to import them in the distribution build system. all that can take quite some time.

which is why i'm currently working on app::cpan2pkg (readers of this blog have already heard about that), which does everything automatically - well, that's the goal, at least. if you're a contributor, just run:

$ cpan2pkg Foo::Bar

it will build automatically the packages for Foo::Bar and all its dependencies, import them in the distribution repository, and send them (in order) on the distribution build system, where they will be made available for users later on...

and the end result is achieved, as an end user:

$ sudo urpmi perl-Foo-Bar

that will install Foo::Bar along with all its dependencies. no build fuss, no dependency done wrong, no need to have this gcc or that lib installed. everything will be fetched & installed for you...

so, we still retain the testing culture because packagers continue the regular build with all the test suite. users have their modules ready to be installed as easy as ever. and maybe we'll stop hearing about people complaining about long dependency chains. because dave is right: who gives a shit about that?

note to self: now just go back to work on it so i can release v1.0.0!

cpan2pkg now interacts with upstream buildsystem

as you're reading those lines, app::cpan2pkg 0.5.0 is propagating on cpan.

the changes are quite interesting, since it allows cpan2pkg to go from local build to upstream buildsystem:

• importing package upstream
• submitting package upstream
• waiting for user confirmation when package is available on upstream build system
  

warning, though: currently, it doesn't take the module requirements into account before submitting a module on upstream build system. this will be fixed in next version.

but it's quite an achievement... just running "cpan2pkg Foo::Bar" will build the package with all its prereqs, install them locally, import them in upstream repository, and submit them on upstream build system. (i know, i know, only mandriva is supported as of today.)

all in all, this means that i'm nearing v1.0.0. all the basic pieces are now in place, and it's just a matter of re-ordering things, adding a couple of conditions, some fixmes to deal with... and some cleaning. a lot in fact! :-) stay tuned.

app::cpan2pkg 0.4.1 released

now that padre is working again on mandriva, i've been able to make some progress on cpan2pkg...

as explained in a previous post, version 0.4.0 of app::cpan2pkg was providing the basis to completely generate and install local packages.

so, the changelog for 0.4.1 is pretty straightforward:

• storing when a module is complete
• build module when all prereqs have been built & installed
• better prereqs extraction

i had the, ahem, pleasure (for a given definition of pleasure) to discover that cpan2dist is pretty dumb in its arguments parsing.

option --nobuildprereqs does not work as one might think - it really means: "don't rebuild prereqs if they are at latest version, but rebuild anyway if we're not at latest version". yeah, right, i'm stuffing --nobuildprereqs but please rebuild them anyhow if you want...

so i cheated and wanted to use the --ignore option of cpan2dist, but somehow, using a regex with look-behind assertion such as /(?<!$name)$/ does not seem to work - while it is working as intended when testing it on various strings. sigh.

anyway, i've managed to go through this. and this means that the local part of cpan2pkg is complete!

so, now i can work on upstream package importing. and maybe on some mechanism to prevent 50+ processes to be launched at the same time, for modules with a lot of prereqs... but that's, as they say, refinment! :-)

app::cpan2pkg 0.4.0 released

app::cpan2pkg 0.4.0 is live. packages are now being built (with cpan2dist from CPANPLUS) and installed - if there's no missing prereqs, of course.

note that not all the dependency updates are treated afterwards, since it needed to have the build-package-and-install-it-locally step completed first (obviously). that would be next on the agenda.

once this is done, everything related to package generation / install will be done. and i'll then work on submitting packages upstream...

app::cpan2pkg now on cpan!

app::cpan2pkg 0.3.0 is currently spreading its way on cpan. since it's the first public release, give it some time to propagate.

you should be able to download it soon, and then play with it:

cpan2pkg Foo::Bar Acme::Baz

here's a list of what it's doing:

• check if module is already installed locally
  
• check if module is provided by your distribution
• install from your distribution if possible
• find the module prereqs and starts over for each of them
  

note that only mandriva is supported currently. i intend to support more distribution later on (read: after v1.0.0), but don't hesitate to join the fun if you want to add support for your distro right now!

next steps are synchronizing prereqs, running cpan2dist for the module, importing resulting package into upstream...

since it's a bit rough on the edges (curses::ui is not the best toolkit ever - i'm waiting for your new toolkit dams! hint, hint), i also plan to have a better gui someday...

once again, don't hesitate to ping me if you want to contribute. the git repository is online: clone it, everyone can also push on the mob branch, or i can even give you commit rights on it.

cpan modules & native linux packages

cpan is great - one finds a lot of modules for really everything. the problem comes from installing those modules.

indeed, your system comes with a perl and some cpan modules packaged for your distribution. and mixing them with modules installed directly by cpan(1) or cpanp(1) is the best way to wreck havoc on your system.

enters cpanplus backends, allowing one to easily create native linux packages for you distribution using:

cpan2dist Foo::Bar

pretty straighforward, assuming there's a backend available for your distribution...

so i created cpanplus::dist::mdv, and life was good: i created rpms when i wanted some new modules. it has even become the official way to create rpm packages from cpan modules for mandriva.

unfortunately, it's still not enough, even if jos agreed to integrate a --install flag in cpan2dist to install prereqs after building them.

indeed, you don't want to build a prereq when this one might be already available from your distribution. moreover, once a package is installed, you'd like to submit the new package to your distribution, for other users to benefit from them (assuming you're a contributor for your distribution).

lazyness helping, i always let it go. but lazyness strikes back: always repeating the same steps to check if a prereq is provided by mandriva, building them or installing them, and then submitting them is not that a good way to spend my time.

so one week ago, i've decided to create an app::cpan2pkg module (not yet on cpan), a curses application build around cpan2dist and taking care of every itch mentioned earlier.

it is really a glue application, meant to reuse existing command lines, but taking the decisions automatically and chaining the steps.

note that it's called app::cpan2pkg, and not app::cpan2mdv. i intend to write it so that one can adapt it to other distributions workflows. of course, currently i'm focusing on mandriva, the distribution i'm using. but if others want to plug in, they're more than welcome!

it is currently in version 0.1.0 (except to see it soon on cpan), and is able to extract the module prereqs. i'll let you know about the progress...