|Problems? Is your data what you think it is?|
A preview of DPANby brian_d_foy (Abbot)
|on Nov 11, 2008 at 12:03 UTC||Need Help??|
...This is a preview just for Perlmonks of the continuing work I'm doing with my BackPAN indexer....
MyCPAN can now create CPAN-like directories out of a directory of distributions. Run a script then point CPAN.pm at your directory to use it as your CPAN source. This worked was sponsored by a customer at the day job (and talk to me if you can convince your boss that this might be something worthwhile to sponsor too).
Previously, you could do this task with a minicpan and CPAN::Mini::Inject. You kept two repositories. You updated minicpan, which undid all of your private stuff, then you re-injected everything. CPAN::Mini::Inject then updated the modules/02package.details.txt.gz and CHECKSUMS files. That's fine if you're injecting a few things.
My task is to create a CPAN-like structure of stuff that is mostly not on CPAN, or when nothing in the private CPAN comes from the real CPAN. We've been calling this "DPAN", for DarkPAN. You don't have to worry about what's in a distro or which author it should belong too, and you don't have parallel directories. Just dump a bunch of distros in a directory. Those might be private modules, CPAN modules, forked modules, vendor modules, and so on. DPAN doesn't care. Just dump them in a directory.
MyCPAN::Indexer pulls out all of the information and turns the source directory into something that the CPAN tools can understand.
You start with MyCPAN::Indexer. It's still in development, so some things are a little rough. Install it or get it from Github. Install the dependencies.
Inside MyCPAN::Indexer is an examples/ directory with a bunch of junk in it. You want the dpan script.
With the defaults, this looks for all distributions under my_modules_dir, collects information about each and puts it in the indexer_reports/ directory. It then goes through all of the reports and collects the information it needs for the CPAN index files. Finally, in my_modules_dir/ it creates the modules/ directory with the index files the CPAN tools need and puts a CHECKSUMS file in each directory that has distributions in it. You can now point CPAN.pm to this directory and install directly from it.
There are a couple of things to watch out for:
The lastest version of my cpan script might help you. You can dump and load configs without fooling with the shell. The -J (capital J) will dump the current config to STDOUT. It's the same format as CPAN::Config:
Edit that file how you like. I change the urllist.
I have several versions for testing different things. If I want to install Foo::Bar with my DPAN config pointing to my DarkPAN, I load the right configuration with -j (lowercase j):
Now, I've said that DPAN is for DarkPAN, but it's also for another thing I want to do: DistributedPAN. If you look in 02packages.details.txt, you'll see lines like:
When I created CPAN::PackageDetails to play with this, we discovered that CPAN.pm will happily deal with absolute paths there. The distributions files could be anywhere:
Once I started thinking about that, I wanted to make it so the files don't even have to be local:
Once that third column handling is refactored into a general URl or file fetcher, things get more interesting. I haven't looked at what that might take in CPAN.pm though.
And, since I was writing CPAN::PackageDetails, I wanted to support another possible format. This one has a column for the author and might list the same namespace several times
Remember Synopsis 11? Perl 6 supports not only version restrictions on loading a module, but loading the same module from different authors:
With a change to 02packages.details.txt, the CPAN tools can support this too.
Not to worry though. That's just something fun to think about right now. Once the rest of DPAN seems stable, I can start adding cool features like that.
brian d foy <email@example.com>
Subscribe to The Perl Review