AFAIK CPAN is regularly "trimmed" so that it can fit on a single CD-ROM, so just using wget should be fine. If it downloads too much you can always just delete what you don't need.

Good luck.

AFAIK CPAN is regularly "trimmed" so that it can fit on a single CD-ROM, ...

Not unless you have a 1.3G CDROM:

blue.stonehenge.com>> du -sk /data/mirror/CPAN/
1255258 /data/mirror/CPAN/
[download]

-- Randal L. Schwartz, Perl hacker

See http://www.cpan.org/misc/cpan-faq.html#What_do_I_need

bluto

Hi,

we're mirroring CPAN via rsync here too. As merlyn already pointed out, you need at least 1,3GB of free space NOW. I'd recomend at least 2.0GB as CPAN is growing and you need some overhead for logs etc.

arkab:/space/scrap/mirrors # du -s cpan
1290032 cpan
[download]

As for the time one needs for rsyncing: It's very fast. With a 2MBit line you need about 70minutes for the initial fetch. The updates are rsynced in about 5 minutes (or less) each.

Bye
 PetaMem

1. You may need to find the package called mirror elsewhere; at least from my part of the net I can't see it on sunsite.

2. Wget or rsync may be very good. But watch out, though I have no idea who did it, search.cpan.org once blackholed a huge network area due to some idiot who tried to snarf down every page from their engine and overloaded the site. They recently un-blackholed the network (which happens to be a giant Internet provider that also happens to serve an Internet cafe near my house in Tokyo), but felt it was a completely valid response. So watch out with that wget partner! Conversely, it may not matter but if wget hits a robots.txt exclusion file it ignores it for the top url only in a recursive download.

3. I'm guessing that if you don't download obsolete versions it might really fit into a CD. Anybody?

4.Possibly the programmer's interface to cpan (I suppose CPAN::Shell->expand("Module","/*/") or something similar might do it but not sure if you wouldn't miss some things. Conceivably CPAN's get command might just do it all for you, dunno.

4. Note some modules apparently don't have version numbers.

Can any one suggest a sane way to do this, obviously some munging of 02packages.details.txt is required.

The current "mini-CPAN" size is just a bit under 300M. Easily fit on a CDROM. Cool.

The first time you run it, it sequentially fetches all of the modules, which is nicer on the source machine than doing them in parallel. After that, it fetches just what might have changed.

-- Randal L. Schwartz, Perl hacker

Cool!!! Mad props to Merlyn.

The ultimate present for the japh who has got it all.
And still plenty of room to spare on the CD!

I starting wondering how long before CPAN doubles to almost fill that CD.. but couldn't find a graph of CPAN growth. Can anybody post copies of du-k.gz with the date of download on it? A small sample and we can build a curve. I just downloaded one and called it du-k.2002-0808.gz (though I think the browser or somebody decompressed it before I saw it).

I'm the author of the one on midnightrealm. The reason it was written to be nasty and use Parallel is that the machine from which it was copying was on the local network. And it was for a one off burn, thus the lack of deletion features.

Use merlyn's. He's seen mine and has made it much better.

-- Iain, aka Koschei.