Re^3: Sorting a (very) large file

[creamygoodness]

Y'know, Sort::External is designed for exactly what the original poster wants to do... but few people seem to find it on CPAN.

Sort::External was my first CPAN distro, and the people on the modules mailing list prevailed upon me to release it it under the academically correct name. I occasionally regret having taken that advice.

FWIW, here's the definition of "external sort" from nist.gov:

Any sort algorithm that uses external memory, such as tape or disk, during the sort. Since most common sort algorithms assume high-speed random access to all intermediate memory, they are unsuitable if the values to be sorted don't fit in main memory.

[salva]

but few people seem to find it on CPAN

On search.cpan.org, searching for "sort" returns almost 2000 hits, so it's hardly surprising that people doesn't find your module :-(

[tye]

And if you try to search for something more specific than just "sort", then search.cpan.org gives about the worst reaction possible. It searches for any hits that match any of your keywords and then sorts them alphabetically (it appears, the last times I tested). So you get the 2000 hits for "sort" plus another huge batch of hits for "external" or "large" or whatever, and the matches that include both keywords get no preferential position in the output.

Luckily, you can just skip search.cpan.org's stupid search technique and use google w/ "site:search.cpan.org". Compare: Using "large", search.cpan (sort large) completely misses the point (the first hit never even mentions "sort" and Sort::External is in the first page of results only because the "E" from "External" comes quite early in the alphabet) while google (site:search.cpan.org sort large) understands the idea and includes real hits, including Sort::External on the first page (our search terms are still too general). Using "huge" (a less common term), search.cpan (sort huge) does no better while google (site:search.cpan.org sort huge) gives us Sort::External as the very first match.

We should certainly keep the "perldoc" flavor of 'searching' at search.cpan.org, but the other searching should probably just be dropped and replaced with a form that uses google. The current "general" search is just sad.