Re^6: Searching for Modules and Descriptions on CPAN Remotely

[ghenry]

Yeah, I've now figured that out.

If you search for Catalyst::Plugin, you can see the results.

I just want the module name and the text directly below the link result.

Looks like I'll be turning to WWW::Mechanize after all :-(

[randyk]

At search.cpan.org, as well as within CPAN-Search-Lite, attempts are made to obtain module and package descriptions from the pod, if the CPAN indices don't have them. An alternative to downloading and extracting the pod, or using WWW::Mechanize, is to use the csl_soap script that comes with CPAN-Search-Lite to query, via a soap interface, the database that cpan.uwinnipeg.ca uses. If you set, within the script,

my $soap_uri = 
   'http://theoryx5.uwinnipeg.ca/Apache/InfoServer';
my $soap_proxy = 
   'http://theoryx5.uwinnipeg.ca/cgi-bin/ppminfo.cgi';

then calling the script as:

perl csl_soap --module Catalyst::Plugin::Geography

will fetch the available information from the database, including the description.

The mod_search function of PPM::Make::Util indicates how this soap service can accept an array reference of module names. Currently there isn't support for wild-card searches, but that could easily be added, if there was interest.

[ghenry]

Thanks randyk,

This looks interesting, and I will have a play when I get a bit more time.

I gave up on CPAN-Search-Lite as I just couldn't get Apache2-Request installed, as per the install requirement.

Gavin.

[randyk]

It may not have been obvious from what I wrote, but the csl_soap script doesn't require installation of CPAN-Search-Lite; all it needs is SOAP::Lite and Getopt::Long.

[adrianh]

Try this:

#! /usr/bin/perl

use strict;
use warnings;
use CPANPLUS::Backend;

# this will need to be more clever in the general case
# (cross platform paths, odd locations, generated code, etc.)
sub find_pod {
    my ( $dir, $module_name ) = @_;
    my $module_path = join '/', split /::/, $module_name;
    foreach my $extension ( qw( pod pm ) ) {
        my $file = "$dir/lib/$module_path.$extension";
        return $file if -e $file;
    }
    return;
}

sub get_description_from_pod {
    my ( $filename, $module_name ) = @_;
    open my $fh, "<", $filename or die "open failed ($!)\n";
    while ( my $line = <$fh> ) {
        chomp $line;
        return $1 if $line =~ m/^$module_name\s+-\s+(.*)$/s;
    }
    return;
}

sub description {
    my $module = shift;
    return $module->description if $module->description;
    $module->fetch or die CPANPLUS::Error->stack_as_string;
    my $dir = $module->extract or die CPANPLUS::Error->stack_as_string
+;
    return 'unknown' unless my $pod_file = find_pod( $dir, $module->na
+me );
    return get_description_from_pod( $pod_file, $module->name ) 
        or 'unknown';
}

my $cb = CPANPLUS::Backend->new;
my @modules = $cb->search( type => 'module', allow => [ qr/\ACatalyst:
+:Plugin/ ] );
print $_->name, " : ", description( $_ ), "\n" foreach @modules