Looking for news crawler module

[nysus]

What modules are people using out there to crawl newspaper sites for articles containing specific keywords? I'm all ears. Thanks.

$PM = "Perl Monk's";
$MCF = "Most Clueless Friar Abbot Bishop";
$nysus = $PM . $MCF;
Click here if you love Perl Monks

[Ovid]

My first thought would be to check out the news related modules on the CPAN. Additionally, check out the RSS modules. RSS (RDF Site Summary) is a great tool and should help out. However, that requires that the sites you wish to crawl have RSS feeds.

There are also the google-related modules. I'm particularly curious about the DBD::Google module. It might be perfect, but it looks new.

Cheers,
Ovid

New address of my CGI Course.
Silence is Evil (feel free to copy and distribute widely - note copyright text)

[cacharbe]

The only caveat I would put out there for news crawlers is that if you are going to use Google as your source, please, please, please read their usage requirements. They are extremely touchy about automated processes using their data.

And on that note, may I point you at a new book by O'Reilly

C-.

---
Flex the Geek