Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer
 
PerlMonks  

WWW::Robot no longer compatible with LWP::RobotUA. Now what?

by nysus (Parson)
on Nov 13, 2005 at 04:49 UTC ( [id://508069]=perlquestion: print w/replies, xml ) Need Help??

nysus has asked for the wisdom of the Perl Monks concerning the following question:

I want to create a robot that spiders a handful of small sites a couple of times a day and sends me an e-mail notification when it finds an updated page. I tried to creating a program that used the WWW::Robot module but can't because WWW::Robot is badly out of date. This has already been discussed at a WWW::Robot robot ?.

So I'm trying to figure out where to go from here. I'd really appreciate some ideas that will save me from reinventing the wheel.

$PM = "Perl Monk's";
$MCF = "Most Clueless Friar Abbot Bishop Pontiff";
$nysus = $PM . $MCF;
Click here if you love Perl Monks

Edit: g0n - fixed link

Replies are listed 'Best First'.
Re: WWW::Robot no longer compatible with LWP::RobotUA. Now what?
by fokat (Deacon) on Nov 13, 2005 at 05:23 UTC

    Dear nysus:

    I want to create a robot that spiders a handful of small sites a couple of times a day (...) I'd really appreciate some ideas that will save me from reinventing the wheel

    Have you seen WWW::Mechanize? I have used it a few times, with excellent results.

    Best regards

    -lem, but some call me fokat

      No, I have not heard of that module before. Thanks. I will look into it.

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff";
      $nysus = $PM . $MCF;
      Click here if you love Perl Monks

Re: WWW::Robot no longer compatible with LWP::RobotUA. Now what?
by Cody Pendant (Prior) on Nov 14, 2005 at 02:56 UTC
    I just found WWW::CheckSite::Spider which looks very useful and recently updated.


    ($_='kkvvttuu bbooppuuiiffss qqffssmm iibbddllffss')
    =~y~b-v~a-z~s; print
Re: WWW::Robot no longer compatible with LWP::RobotUA. Now what?
by spiritway (Vicar) on Nov 14, 2005 at 05:31 UTC

    If you want some more ideas, you might check out Spidering Hacks. It has some great ideas in it, and the examples are in Perl. You can download the code from here.

Re: WWW::Robot no longer compatible with LWP::RobotUA. Now what?
by ghenry (Vicar) on Nov 13, 2005 at 10:04 UTC

    I've looked at LWP::RobotUA, but I can't see that it needs WWW::Robot anywhere? Maybe I've missed something?

    If I have, apologies,
    Gavin.

    Walking the road to enlightenment... I found a penguin and a camel on the way.....
    Fancy a yourname@perl.me.uk? Just ask!!!
      WWW::Robot does not need RobotUA. It's the other way around. WWW::Robot had some features that would have saved me some time, however.

      $PM = "Perl Monk's";
      $MCF = "Most Clueless Friar Abbot Bishop Pontiff";
      $nysus = $PM . $MCF;
      Click here if you love Perl Monks

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://508069]
Approved by spiritway
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2024-03-19 11:14 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found