Beefy Boxes and Bandwidth Generously Provided by pair Networks
Pathologically Eclectic Rubbish Lister
 
PerlMonks  

Re: Passing timeout params through WWW::RobotRules

by trwww (Priest)
on Apr 17, 2010 at 16:12 UTC ( #835261=note: print w/replies, xml ) Need Help??

Help for this page

Select Code to Download


  1. or download this
    use LWP::UserAgent;
    use WWW::RobotRules;
    ...
    } else {
      die "cant fetch $robots_url: " . $response->status_line;
    }
    

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://835261]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others perusing the Monastery: (8)
As of 2020-04-06 17:49 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    The most amusing oxymoron is:
















    Results (40 votes). Check out past polls.

    Notices?