Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re: Passing timeout params through WWW::RobotRules

by trwww (Priest)
on Apr 17, 2010 at 16:12 UTC ( #835261=note: print w/replies, xml ) Need Help??

Help for this page

Select Code to Download


  1. or download this
    use LWP::UserAgent;
    use WWW::RobotRules;
    ...
    } else {
      die "cant fetch $robots_url: " . $response->status_line;
    }
    

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://835261]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (5)
As of 2019-07-22 20:16 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    If you were the first to set foot on the Moon, what would be your epigram?






    Results (21 votes). Check out past polls.

    Notices?