Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked

Re^3: What Tools Do You Use With WWW::Mechanize

by Limbic~Region (Chancellor)
on Oct 04, 2011 at 18:43 UTC ( #929637=note: print w/replies, xml ) Need Help??

in reply to Re^2: What Tools Do You Use With WWW::Mechanize
in thread What Tools Do You Use With WWW::Mechanize

Have you seen WWW::Mechanize::Sleepy? Personally, I use something along the lines of:
# Sleep a random interval between $duration and 2 * $duration - 1 unit sub rest { my ($duration) = @_; sleep $duration; sleep rand($duration); } sub fetch_page { my ($mech, $action, $target, $max, $duration) = @_; for (1 .. $max) { rest($duration); eval {$mech->$action($target);}; return if ! $@ && $mech->status == OK; } die "Failed to fetch '$url' after '$max' attempts\n"; }

Of course, if you want to allow for HTTP redirects then you will need to change status == OK to include acceptable HTTP codes. Additionally, if you use Time::HiRes to overload sleep, you can easily sleep for partial minutes. In truth, I typically use milliseconds.

Cheers - L~R

Replies are listed 'Best First'.
Re^4: What Tools Do You Use With WWW::Mechanize
by OfficeLinebacker (Chaplain) on Oct 05, 2011 at 13:34 UTC
    I hadn't even heard of WWW::Mechanize::Sleepy! That's great. During debugging I get impatient but once the product is rolled out it only has to run once a day or so, so using seconds is fine. Also I think you mean "you can easily sleep for partial seconds," right?

    I like computer programming because it's like Legos for the mind.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://929637]
and all is quiet...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (7)
As of 2018-06-25 10:48 GMT
Find Nodes?
    Voting Booth?
    Should cpanminus be part of the standard Perl release?

    Results (126 votes). Check out past polls.