retrying an action that could fail

[perrin]

For a project using WWW::Mechanize for some site-scraping, I ended up needing to wrap most of the calls that do something requiring a network to allow them to be retried. The combination of networks and servers is just a little too flaky to make thousands of HTTP requests without any of them failing, but they usually go through if you retry them a couple of times.

I wanted a succint way to add this to any action, so I wrote it to take sub refs. I can call it like this:

retry(
      sub {
           $browser->get( $link->url() );
          }
     );

And this is the definition of the retry() sub:

sub retry {
    my $sub_ref = shift;

    for ( 1 .. $conf->max_tries() ) {
        eval { $sub_ref->(); };
        last unless $@;
        warn "Failed try $_, retrying.  Error: $@\n"
          if $conf->debug();
    }

    if ($@) { die "failed after " . $conf->max_tries() . " tries: $@\n
+" }
}

This assumes that a failed action will throw an exception, which is what happens with the settings I'm using with Mechanize.

Now this works great, but I felt like it was sort of a naive approach, and I just wondered if anyone had something clever for this, maybe using some of those less commonly applied loop control constructs that Perl allows. I'm not looking for golf here (although feel free to amuse yourself if you think it sounds fun), but really just wondering if there's a more elegant solution.

[tachyon]

The only thing I would add is a sleep $SLEEP at the end of your loop. Some issues will already have a timout effectively built in but some may benefit from a brief respite (say 503 and 504 errors)

[perrin]

Hmm, maybe an optional additional sub ref for what to do in order to reset between tries would be a good way to handle this. Then I could pass in a sleep call in there if I need one.

[Fletch]

You might could make your syntax a bit less cumbersome by using a & prototype (see perldoc perlsub) to eliminate the need for saying "sub" each time.

[perrin]

It's a good thought. In this case I actually made it explicit like this on purpose so that I don't forget that I'm dealing with sub refs and get myself into trouble with scoping and memory leak issues. I used to have that happen with Error which uses the approach you're talking about.

[dws]

... this works great, but I felt like it was sort of a naive approach, and I just wondered if anyone had something clever for this, maybe using some of those less commonly applied loop control constructs that Perl allows.

It looks O.K. to me on first read.

Your approach reminds me of a talk on resilient computing I heard some folks from Bell Labs give many years back. Part of their approach was to retry I/O operations on failure. If I recall correctly, retrying at most 3 times was enough to make something like 98% of error go away.

To those raised on theoretical Computer Science (which somehow neglected to mention that things like disk drives had moving parts), this was a jaw dropper.

[gjb]

You might want to have a look at Sub::Attempts.

Hope this helps, -gjb-

[Abigail-II]

Yeah, I usually use something like that as well. I often look at the actual respons as well, because some sites I connect to do reply with a message like "Could not connect to the database, retry later". From an HTTP (and hence WWW::Mechanize) point of view, the request succeeded. I typically wait a few seconds before retrying.

Abigail

[demerphq]

You wanted an equivelent that uses the loop operators:

my $count=0;
ATTEMPT:{
        do_something() or ($count++ < $maximum and redo ATTEMPT)
}

Style to taste. :-)

[perrin]

Thanks. Here's a working demo, adapting form the code I had before. (Note that $max normally comes from my config file.)

my $max = 3;
retry( sub { die "failure" } );

sub retry {
    my $sub_ref = shift;
    my $try     = 0;
  ATTEMPT: {
        eval { $sub_ref->(); };
        if ( $@ and $try++ < $max ) {
            warn "Failed try $try, retrying.  Error: $@\n";
            redo ATTEMPT;
        }
    }
    if ($@) { die "failed after $max tries: $@\n" }
}

[demerphq]

Hi Perrin. That was mostly an example. Usually for that kind of thing its more intuitive to use a for loop and last out of it at the appropriate time. Where redo can be useful however is if you want to back out of an position in an if structure. But thats another node.

What I more would be thinking is the stuff below. :-) (This is incidentally how the SKIP blocks work in tests.)

#!perl -l
use strict;
use warnings;
no warnings 'exiting';

my ($x);

sub do_onething {
    if ($x) {
        print $x--;
        redo ATTEMPT
    } else {
        $x=10;
        print "Done onething.";
    }
};
sub do_another {
    if ($x) {
        print "Done another.";
        last ATTEMPT
    }
};


$x=10;

ATTEMPT:{
   do_onething
}

ATTEMPT:{
  for ('A'..'Z') {
    print $_;
    do_another;
  }
}

That is that you can precan a bunch of routines that know how to jump out of specifically named places. So now we can do stuff like

sub retry(&$) {
    my $sub_ref = shift;
    my $max     = shift ||3;

  ATTEMPT:
    for my $try (1..$max) {
        eval { $sub_ref->(); };
        last unless $@;
        warn "Failed $try, retrying.  Error: $@\n"
    }
    if ($@) { die "failed after $max tries: $@\n" }
}

my $t=0;
retry {
        print "Seeya!" and last ATTEMPT if $t++>4;
        die "failure";
      } 10;

:-)

[fglock]

I wonder if you could use LWP::UserAgent::Determined - "a virtual browser that retries errors".

The problem is, I don't know how to make WWW::Mechanize inherit from "LWP::UserAgent::Determined" instead of "LWP::UserAgent".

[simonm]

The problem is, I don't know how to make WWW::Mechanize inherit from "LWP::UserAgent::Determined" instead of "LWP::UserAgent".

You could try just declaring it with @WWW::Mechanize::ISA = "LWP::UserAgent::Determined" after the modules have been loaded; from a first glance at the source it looks like that'd be sufficient.