LWP Timeout and Alarm handling

[daviddhall]

Ok, I've looked at the random bits of coded suggestions people have posted for handling http timeouts and unreachable servers. And am still a bit confused.

I have a script I've written to grab a bunch of urls from a server; however, every so often something chokes (I'm presuming on the server side) and my code just sits there trying to grab the page without ever returning. So, I was investigating the LWP::UserAgent timeout settings and it didn't seem to work (never times out) for this particular problem. I'm presuming that the server is basically giving some sort of error similar to the richard.com delima. But I don't know exactly and don't know how to figure it out.

So, I'm trying to hardcode a timeout so that it will try the url again after a certain amount of seconds since I'm pretty sure the server is available and should return a response.

The code below seems to work but looks kinda sloppy. Is there a better way to do all this? (I'm using LWP::Simple to keep things simple but could use whatever) Thanks in advance for your help!

$SIG{ALRM} = \&timed_out;

$tryurl = 1;
while ($tryurl)
{
    eval 
    {
        print "Getting url... $url\n";
        alarm(10);
        $response = get $url; 
        alarm(0);
    };

    if ($@ =~ /BLAH/) 
    {
        print "Timed out. Trying again...\n";
        $tryurl = 1;
    }
    else
    {
        $tryurl = 0;
    }

}
print "Got url!\n";

sub timed_out {
    die "BLAH"
}

[Cine]

If shorter is better: ;)

$SIG{ALRM} = sub {die}
while (1) {
    print "Getting url... $url\n";
    eval   {
        alarm(10);
        $response = get $url; 
        alarm(0);
        last; #It should die before coming here
    };
   print "Timed out. Trying again...\n" :
 }
print "Got url!\n";



T I M T O W T D I

[blakem]

I've had trouble initializing LWP::UserAgent timeouts when the ua object is created (i.e. my  $ua = LWP::UserAgent->new(timeout => 30);) But have had better luck setting the timeout somewhere outside of the new call:

my $ua = new LWP::UserAgent();
$ua->timeout(30);

Of course your mileage may vary....

-Blake