Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Re^2: Get Timout with LWP

by IlyaM (Parson)
on Jun 21, 2004 at 15:54 UTC ( #368490=note: print w/ replies, xml ) Need Help??


in reply to Re: Get Timout with LWP
in thread Get Timout with LWP

LWP::Simple is just that. Simple. It will hang if it can open a socket to a server but the server then plays dead and does not respond. LWP::UserAgent is much more sophisticated with its timeouts and will return if it times out.......

This is quite wrong. LWP::Simple is just a wrapper on top of LWP::UserAgent so it behaves exactly same way as LWP::UserAgent concerning timeout handling. The only exception is that LWP::Simple always uses default timeout value (180 seconds) and with LWP::UserAgent timeout can be set.

--
Ilya Martynov, ilya@iponweb.net
CTO IPonWEB (UK) Ltd
Quality Perl Programming and Unix Support UK managed @ offshore prices - http://www.iponweb.net
Personal website - http://martynov.org


Comment on Re^2: Get Timout with LWP
Re^3: Get Timout with LWP
by tachyon (Chancellor) on Jun 22, 2004 at 01:45 UTC

    Unfortunately I am not wrong. The hanging behaviour is real. The underlying reason is LWP::Simple uses (for http) the _http_trivial_get() function, not LWP::UserAgent. This implements a 60 second IO::Socket::INET make connection timeout, but no timeout on data recieve. Modifying the test code to use LWP::Simple proves the point (LWP::Simple will wait 500 seconds to get the data - it gets an instant socket, then nothing for 500 seconds). This function remains active in the latest LWP::Simple.

    I agree it should really be a subclass of LWP::UserAgent but for http it is not. The timeout behaviour for LWP::Simple and LWP::UserAgent could IMHO use some work.

    C:\>type server.pl #!/usr/bin/perl use IO::Socket; use IO::Select; $lsn = IO::Socket::INET->new( Listen => 1, LocalAddr => 'localhost', LocalPort => 9000,); my $client = new IO::Select( $lsn ); while( my @ready = $client->can_read ) { for my $fh (@ready) { if($fh == $lsn) { warn "Accepted new socket\n"; my $new = $lsn->accept; $client->add($new); } else { # Process socket warn "Getting data\n"; $data = <$fh>; # yeah yeah warn "Got $data\nDoing nothing forever!\n"; my @response = split '', "HTTP/1.1 200 OK\n\nHello World!\ +n"; sleep 500; print {$fh} @response; $client->remove($fh); $fh->close(); } } } C:\>type lwp.pl #!/usr/bin/perl $|++; use LWP::Simple; $start = time(); print "Begin at $start\n"; $response = get('http://localhost:9000/'); $end = time(); my $time = $end - $start; print "Done at $end\nTook $time seconds\nGot:\n$response\n"; C:\>perl lwp.pl Begin at 1087868116 Done at 1087868616 Took 500 seconds Got: Hello World! C:\>

    cheers

    tachyon

Re^3: Get Timout with LWP
by tachyon (Chancellor) on Jun 22, 2004 at 02:47 UTC

    Further to the previous, here is a possible patch to implement timeout behaviour in LWP::Simple. There is no way to set $timeout passed to _trivial_http_get() included. A global would be easiest I guess given the very simple interface. Defaults to 180 seconds.

    $ diff -u Perl/site/lib/LWP/Simple.pm Perl/site/lib/LWP/Simple-patch.p +m --- Perl/site/lib/LWP/Simple.pm Tue Jun 22 12:40:14 2004 +++ Perl/site/lib/LWP/Simple-patch.pm Tue Jun 22 12:42:15 2004 @@ -154,9 +154,10 @@ sub _trivial_http_get { - my($host, $port, $path) = @_; + my($host, $port, $path, $timeout) = @_; #print "HOST=$host, PORT=$port, PATH=$path\n"; - + $timeout ||=180; + my $hard_timeout = time() + $timeout; require IO::Socket; local($^W) = 0; my $sock = IO::Socket::INET->new(PeerAddr => $host, @@ -174,8 +175,15 @@ my $buf = ""; my $n; - 1 while $n = sysread($sock, $buf, 8*1024, length($buf)); - return undef unless defined($n); + while( 1 ) { + my $remaining = $hard_timeout - time(); + return undef if $remaining <= 0; + if ( can_read( $sock, $remaining ) ) { + $n = sysread($sock, $buf, 8*1024, length($buf)); + return undef unless defined($n); + last if $n == 0; # we are eof + } + } if ($buf =~ m,^HTTP/\d+\.\d+\s+(\d+)[^\012]*\012,) { my $code = $1; @@ -191,6 +199,15 @@ } return $buf; +} + +sub can_read { + my($sock, $timeout) = @_; + my $fbits = ''; + vec($fbits, fileno($sock), 1) = 1; + my $nfound = select($fbits, undef, undef, $timeout); + die "select failed: $!" unless defined $nfound; + return $nfound > 0; } Administrator@JAMES /cygdrive/d

    cheers

    tachyon

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://368490]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (7)
As of 2015-07-07 04:52 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The top three priorities of my open tasks are (in descending order of likelihood to be worked on) ...









    Results (87 votes), past polls