Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

IO::Socket, Multiple GET.

by adismaug (Acolyte)
on Jul 02, 2009 at 11:13 UTC ( #776706=perlquestion: print w/ replies, xml ) Need Help??
adismaug has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,
I am trying to write a script that will perform an HTTP get and analyze the server response.
This should be done in a loop and for performance I would like to create the socket to the server only once and perform the GET on the open socket.
When running my script the socket gets closed and I get an error. Only if I open a new socket, which makes me perform the syn, syn-ack, ack all over again, am I able to perform the GET.
How can I accomplish this with one socket?
This is the main script:

#!/usr/bin/perl -w use IO::Socket; use Encode; use utf8; unless (@ARGV > 0) { die "usage: host ..." } $host = shift(@ARGV); $EOL = "\015\012"; $BLANK = $EOL x 2; $remote = IO::Socket::INET->new( Proto => "tcp", PeerAddr => $host, PeerPort => "http(80)", ); unless ($remote) { die "cannot connect to http daemon on $host" } $remote->autoflush(1); while (1) { $msg = "GET /engine3.php?req_sale=5205&page=1\r\n"; $remote->send($msg); while ( <$remote> ) { $temp .= $_; } utf8::encode($temp); print "$temp\n"; undef($temp); sleep(2); # close $remote; }

Comment on IO::Socket, Multiple GET.
Download Code
Re: IO::Socket, Multiple GET.
by Anonymous Monk on Jul 02, 2009 at 11:26 UTC
Re: IO::Socket, Multiple GET.
by jrsimmon (Hermit) on Jul 02, 2009 at 11:48 UTC
    Have you tried setting ReusePort? It's not implemented on my system, so I can't tell whether that would help or not...but worth a shot.
      Using this did not help I am still getting an error.
Re: IO::Socket, Multiple GET.
by jhourcle (Prior) on Jul 02, 2009 at 14:26 UTC
    $msg = "GET /engine3.php?req_sale=5205&page=1\r\n";

    You're sending HTTP/0.9, which doesn't have provisions for persistent connections, and the server will always disconnect you. You need to either send an HTTP/1.1 request, or HTTP/1.0 with the appropriate headers ... and for that, I'd do as poolpi and our anonymous friend have already mentioned, and use LWP::UserAgent (search the instructions for 'keep_alive' ).

      You need to either send an HTTP/1.1 request, or HTTP/1.0 with the appropriate headers

      • HTTP/1.1 does not automatically keep the connection open. An appropriate Keep-Alive header must be sent.

      • Keep-Alive was introduced in HTTP/1.1. HTTP/1.0 does not support keeping the connection open.

        However, since Keep-Alive is compatible with HTTP/1.0, many servers that claim to be HTTP/1.0 support Keep-Alive. This was the cases even when HTTP/1.1 was still a draft.

      • Finally, servers are under no requirement to honour Keep-Alive requests.

Re: IO::Socket, Multiple GET.
by chomzee (Beadle) on Jul 02, 2009 at 16:05 UTC
    Try sending something like:
    GET / HTTP/1.1\r\n Host:\r\n Connection: keep-alive\r\n \r\n
    STFW for "persistent connection" and "HTTP version 1.1" for more information.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://776706]
Approved by ww
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others exploiting the Monastery: (17)
As of 2014-08-27 13:44 GMT
Find Nodes?
    Voting Booth?

    The best computer themed movie is:

    Results (238 votes), past polls