edan has asked for the wisdom of the Perl Monks concerning the following question:


I have a script that makes an HTTP request directly by opening a socket and reading the response (I can't use LWP::UserAgent because the program is multi-tasking, so it can't block while waiting for the response). Okay, so now that I have the HTTP response in a buffer, I want to know what I've got there. Is there an easy way to turn it into an HTTP::Response object, so I can query some info out of it? I didn't see an appropriate method in the perldoc...

In fact, all I really need is the response code. So, if no one has a good idea of how to turn it into an HTTP::Response, perhaps you can have a look at this snippet and see if it looks okay as far as parsing the response for the code:

# $buf has the reponse read directly off the socket # the first line should be the status line my $response_ok = $buf =~ /^(HTTP[^\r\n]+)\r?\n/i; my $status_line = $1; if (not $response_ok) { # yikes, response has no status line!? (bail out) } my ($protocol, $status, $msg) = split ' ', $status_line; if ($status == 200) { # success, got 200 (OK) response } else { # error, not OK response }



Replies are listed 'Best First'.
Re: create a HTTP::Response object from a raw buffer?
by rob_au (Abbot) on Mar 30, 2003 at 11:38 UTC
    There is a module from the POE project which performs something similar to this task - POE::Filter::HTTPD. This module can be used to convert a raw stream into a HTTP::Request object and a HTTP::Response object into the corresponding stream of data. While this does not exactly meet your requirements, it should provide a good start on any endeavour to create such a tool.

    Note too, that the filter modules from the POE project can be used independently of POE and as such, there is no requirement to employ this entire framework (although, depending upon your task, it could make things immensely easier).


    perl -le 'print+unpack("N",pack("B32","00000000000000000000001001000100"))'

Re: create a HTTP::Response object from a raw buffer?
by tachyon (Chancellor) on Mar 30, 2003 at 12:02 UTC

    If you just want the codes (as you say) just read the header....If you want an HTTP::Response Object just create one by splitting the return into header and content. Parse the header ( All in form Key: Value\n - should unwrap soft line breaks first with s/\n^\s+//gm first ) then build your HTTP::Response object out of the parsed header and content. Note HTTP::Response is a subclass of HTTP::Message which also uses HTTP::Headers so you will have to read the (brief) docs for all of these. Data::Dumper will show you the Response object which is just a hash of hashes structure.

    C:\>type test.pl use IO::Socket::INET; $get = 'www.perl.org'; $sock = IO::Socket::INET->new(PeerAddr => $get, PeerPort => 'http(80)', Proto => 'tcp'); print $sock "GET http://$get HTTP/1.0\015\012\015\012"; read( $sock, $chopped_header , 100); close $sock; my ( $rc, $msg ) = $chopped_header =~ m/HTTP[^\s]+\s+(\d+)\s+([^\n]+)/ +; print "$chopped_header.....[blah]\n\n"; print "RC=$rc and MSG=$msg\n"; C:\>perl test.pl HTTP/1.0 200 OK Date: Sun, 30 Mar 2003 11:59:58 GMT Server: Apache/2.0.44-dev (Unix) DAV/2 Accept.....[blah] RC=200 and MSG=OK C:\>




Re: create a HTTP::Response object from a raw buffer?
by BrowserUk (Pope) on Mar 30, 2003 at 11:47 UTC

    If you can't use a particular module that performs the task you have to do for any reason, then there is nothing to stop you inspecting the source of that module and seeing how they do it.

    For your particular purpose--extracting the status code--you might take a look at the function  _trivial_http_get() in LWP::Simple. It looks like this does pretty much exactly what you want to do, though you may not want the redirection processing. Then again, maybe you do:)

    Examine what is said, not who speaks.
    1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
    2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
    3) Any sufficiently advanced technology is indistinguishable from magic.
    Arthur C. Clarke.
Re: create a HTTP::Response object from a raw buffer?
by pg (Canon) on Mar 30, 2003 at 18:13 UTC
    Depends on the nature of your application, you may want to consider mod_perl. LWP::UserAgent requires a new thread/process for each session in a multi-user environment. If your site is "hot", and may get many hits within a minute, LWP::UserAgent is absolutely not a choice.

    Anyway, the following piece of code receives the response, and parse the header into a hash.
    use IO::Socket::INET; use Data::Dumper; use strict; my $s = new IO::Socket::INET(Proto => "tcp", PeerAddr => "www.yahoo.com", PeerPort => 80) || die "failed to new socket\n"; print $s "GET http://www.yahoo.com HTTP/1.1\r\nHost: www.yahoo.com\r\n +\r\n"; my $response; while (<$s>) { chomp; if (m/^\s*$/) {last}; m/(.*?)[:| ]\s*(.*?)[\r|\n]/; $response->{$1} = $2; } close($s); print Dumper($response);
Re: create a HTTP::Response object from a raw buffer?
by jdporter (Canon) on Mar 31, 2003 at 15:06 UTC
    Sounds to me like you probably want the Net::HTTP::NB module (a non-blocking HTTP client), and especially its read_response_headers method. See the module for example and description, but in a nutshell you can do this:
    use Net::HTTP::NB; my $s = Net::HTTP::NB->new( Host => 'www.perlmonks.org' ); $s->write_request( GET => "/" ); my( $code, $msg, %headers ) = $s->read_response_headers;
    (However, don't actually do it like that. See the module for a better example.)

    The 6th Rule of Perl Club is -- There is no Rule #6.