Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"

LWP::UserAgent hacking

by DaTa! (Acolyte)
on Mar 04, 2006 at 15:09 UTC ( [id://534473]=perlquestion: print w/replies, xml ) Need Help??

DaTa! has asked for the wisdom of the Perl Monks concerning the following question:

I want LWP::UserAgent to log the following information:

requested uri, time it took to fetch, content length and header length.

The following code works unless somebody uses :content_cb, in this case $content_length is zero, because the content is not included in the HTTP::Response object. Has somebody an idea to get the content-length anyway?

Here is my code.

package LWP::UserAgent::Trace; use Time::HiRes; use Data::Dumper; use base qw/LWP::UserAgent/; sub send_request { my($self, $request, $arg, $size) = @_; my $t0 = [Time::HiRes::gettimeofday]; my $response = $self->SUPER::send_request($request,$arg,$size); my $elapsed = Time::HiRes::tv_interval($t0); my $content_length = length($response->content); my $header_length = length($response->headers_as_string); print STDERR sprintf "%s %f %d %d %d\n", $request->uri, $elapsed, $content_length, $header_length, $content_length+$header_length; return $response; } 1;

Replies are listed 'Best First'.
Re: LWP::UserAgent hacking
by Thelonius (Priest) on Mar 04, 2006 at 19:57 UTC
    Here's a method that will work for chunked responses, too:
    sub send_request { my($self, $request, $arg, $size) = @_; my $content_length = 0; if (ref $arg && ref $arg eq "CODE") { my $original_arg = $arg; $arg = sub { $content_length += length($_[0]); return &$original_arg(@_); }; } my $t0 = [Time::HiRes::gettimeofday]; my $response = $self->SUPER::send_request($request,$arg,$size); my $elapsed = Time::HiRes::tv_interval($t0); $content_length = length($response->content) unless $content_length; my $header_length = length($response->headers_as_string); print STDERR sprintf "%s %f %d %d %d\n", $request->uri, $elapsed, $content_length, $header_length, $content_length+$header_length; return $response; }
Re: LWP::UserAgent hacking
by Thelonius (Priest) on Mar 04, 2006 at 17:10 UTC
    As long as LWP doesn't support chunked responses (I think it still doesn't), this will work:
    my $content_length = $response->content_length;

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://534473]
Approved by wfsp
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others avoiding work at the Monastery: (2)
As of 2024-07-19 22:32 GMT
Find Nodes?
    Voting Booth?

    No recent polls found

    erzuuli‥ 🛈The London Perl and Raku Workshop takes place on 26th Oct 2024. If your company depends on Perl, please consider sponsoring and/or attending.