Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl Monk, Perl Meditation

comment on

( [id://3333]=superdoc: print w/replies, xml ) Need Help??

Fellow Monks,

A strange problem has arisen today. Most strange. I have the following code:

my $res = $ua->get( $url, ':content_file' => $filename );

being used under LWP::UserAgent. The returned content, stored in the file $filename, seems to contain the HTTP headers as well as the content - I was expecting just the content. So, for example, if $url references a JPG on the Internet, I don't get a readable JPG back - I seem to get a full HTTP response.

The code, under Windows, works fine. Ran on a Linux box, broken. Now that is odd!

Same issue if getstore(...) is used from LWP::Simple instead of the full UserAgent... Same issue if I store the Response in a variable and use/print it. No matter what I do, I get these unwanted HTTP header lines at the top of the file/response which means the file returned by LWP is actually useless with manual modification...

Opening the resulting file in an editor I seem to see a ^M at the end of the lines - certainly in the unwanted header-looking stuff. Could this be something significant? If so, why is LWP putting it there? Don't worry about this bit anymore ;)

This is a brand new build of a Fedora 14 desktop. Windows was using an older build of ActiveState.

The docs say that I should ONLY get the content into $filename - so why are those HTTP headers there too!.... :(

**UPDATE** A quick review of my code shows I am calling the "get" or "getstore" from within a thread. Outside of threading, files are retrieved as expected. In a threaded environment, the data stored contains these unwanted HTTP headers. Question still remains though......why?

fx, Infinity is Colourless

In reply to LWP UserAgent and Simple keeping headers in content by fx

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?

What's my password?
Create A New User
Domain Nodelet?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (4)
As of 2024-05-30 03:50 GMT
Find Nodes?
    Voting Booth?

    No recent polls found