Beefy Boxes and Bandwidth Generously Provided by pair Networks
Come for the quick hacks, stay for the epiphanies.
 
PerlMonks  

Re: LWP UserAgent and Simple keeping headers in content

by ikegami (Patriarch)
on Nov 29, 2010 at 22:43 UTC ( [id://874381]=note: print w/replies, xml ) Need Help??


in reply to LWP UserAgent and Simple keeping headers in content

Works fine for me on linux with 5.837:

$ perl -MLWP::UserAgent -e'LWP::UserAgent->new->get("http://www.raptor +recoverynebr.org/Imm.%20Snowy%20Owl.jpg", ":content_file" => "image.j +pg");' {17} eric@fmdev10 [~/tmp]$ od -c image.jpg | head 0000000 377 330 377 340 \0 020 J F I F \0 001 001 001 002 +X 0000020 002 X \0 \0 377 333 \0 C \0 005 003 004 004 004 003 00 +5 0000040 004 004 004 005 005 005 006 \a \f \b \a \a \a \a 017 \ +v 0000060 \v \t \f 021 017 022 022 021 017 021 021 023 026 034 027 02 +3 0000100 024 032 025 021 021 030 ! 030 032 035 035 037 037 037 023 02 +7 0000120 " $ " 036 $ 034 036 037 036 377 333 \0 C 001 005 00 +5 0000140 005 \a 006 \a 016 \b \b 016 036 024 021 024 036 036 036 03 +6 0000160 036 036 036 036 036 036 036 036 036 036 036 036 036 036 036 03 +6 * 0000220 036 036 036 036 036 036 036 036 036 036 036 036 036 036 377 30 +0 $ perl -MLWP::Simple -e'getstore("http://www.raptorrecoverynebr.org/Im +m.%20Snowy%20Owl.jpg", "image.jpg");' $ od -c image.jpg | head 0000000 377 330 377 340 \0 020 J F I F \0 001 001 001 002 +X 0000020 002 X \0 \0 377 333 \0 C \0 005 003 004 004 004 003 00 +5 0000040 004 004 004 005 005 005 006 \a \f \b \a \a \a \a 017 \ +v 0000060 \v \t \f 021 017 022 022 021 017 021 021 023 026 034 027 02 +3 0000100 024 032 025 021 021 030 ! 030 032 035 035 037 037 037 023 02 +7 0000120 " $ " 036 $ 034 036 037 036 377 333 \0 C 001 005 00 +5 0000140 005 \a 006 \a 016 \b \b 016 036 024 021 024 036 036 036 03 +6 0000160 036 036 036 036 036 036 036 036 036 036 036 036 036 036 036 03 +6 * 0000220 036 036 036 036 036 036 036 036 036 036 036 036 036 036 377 30 +0

Could you disclose the URL in question? And perhaps the actual code you used?

Opening the resulting file in an editor I seem to see a ^M at the end of the lines

HTTP headers end with CR LF. On Windows, that's the standard text file line ending. On unix, LF is the standard text file line ending, so the "^M" is your editor trying to represent the CR.

Replies are listed 'Best First'.
Re^2: LWP UserAgent and Simple keeping headers in content
by fx (Pilgrim) on Nov 30, 2010 at 00:38 UTC

    Ah....potentially more info from me now!

    Just tried a simple command line "get" and "getstore" and it all works ok. Just re-examined my code and I'm actually calling my "get" or "getstore" from within a thread.

    Now I know threads can cause issues, but is this connected somehow? The "get" is just about working...but not quite completely working. The program isn't completely crashing out or anything...it's just not doing exactly what I think it should.

    Could there be any specific reason why running a UserAgent "get" inside a thread causes it to store slightly different data compared to calling when not threading?....

    fx, Infinity is Colourless

      Don't make me guess; provide code to produce.

        I can't just paste the entire thing as there's some sensitive stuff in there. However, I am trying to work on reproducing the error with some basic code....

        HOWEVER....that's not exactly working out for me as I can't reproduce it yet! Dammit :(

        I'll paste the code when/if I get it to fail again...

        fx, Infinity is Colourless

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://874381]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others meditating upon the Monastery: (4)
As of 2024-05-26 22:18 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found