Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

This is neither just a test, nor extreme. This snippet is exactly what every octet oriented Perl function should do.

Doubtless, this will "break" some existing code, as it has clearly done in this case. But believe me, clear breakage with a meaningful error message is SO much better than the vague semi-random encoding trouble you could have gotten without it!

An HTTP message is strictly a stream of octets. It is impossible to have unencoded text, because TCP does not support that. If you would try to use the rejected message over a socket, you would get Perl warnings about "wide characters", a sign that Perl is doing the best it can to force the invalid value into an octet form: it UTF-8 encodes and warns about it. However, Perl first tries to downgrade. If that succeeds, it uses the downgraded string and does not complain. This repairs several bugs caused by sloppy programming.

The snippet that you cited does the same thing: downgrade and use the downgraded form, but if it cannot be downgraded (that is: if the string contains a character with an ordinal value greater than 255) to byte form, complain. The way in which it complains is different from Perls built in. This code DIES. Your gain: you now know exactly that there is a problem, and where it is. Without this new addition to LWP you would probably not have know that there was a problem, and if you had, you would probably still be looking for the source. And if you're like most Perl programmers who don't yet fully understand the octet/text separation logic, you would probably have mis-identified something else as the source, and "fixed" it by making your code forward incompatible. Much like PoCoCli::HTTP did.

All I can say is: be glad, very glad, that LWP got this change. The pain it causes is temporarily, and much less than the pain it prevents.

Juerd # { site => 'juerd.nl', do_not_use => 'spamtrap', perl6_server => 'feather' }


In reply to Re^3: libwww-perl, POE and Unicode by Juerd
in thread libwww-perl, POE and Unicode by OverlordQ

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?
    Username:
    Password:

    What's my password?
    Create A New User
    Chatterbox?
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others musing on the Monastery: (8)
    As of 2014-09-01 20:07 GMT
    Sections?
    Information?
    Find Nodes?
    Leftovers?
      Voting Booth?

      My favorite cookbook is:










      Results (17 votes), past polls