http://www.perlmonks.org?node_id=11105386


in reply to Re: Is there some universal Unicode+UTF8 switch?
in thread Is there some universal Unicode+UTF8 switch?

use utf8::all; sounds the most promising, thank you and I'll try it. The reason I didn't use it yet is that the main doc https://perldoc.perl.org/utf8.html doesn't have a single mention of this option - so either you know about utf8::all in advance, or you are out of luck.

The JSON function shortcut decode_json has UTF8 decoding hardcoded to "on". To make it "off" and to avoid double encoding I had to use the full call like JSON->new->utf8(0)->decode($response->content) If utf8::all solves this problem as well, then I can use the function shortcut. I will check everything later today.

(Update) Noop, I rechecked - only the current long code reliably working for non-ASCII. For the sample URL above I do my $response = LWP call and then

  1. my $data1 = JSON->new->utf8(0)->decode($response->content);
  2. my $data2 = decode_json($response->content);
  3. my $data3 = $response->decoded_content;
and then my $test = $data1->{query}->{allusers}[0]->{name};

1) is always working for my needs. 2) is woking if called in some obvious scalar context. One tries slice referenced array or anything complex - it falls to the "Perl branded jam" with and the like. 3) is stably DOA (dead on arrival) so the same 2) but right away.

So utf8::all should be written and extended to some utf8::all_throughout "Written" means to the reliability and stability level to be included in prominent Perl distributions. Until then the answer to my initial question seems negative.

  • Comment on Re^2: Is there some universal Unicode+UTF8 switch?

Replies are listed 'Best First'.
Re^3: Is there some universal Unicode+UTF8 switch?
by daxim (Curate) on Sep 02, 2019 at 10:09 UTC
    slice referenced array or anything complex - it falls to the "Perl branded jam"
    I'm sceptical about that claim. Show your code.