Re: Decoding @ARGV [Was: uparse - Parse Unicode strings]

[kcott]

[A follow-up to "Re: Decoding @ARGV [Was: uparse - Parse Unicode strings]".]

I'm not going to have sufficient spare time to do all that I wanted this weekend. I have managed to incorporate your changes and do a couple of other minor things.

When prefixing the uparse command with PERL_UNICODE=A or PERL_UNICODE=SDAL, I just get "Wide character at ..." and no other output. I made these changes:

• Added changes from your patch.
• Changed "use open IO ..." to "use open OUT ...".
• Modified the code layout (mostly to avoid wrapping in PM).

Here's the new code:

#!/usr/bin/env perl

BEGIN {
    if ($] < 5.007003) {
        warn "$0 requires Perl v5.7.3 or later.\n";
        exit;
    }

    unless (@ARGV) {
        warn "Usage: $0 string [string ...]\n";
        exit;
    }
}

use 5.007003;
use strict;
use warnings;
use open OUT => qw{:encoding(UTF-8) :std};
use constant {
    SEP1 => '=' x 60 . "\n",
    SEP2 => '-' x 60 . "\n",
    FMT => "%s\tU+%-6X %s\n",
    NO_PRINT => "\N{REPLACEMENT CHARACTER}",
};

use Encode qw{decode is_utf8};
use Unicode::UCD 'charinfo';

for my $raw_str (@ARGV) {
    my $str = is_utf8($raw_str)
            ? $raw_str
            : decode('UTF-8', $raw_str);
    print "\n", SEP1;
    print "String: '$str'\n";
    print SEP1;

    for my $char (split //, $str) {
        my $code_point = ord $char;
        my $char_info = charinfo($code_point);

        if (! defined $char_info) {
            $char_info->{name}
                = "<unknown> Perl $^V supports Unicode "
                . Unicode::UCD::UnicodeVersion();
        }

        printf FMT, ($char =~ /^\p{Print}$/ ? $char : NO_PRINT),
                    $code_point, $char_info->{name};
    }

    print SEP2;
}

Here's a test run with just uparse:

$ uparse 👮🏼 👮🏼‍♀️ 👮🏼‍♂️

============================================================
String: '👮🏼'
============================================================
👮      U+1F46E  POLICE OFFICER
🏼      U+1F3FC  EMOJI MODIFIER FITZPATRICK TYPE-3
------------------------------------------------------------

============================================================
String: '👮🏼‍♀️'
============================================================
👮      U+1F46E  POLICE OFFICER
🏼      U+1F3FC  EMOJI MODIFIER FITZPATRICK TYPE-3
        U+200D   ZERO WIDTH JOINER
♀       U+2640   FEMALE SIGN
        U+FE0F   VARIATION SELECTOR-16
------------------------------------------------------------

============================================================
String: '👮🏼‍♂️'
============================================================
👮      U+1F46E  POLICE OFFICER
🏼      U+1F3FC  EMOJI MODIFIER FITZPATRICK TYPE-3
        U+200D   ZERO WIDTH JOINER
♂       U+2642   MALE SIGN
        U+FE0F   VARIATION SELECTOR-16
------------------------------------------------------------

And again, this time with PERL_UNICODE=A:

$ PERL_UNICODE=A uparse 👮🏼 👮🏼‍♀️ 👮🏼‍♂️

============================================================
String: '👮🏼'
============================================================
👮      U+1F46E  POLICE OFFICER
🏼      U+1F3FC  EMOJI MODIFIER FITZPATRICK TYPE-3
------------------------------------------------------------

============================================================
String: '👮🏼‍♀️'
============================================================
👮      U+1F46E  POLICE OFFICER
🏼      U+1F3FC  EMOJI MODIFIER FITZPATRICK TYPE-3
        U+200D   ZERO WIDTH JOINER
♀       U+2640   FEMALE SIGN
        U+FE0F   VARIATION SELECTOR-16
------------------------------------------------------------

============================================================
String: '👮🏼‍♂️'
============================================================
👮      U+1F46E  POLICE OFFICER
🏼      U+1F3FC  EMOJI MODIFIER FITZPATRICK TYPE-3
        U+200D   ZERO WIDTH JOINER
♂       U+2642   MALE SIGN
        U+FE0F   VARIATION SELECTOR-16
------------------------------------------------------------

Using "PERL_UNICODE=SDAL" gives the same output as "PERL_UNICODE=A".