Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re^3: Unknown charnames when building Encode

by kcott (Bishop)
on Jan 20, 2017 at 15:08 UTC ( #1180010=note: print w/replies, xml ) Need Help??


in reply to Re^2: Unknown charnames when building Encode
in thread Unknown charnames when building Encode

Firstly, it seems I gave you something of a bum steer regarding &Unicode::UCD::charprops_all. I've been using it for a while and forgot that it was a fairly recent function (in terms of Perl versions). It was added in v5.22.0, along with some other functions, so won't be available on any of the Perl versions you're using. Sorry about that. See perl5220delta: Updated Modules and Pragmata.

[Just a quick note on markup. While it's generally preferable to use '<code>' tags for code and data, which you've been doing, this doesn't work too well with Unicode characters (outside the ASCII range). In these cases, '<pre>' tags work better: for instance, you'll see 'α' instead of '&#945;'. For inline text, such as in paragraphs, '<tt>' tags serve the same purpose.]

As I have neither Ubuntu nor AIX, I can't effectively reproduce your results. However, I looked into this a bit further and have a few other suggestions.

As you successfully printed the characters from the codepoints:

$ perl -C -E 'say "\x{3b1} - \x{df} - \x{a3}"'
α -  - 

[I didn't need it, but you may need to add -Mutf8 to get rid of the "Wide character" message you're seeing.]

See what names you get for those characters:

$ perl -Mcharnames=:full -E 'say charnames::viacode($_) for (0x3b1, 0x +df, 0xa3)' GREEK SMALL LETTER ALPHA LATIN SMALL LETTER SHARP S POUND SIGN

In terms of what you're referring to as "aliases", I suspect there's a very large number of these. Have a look at charnames; in particular, read what it says about :full, :loose and :short. There's a link to the algorithm for :loose matching, but it's horribly broken: it should be "http://www.unicode.org/reports/tr44/#Matching_Names". How :short is determined, is explained on the charnames page.

:full is fairly straighforward:

$ perl -C -E 'say "\N{GREEK SMALL LETTER ALPHA}"'
α

Based on that #Matching_Names algorithm, I then tried:

$ perl -C -E 'say "\N{greek small letter alpha}"' Unknown charname 'greek small letter alpha' at -e line 1, within strin +g Execution of -e aborted due to compilation errors.

However, when I specified -Mcharnames=:loose, it worked:

$ perl -Mcharnames=:loose -C -E 'say "\N{greek small letter alpha}"'
α

Bearing in mind the :loose algorithm, you can see there's a huge number of possibilities. Here's a few examples:

$ perl -Mcharnames=:loose -C -E 'say "\N{GREEK_SMALL_LETTER_ALPHA}"'
α
$ perl -Mcharnames=:loose -C -E 'say "\N{GREEK-SMALL-LETTER-ALPHA}"'
α
$ perl -Mcharnames=:loose -C -E 'say "\N{GREEK-SMALL_LETTER-ALPHA}"'
α
$ perl -Mcharnames=:loose -C -E 'say "\N{greek_small_letter_alpha}"'
α
$ perl -Mcharnames=:loose -C -E 'say "\N{greek-small_letter-alpha}"'
α
$ perl -Mcharnames=:loose -C -E 'say "\N{greek small-letter alpha}"'
α
$ perl -Mcharnames=:loose -C -E 'say "\N{GrEeK SmAlL-LeTtEr aLpHa}"'
α

Now, as shown in my earlier post, I was able to use the :short forms directly:

$ perl -C -E 'say "\N{greek:alpha}"'
α
$ perl -Mcharnames=greek -C -E 'say "\N{alpha}"'
α

They didn't work for you, but maybe these might:

$ perl -Mcharnames=:short -C -E 'say "\N{greek:alpha}"'
α
$ perl -Mcharnames=:short,greek -C -E 'say "\N{alpha}"'
α

I also had a brief look at the source code for charnames.pm and _charnames.pm; although, I didn't delve into them too deeply. There's a lot of (non-POD) comments that may be of interest. Perhaps have a look at those for the versions you're using.

— Ken

Replies are listed 'Best First'.
Re^4: Unknown charnames when building Encode
by yulivee07 (Sexton) on Jan 23, 2017 at 12:39 UTC
    Ken,

    thank you very much for your input. I found the source of my problems:

    Name.pl is missing in AIX:
    perl@t72:/usr/opt/perl5/lib/5.20.1/unicore $ ls Blocks.txt Decomposition.pl Name.pm SpecialCas +ing.txt UCD.pl CombiningClass.pl Heavy.pl NamedSequences.txt To + lib


    compare that to my linux distribution:
    perl@pod-racer:/usr/share/perl/5.18/unicore $ ls Blocks.txt Decomposition.pl lib Name.pl Spec +ialCasing.txt UCD.pl CombiningClass.pl Heavy.pl NamedSequences.txt Name.pm To + version


    As Name.pl is generated during the compilation of perl, the necessary program mktables was not to be found on the AIX System, as I does not seem to be delivered with a packaged perl.

    Solution to this: Download the perl-Version which is installed from CPAN into a temporary directory. On AIX 7.2 I have perl 5.20, so I download perl 5.20 from cpan (http://www.cpan.org/src/).

    Extract it, and cd to lib/unicore. There you can find the mktables program. Then generate the unicode files:

    chmod 755 mktables ./mktables


    This generates a bunch of files, including Name.pl. I diffed all newly generated files against the ones from my systems perl and found no difference, so I only copied Name.pl to my systems unicore-path. The Errors then vanished.

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://1180010]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (3)
As of 2020-04-10 06:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    The most amusing oxymoron is:
















    Results (49 votes). Check out past polls.

    Notices?