Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Comment on

( #3333=superdoc: print w/ replies, xml ) Need Help??

Node display problem: I really wanted to post this earlier, but was having a hard time getting past how PerlMonks obliterate the UTF-8 literal within code tags. In specific, the line starting with my $string = "... should contain a string of the following characters: GREEK SMALL LETTER LAMDA, GREEK SMALL LETTER ALPHA, GREEK SMALL LETTER OXIA, GREEK SMALL LETTER ALPHA WITH VARIA, GREEK SMALL LETTER OMICRON WITH VARIA, GREEK SMALL LETTER FINAL SIGMA. You'll have to paste them into the code yourself (sorry).

In other words, line 21 should look like this: my $string = "λαάὰὸς";, but that can't be displayed within code tags.

I believe the example code below answers both of your questions:

# If using a Perl version prior to v5.16, comment out the "use feature +" line, # and uncomment the BEGIN{...} block. use feature ':5.16'; #BEGIN { # die "Must install Unicode::CaseFold." if ! eval "use Unicode::CaseF +old; 1;"; #} use strict; use warnings FATAL => 'utf8'; use utf8; use charnames ':full'; use Unicode::Normalize qw(NFD NFC); binmode STDOUT, ':encoding(UTF-8)'; my $string = "&#955;&#945;&#8049;&#8048;&#8056;&#962;"; while ( $string =~ m/(?<grapheme>\X)/g ) { my $grapheme = $+{grapheme}; print explain( $+{grapheme} ), "\n"; } sub explain { my $grapheme = shift; my %pri = decompose( $grapheme ); my %base = decompose( $pri{base} ); my $output = <<"END_OUTPUT"; Grapheme:($grapheme) Dec, Hex, Name: [$pri{cp}], [$pri{hex_str}], '$pri{name} +' Case: (Fold,Lower,Upper): ($pri{fc}), ($pri{lc}), ($pri{uc}) Grapheme Base: ($pri{base}), [$base{hex_str}], '$base{n +ame}' END_OUTPUT foreach my $extend ( @{$pri{comb}} ) { my %ext = decompose( $extend ); my $grapheme = fc $ext{grapheme}; $output .= <<"END_OUTPUT"; Combining Mark: ($grapheme ) Dec, Hex, Name: [$ext{cp}], [$ext{hex_str}], '$ext{name}' END_OUTPUT } return $output; } sub decompose { my $grapheme = shift; my $decomp = NFD( $grapheme ); my $cp = ord $grapheme; my ( $base ) = substr($decomp, 0, 1 ); my ( @comb ) = map { substr $decomp, $_, 1 } 1 .. length($decomp)-1; return ( grapheme => $grapheme, cp => $cp, hex_str => sprintf( "%#0.4x", $cp ), name => charnames::viacode( $cp ), lc => lc $grapheme, uc => uc $grapheme, fc => fc $grapheme, base => $base, comb => [ @comb ], ); }

I won't post the output, as the Monastery seems will trash the target graphemes within code tags. For those without the ambition to run it, it will display the grapheme, its code point and name, and then the decomposed base and combining characters graphemes, code points, and names.

The first question you're asking can be accomplished by matching the grapheme cluster with \X, obtaining its code point, and then calling charnames::viacode on it.

The second question you're asking deals with decomposing the grapheme. Unicode::Normalize provides NFD, which is "normalize formed by canonical decomposition". This function decomposes graphemes into their base character, followed by its combining marks. It places them into a reliable order too. substr and length will treat a decomposed string as being of a length equal to all the base characters plus all the combining marks.

If the goal is to just do a comparison of the base code-points, you should probably be using Unicode::Collate, at level 1: "alphabetic ordering". The next higher level provides "diacritic ordering", followed by "case ordering" (which combines the previous levels), and finally "tie-breaking".


In reply to Re: getting Unicode character names from string by davido
in thread getting Unicode character names from string by csthflk

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • Outside of code tags, you may need to use entities for some characters:
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.
  • Log In?

    What's my password?
    Create A New User
    and the web crawler heard nothing...

    How do I use this? | Other CB clients
    Other Users?
    Others pondering the Monastery: (7)
    As of 2014-07-12 15:21 GMT
    Find Nodes?
      Voting Booth?

      When choosing user names for websites, I prefer to use:

      Results (240 votes), past polls