Beefy Boxes and Bandwidth Generously Provided by pair Networks
We don't bite newbies here... much
 
PerlMonks  

(dchetlin: beware the unicode beast) Re(2): Number?

by dchetlin (Friar)
on Oct 25, 2000 at 06:41 UTC ( #38278=note: print w/ replies, xml ) Need Help??


in reply to Re: Number?
in thread Number?

<pedantic type=unicode>

Not so fast! Believe it or not, \d matches 178 characters in utf8. [1]

So, while your second two solutions are equivalent, the first one is only correct if you know you're dealing with ASCII.

[1]: Just in case you don't believe me :-)

[~] $ perl -anle'next if/[^\s\da-f]/i;$t+=hex($F[1])-hex($F[0]); END{print$t}' bleadperl/lib/unicode/Is/Digit.pl 178

</pedantic>

-dlc


Comment on (dchetlin: beware the unicode beast) Re(2): Number?
Download Code
RE: (dchetlin: beware the unicode beast) Re(2): Number?
by Fastolfe (Vicar) on Oct 25, 2000 at 19:13 UTC
    I would /expect/ that the regexp \d would match numbers, and numbers only. If 178 items in Unicode match, why? Are these just different ways of representing numbers in other languages perhaps? Perhaps things like 1/2 or 1/4? Or are you generating Unicode that is overly long, with different Unicode representations for the same numbers?

    Does anyone have any explanation for why this would be the case? Does it even violate the assumption that \d matches numbers?

      You're exactly right that they're different ways of representing numbers in other languages. If you'd like to see an example of what such a set of numbers might look like, try here (chosen at random). The digits are in the 5th column from the left, labelled 104, in rows 0 through 9.

      Whether or not having \d match 178 different characters is a good thing depends on the situation. I've been treating the Unicode situation somewhat similar to Y2K -- it's overhyped, but you still need to worry a bit. Any code that might at some point need to be internationalized should be thought through, and idioms like tr/0-9//c discarded.

      Of course, I don't turn utf8 on yet, because the support for Unicode is still immature and shaky, and I'd hate to have a random string be validated as a number just because it contained two bytes next to each other that happened to be 0x1048. Line disciplines will solve that, eventually.

      In sum: I would certainly urge Monks to be early adopters, or at least stay aware of Unicode issues, if for no other reason than to avoid subtle bugs in the future.

      -dlc

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: note [id://38278]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (4)
As of 2014-08-23 09:55 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    The best computer themed movie is:











    Results (173 votes), past polls